Most sites we've been on insist on breathable ground sheets if you are going to use them. I like the idea of not using one at all though. However we've used either one or the other for years and not caused any damage to the grass yet.
I suspect the issue for site owners is when they are busy and the grass doesn't get chance to recover inberween visitors.
If the site is dry enough, then why not leave it? The problem is few are.
In our first year of caravanning we always used the groundsheet, in our second year we hardly used an awning because we only went away for weekends. This year we've used the awning but not bothered with groundsheet at all, just seems like extra work to me.