I'm afraid that it boils down something much more base.
Namely, power.
It's ALL about some people who wish to have their will more important than those around them.
Sure, there are those who leave holes, who are belligerent in the field when dealing with people, sneak onto spots without permission, hunt by the dark of night so they won't be seen, on and on.
If everyone who took up this hobby acted like this, then there might be some justification for them to treat us like vermin. But, the truth is, the MAJORITY of us are not like that and are good citizens. The concept that is being used by the "powers that be" are to take the easiest route. Easiest, for them. Just throw the baby out with the bath water.
This same concept could be used for a LOT of other things. Golfer's and their divots, tennis players and their empty ball cans and/or used tennis balls left everywhere, any cleated shoe sport and the way they tear up the grass and dont put it back, picnickers who just ball up their foil and toss it on the ground, beach goers who bury the used beer cans in the sand, kids walking to school tossing their wrappers everywhere, etc, etc, etc.
What would happen if we started calling the "powers that be" every single time we see one of those things? Would they close beaches, parks, Maybe even eliminate walking to school, etc?
I'd say, if they want to start eliminating things, let's start our own campaign and see just how far they're willing to go!
Heck, with that attitude, if one single person runs a stop sign, then the ONLY thing to do is to eliminate driving cars! Problem solved!
On the other side of the coin, this new school where I've been going recently, had a groundskeeper come over to me and said I was watching to see if you were refilling your holes. Since I was treating his grounds with respect, he had no problem. He even commented that he checked the next day or two and couldn't find but one or two spots where he could even tell I dug. (even the best of us wind up with an ugly hole now and then

).
We ARE being watched, so, act like it and show some respect!