Huh, if you "have permission", then what's to fear from "attracting attention" ? If any busy-bodies approach you and gripe, you merely whip out "your permission" and presto, they slink away embarassed for having questioned you, right?
Wrong. As you surmise, the "permission" is not able to get you out jams. There's no shortage of stories of persons who "had permission", simply to have it promptly revoked the moment any cop or gardener gets his panties in a wad (because he thinks you'll harm earthworms or leave holes). And if you flash your "permission", he merely gets on his cell-phone, calls city hall and says ... "but he's tearing the place up!!" (which isn't true, of course). And then guess what happens to your "permission" ? Hence your comment about STILL using a little .... uh .... "discretion" of timing, is very accurate.
But what would you have done if whomever you'd asked for permission had said "no" ? And what would you'd have followed that up with "where is that written", and if they could cite no laws that said any such thing? And if you later learned that others detect there with no problems? At some point, would you realize that the "no" is rather arbitrary, and you've done nothing more than preclude yourself from a hunting area? And if you planned (as you say) on going at low traffic times so as not to arouse busy-bodies to begin with, why not just look up the rules for yourself, and as you say, avoid such busy-bodies?