Why yes it is,read the declaration of independence.
"We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness"
I am aware of this document, believe it or not.

Unfortunately, lines must be drawn.
The issue at hand is public property; private property is a no-brainer. But public property is owned by the public, and the public (through elected officials) establishes the rules under which it may be used by the individual. The reason that we do this is so that my use of it does not infringe upon your rights to use it. As an example, suppose that I began dumping trash and broken appliances at the local park. I'd argue that because I own it, and I'm free to live and pursue my happiness, I'm within my rights to do so. You'd argue (rightfully so) that my doing this is now infringing upon your rights, and I shouldn't be allowed to do this. Thus, we've established that in any society, even rights have limits on them - limits established by respecting the rights of others. As they say, my right to throw a punch ends at your nose.
So back to what I was saying earlier, is escalating a confrontation worth the possibility of having the law changed? I look at it very much like riding a motorcycle: I may have the right of way over that SUV that's going to collide with me, but being on the right side of the law is not going to save my life when we eventually occupy the same position. By getting out of the way, I didn't relinquish my rights...I merely made a decision to back off due to the events that were unfolding. The SUV driver was certainly wrong, but I wasn't willing to collide with it in order to assert myself.
This takes us back to the beginning: is metal detecting a right? On your own property, sure. On someone else's property - including the public's - no, it is not, and that's how it can be banned.
Some may look at this as having a flippant view of the foundational principles of this country, but I don't see it that way at all. To quote Sun Tzu on the subject, "For to win 100 victories in 100 battles is not the acme of skill. To subdue the enemy without fighting is the acme of skill." I can force the issue and probably win the battle, but I risk losing the war so to speak. Or, I can adapt and get what I want without any confrontation whatsoever. I go home with the goodies and Mr. or Ms. Busybody goes home thinking that they sure showed me a thing or two. The only difference at the end of the day was that I got what I wanted, but they didn't, even though they thought that they did. I'm confident enough in myself to let someone walk away thinking that they got over on me, particularly when they actually didn't. After all, I got what I wanted in the end, right?
Does this make sense?