To comment on several people's answers so far:
I do not understand what go-go-gopher is talking about, when he says public libraries are on private land. All the states I've been at, a library is public, including the land it sits on, assuming we're talking a "public" library, that is.
As per Charlie P's in-put, why does an acitivity need to be "expressly allowed", or else it's deemed illegal? I mean, for example: If I find no express allowance to whistle dixie in the park, then am I to assume that whistling dixie is illegal? No, of course not, so I don't get your statement. Yes there IS laws that forbid spray-painting graffiti on city hall. It would be seen as the vandalism, defacement, etc.... clauses in any city code. And no, I do not interpret those clauses to apply to md'ing. Because technically, if you do not leave any trace of your presence, then technically, you have not vandalized, or defaced anything, now have you? To think otherwise (that your hobby, by automatic equivalence = "vandalism", "destruction", "alteration", "defacement", etc....) is to have lost the battle already. If you really equate yourself to those destructive phrases, then stick to private property, or sandboxes, or choose another hobby. Because I gaurantee you, if you walk into any city hall and said "Hi, is it ok if I vandalize and tear up the park?", what do you think they're going to say? So on the contrary, if you're leaving no trace of your presence, then technically, those things would not apply to you.
Now is that to say that some busy-body might not agree with those semantics?? SURE! This is why we need to be a little more frickin discreet, like dfxone says, and go at low traffic times (when the library is closed). Because, I mean, Let's face it: we're in an odd-hobby that draws the stares of the curious on-lookers. It has "connotations" (that you might leave a mess), so for pete's sake, why would you go in high traffic times, which simply begs for attention and kill-joys?
Good-joe, as for calling the parks & rec. dept. after this encounter, did the person who told you "no", cite you any such actual rule or code, or wording to that effect? Because I bet it's just her simply morphing something else to apply to your "pressing question". Like .... she simply envisioned geeks with shovels, when truth be told, I bet it would never have occured to her, before you asked. Ie.: I bet if that same lady had been simply driving down the street, and seen a man with a detector in a park, she'd have never paid a second notice. This psychology of "no one cared, till you asked", happens over and over with this topic. NO public official will typically say "sure, go ahead". Because the mere fact you are standing there asking them (especially when you probably started the conversation by saying you'd been booted by a duly appointed city person, who may have been her friend or whatever), simply implies that something must be wrong with your activity, that it needed their sanction, to begin with (lest why would you be asking?). That subconscious inference is not lost on the person you're asking, and they will bestow on you their princely "no" (afterall, you asked). So unless she cited something specific that actually says "no metal detecting", I bet you simply got someone's personal whims. And the trouble now is, if you try to fight it, you may just end up getting a law WRITTEN (or official policy anyhow), from here on out. You'd have been better to look up the muni-codes for yourself (usually on city websites you can find park rules like "no dogs allowed", "Parks close at sunset", and so forth). If you saw nothing specific there, and it's silent on the subject, presto, there's your answer.
As I say, this still doesn't mean you go waltz nilly willy at high traffic times over people's beach blankets. Think of it a little like nose-picking: There may be no rules that forbid it, but if you asked enough people, you will eventually find someone who says you can't do it. So ........ you be a little discreet, and odds are, no one really cared.
I have been booted from parks in various cities before, and I do not take isolated incidents like that to mean there is actually really a rule. If I have a doubt, I can look it up for myself. Otherwise I just avoid *just* that one person's shift, from then on. I mean, good-joe, you said yourself that in the past, authority (cops, workers, etc...) have passed you by, and no one's ever cared. So to me, it sounds like you should have just avoided *just* this one lookie-lou in the future. Her hours would be easy to predict by looking at the library hours sign.