Recently got permission to hit a nice bridge project over the river. Asked the job site foreman. Great old property and we got some silver and nice relics right off the bat. Next day we went back, the company bigwigs were there with the public officials and they gave us a no-go on hunting the site any further.
Yes, liabilities, property and equipment issues, safety and all that were the topics of their concern.
Had we left immediately upon seeing the suits, we'd probably still be working that site. That was the first construction site I've ever worked after getting permission where the circumstances changed.
Some years ago I was hitting the oldest park in our town, it was a fenced in city block being completey dug out and regraded. Walked in with my gear while the guys were breaking for lunch and simply waved, told them I'd like to hit some of the dirt for old coins and such, and the site boss told me go ahead.
Next day the city engineer came by and told me to leave, I told him the site boss was okay with it and he should be also. So he did not push it any further.
On the whole I'd say if you want to hit it, there is usually some way that you can hit some part of it without too much trouble if the crew is fairly friendly.