Already a couple of cases where people have done just that. My understanding is so far in the one case that was settled the owner of the drone received full compensation for its destruction.
So far I've seen the Google video car twice. They use this the video of streets/homes and locations to incorporate it into their mapping program. So what would happen if you saw the Google car with the camera equipment on the roof going by your private property or let's say going over your claim on BLM land? I think we all know if you took a shot at the car or the camera equipment on top, you'd be behind bars pretty quick.
This is not to say that there are not a lot of knuckleheads out there that shouldn't be using a drone. But, I know we can all say the same thing about people driving their own cars. That doesn't mean a somebody cuts me off on the highway I to take a shot at them.
Many major cities especially in Europe and England have cameras on almost every street corner and every subway station! So if you don't want somebody filming what you're doing out in the public, do it in private!