Yes, there are probably a decent amount. There's already a massive portion who enjoy the outdoors, so doesn't seem unlikely that a good amount would go down the geology -> minerology -> prospecting pipeline. It's not weird to be curious, but for safety reasons it is likely that we will either stick to where there's many people, or none at all (in response to your statement about not seeing many in the wild).
I don't own a detector or a gun, but will likely invest in both soon. I like prospecting! It's fun and good exercise in the outdoors, what could be better?
Just the other day I was deep in the Bradshaws of Arizona, and came across some old diggings. I wish I had poked around more attentively, because later I was looking at the photos and missed a sign hanging above the mine (although it looked very old and the color was washed out of it). Found cool pieces of history - an old campfire ring of stone, an old dirt road (long overgrown), a rusty can. Also chipped away at the diggings myself and found/collected some excellent pyrite specimens running thick in the quartz with copper, galena and other mineralization. Anyway, getting off topic. So, hopefully that answers some of your questions. I'm only a novice, but there are many far more seasoned/experienced ladies out there I'm sure.