Sorry to have to disappoint, but it is definitely not a civil war artillery fuze... nor from a grenade or landmine, etc. A threaded metal fuze typically had some kind of provision (such as slots or a hexagonal collar) for a fuze-wrench to grip the fuze firmly enough to screw it TIGHTLY into the shell or grenade or landmine. There seems to be nothing like that on this object. You say its other side has a square indentation, but I don't see it in any of the photos you've posted. There was a civil war era British artillery shell "shipping plug" which had a square indentation for a tool to screw it into the shell, but its underside did not have a round depression in it, like this object has. I cannot identify the threaded plug you found, but I'm 100%-certain it is not from a civil war era explosive device.
Edit-note:
As the timestamps on DCMatt's post and mine show, I didn't get to see his until after I posted mine. The square projection on the screw-in plug he posted shows what I meant about the need to have something for a tool to grip for screwing the plug (or fuze) tightly into whatever it goes into.
