As you guessed, your find is indeed a cannonball fragment. Specifically, it is from a civil war 12-pounder caliber (4.62") Bormann-fuzed cannonball. For me to tell you whether it is a a yankee one or a Confederate one, I'll need you to clean the encrustation out of the fuzehole's threading, and post a couple more close-up photos of the threading.