Your guess is correct. It is the lead "sideplug" from a Confederate "Sideloader" Case-Shot shell. In the photo below, which shows a sawed-in-half "Sideloader" Case-Shot, you can see the sawed-in-half lead sideplug, screwed into the threaded side-loading hole, at approximately the 10 o'clock position of the shell's curved iron body.