Thanks for the comments guys. I have contacted the Department of Interior historian to try and find the original owner if he is still alive. There is a number on the back and I provided that to the historian. If the owner is alive or has family, I'll try to return it. I was on private land, with permission, where a public road used to pass through, but it is now wooded and hard to pass on foot.
The other items are a heel plate, a Whitworth, a brass clip for horse tack (my guess), a Mississippi Tax token, a copper coin or plantation token I haven't identified yet, and a button (guess). Any help will be on identification will be greatly appreciated.