It appears to be an early Navy button with an anchor inside a shield on the lower rigt side of the button. I believe it to be an NA 74. See Albert's - Record of American Uniform Buttons - NA 74 navy 1-piece on a lined field without stars. I cannot see a backmark, but there may be one. An Albert's NA 74 would make it an early 1800s button around the time of the War of 1812.
An exceptional find. I've seen these dug in Virginia and other coastal areas.
Possibly my eyesight is not a sharp these days, as I'm not able to see a clearly defined anchor in the shield on this example.
It almost looks like something might be there, but then the mind can play tricks as well, when one studies for awhile at one thing.
Here is an example of an 1840's U.S. military forage cap button, with a plain shield. These are noted in both one-piece and two-piece styles, and can date from the 1830's to the 1850's.
There are also similar navy buttons, with an anchor design in the shield, from about the 1820's period. This particular style of button is post-War of 1812.
I am going to retract my guess on this being an early Naval button, and agree with CC Hunter.
I was not familiar with the forage cap button (I'd never seen one until today).
The picture of the cleaned button that you added clearly shows the detail of the shield, which is identical to a forage cap button. The shield on an early Navy button is a little bit different design.