The specific form of ball-button you found dates anytime from about 1830 through today. Some were worn on Militia uniforms before, during, and after the civil war... but they were also used on Civilian clothing (particularly, women's clothing), not just Military uniforms. Many military-schools used them on cadet uniforms also. For example, the Virginia Military Institute still has them on cadet uniforms today.
Unfortunately, your button has no backmark (maker or dealer's mark), so there's no way to time-date it with absolute certainty. However, because these ball-buttons drastically declined in popularity by the latter-1800s, AND you found yours at a spot where you've been digging 1870s-90s buttons, yours is most probably from the 1830s through the 1870s.
As a fellow relic-digger, I wish I could tell you your ball-button is from a civil war uniform... but it could just as easily be a civilian-used one.