I typed the following reply several hours ago but my computer was acting strangely and I had to wait for my wife to remedy the problem before I could post.
-----------------
Not a clothing-button. Note that the projection on the back looks like a square-nail. You did not tell us the disc's size, so I cannot tell whether it is a luggage "foot", or a trunk skidder, or a furniture-upholstery decorative tack (some were much larger than today's tacks).
In view of the two purely "decorative" circular grooves on its front, it is unlikely to be a carpetbag foot/skidder.
Here are two photos showing to show you what a trunk skidder is. The skidder is somewhat domed, to enable the trunk to slide easily across a hard flat floor. A carpetbag foot is basically the same thing. Many modern-era chairs also have a (steel) "skidder" disc on the bottom of each leg. That being said, your one-piece cast heavy brass luggage/trunk-skidder is from the 1700s or 1800s.