I looked at pictures of rivets online and they did not have the raised bump like in picture 3 and they were shaped differently. Maybe a older rivet if one?
Rivets are a permanent mechanical fastener. By definition they do not come apart.
As stated above, they are fairly common. I dug 4 similar to yours last sunday at a 1750's house. One of mine had the "bump". Two of them still had leather attached.