The multi-columned building inside the state "map" on the token/charm is definitely the Virginia State Capitol building.
Regarding the other buildings on the token (or charm):
I investigated Creskol's suggestion of University of Virginia buildings, particularly the Rotunda. It does resemble the UVA Rotunda, but it's definitely not that. The UVA Rotunda building's main body (excluding its six-columned porch) is round, not square or rectangular. Another of the token/charm's buildings does closely resemble UVA's Alderman Library - but Alderman was constructed in 1938. So, if that is what the token's building is, the token therefore is post-1938.
I also investigated Rebel-KCG's suggestion that one of the buidings is the White House of the Confederacy. It isn't, because the WHotC's portico (columned porch) has four sets of double columns. (See the photo at the end of this post.)
I see a clue that the token/charm was made long after the end of the civil war. The clue is the shape of the Confederate Battleflag - which was square, not rectangular. Therefore, during the war, that flag was shown (in paintings, etc) as square-shaped. Well-after the war, people who didn't know the Battleflag's actual shape portrayed it as a rectangle (like the US flag). The token's flag is rectangular.
Two sidenotes:
1- The Confederate
Navy's flag, showing the same logo as the Army's battleflag, was rectangular. So in actual fact, the token/charm's flag is the Navy's flag. I don't think that's what the token's maker intended. He probably made the same error that many people today make ...thinking that the rectangular flag was what Lee's army (and Johnston's army, etc) used. Nope. The actual Confederate Battleflag was square.
2- Another frequent 20th-century error: the "Stars-&-Bars" Confederate flag is
not the CS army's Battleflag. The "Stars-&-Bars" flag was the First National Flag of the Confederate Government. It has two horizontal wide red bars, with a wide white bar between them - and at upper left, a blue square containing white stars. Viewed from a distance, it was easily mistaken for the yankee Stars-&-Stripes flag, so it was superceded by the Second National flag (a small square battleflag emblem located at the upper left on an all-white field). To view the different Confederate flags (and info about them), go here:
http://en.wikipedia.org/wiki/Flags_of_the_Confederate_States_of_America