Counterfeit coins are very hard for someone to detect if they are good. Indeed high-end fakes of gold coins have been made that use the exact alloy of gold of where the gold was mined and nearly flawless dies!
When it comes to silver coins, often a silver coin will not feel right, it will feel slippery and odd. The magnet test will tell you for sure if you have a fake (if a silver dollar is attracted to a magnet at all it is for sure a fake) but the vast majority of fakes I've dealt with are either silver plated copper or copper-nickel and will not stick to a magnet. Just because a coin won't stick to a magnet doesn't mean its real!
A lot of it comes from simply handling a bunch of coins rather than any particular "flaw" that is easy to describe, the coin just looks "wrong"
Another test that can be used is the "ring test" silver has a particular ring to it, hold a known good silver dollar at the edge of your finger and tap it with something like a pencil eraser, it should make a bell-like tone. Next, do the same thing to something that isn't silver such as an Eisenhower dollar and do the same thing, it should make a "plink" or a thud sound. Then do it to the suspect coin, if its silver it should sound like the known silver coin. Keep in mind you must do this GENTLY, you don't want to scratch up a potentially valuable coin!