Not a stupid question. . . really
Cellar holes are sort of a generic term for what's left of former home-sites.
Around where I live, cellar hole indications are usually shallow depressions in
the ground with scattered bricks or medium-sized rocks around the edges with
TONS of metal, broken glass, ceramics. . . (and other cool stuff that my wife won't allow
in the house

) strewn around the perifery. . . oh, and also old coins

.
Most of the cellar hole locations I've found were by researching land ownership maps
(Plat Maps) from the 19th-century at my local library (in the Geneology section) and on
the internet. Some of these plats I've looked at even had small squares indicating where
buildings once stood.
Many times I will look at Google Earth imagery and HistoricAerials.com of the public areas I
am allowed to hunt and then check out the older plats of these areas that have homesites
indicated. I can then match up the current Google aerials with these maps.
Another good source are out-of-print USGS Topographic maps from the late 1890's & early 1900's.
Some libraries and/or the county court house may have actual copies or reproductions of these maps
that you can look at or make copies of. These will also have cultural features (man-made structures,
roads, etc.) indicated on them that you may be able to match up with current mapping or aerial
photos.
Researching can be half the fun. . . and when you find a site that has cool stuff, it's even more rewarding.
Good luck & HH!