orviskid, yes you can do a plug-&-replace, without evidence of your doing. Here's a few tips:
1) don't go turf hunting in the middle of hot-dry seasons (middle of summer, etc...) because yes, the turf will have a harder time staying alive, and staying in-tact, if the ground is dry and crumbly (a plug will fall apart, roots die, etc... if hard and dry ground). So limit your turf hunting to when the ground is nice and moist.
2) Cut your plug deep, with the entire roots, and the ground still in-tact. Ie.: not just a "plug" of the grass only, but preferably with the entire root and dirt bottom in-tact.
3) Turn it upside down, once removed, to limit the crumbs and dirt getting on to the grass.
4) Use a catchmat for subsequent dirt removal from the hole. A good way to make a home-made catchmat, is to take a truck innertube (can find them in the dumpster behind large-truck truck-tire places). Cut a large-dinner-plate sized section from the portion where it curves to form a concave shape (like a contact lense type shape, yet dinner plate sized). These work great because they roll/fold up tight between uses, pop back out to full size easily, & resist dirt and moisture (ie.: don't get muddy like trying to use a towel would do).
5) When done, repack all the soil and re-set the plug. And.....
6) Very Important: Stomp on the plug to make sure it is level with the ground, & tightly repacked. I see too many people just put the soil and plug back in, then walk away. What happens is the plug sticks up a bit, and soil is not smunched to repack with the existing soil. The plug tends to be more visible, stands up to get kicked back out, etc... So be sure to stomp it for firm reconnection!
7) ruffle up any remaining crumbs with your fingers (in a raking/fluffing-up fashion).