Rust comes from oxygen mixing with iron. What this means is that only iron rusts, strictly speaking. Copper, silver and aluminium (among many others) tarnish. It usually - but not always - comes out to the same thing as iron rusting, so you end up with copper oxide and aluminium oxide. (Silver actually reacts with atmospheric hydrogen sulfide when it tarnishes.
Gold doesn't even do that. What's called tarnishing in gold is really just discoloration and isn't the result of gold reacting with oxygen in the environment. This is because gold is a pretty unreactive element - you need to start throwing a lot of energy at gold before it will start reacting with anything in the atmosphere. For example, while iron oxide can exist at room temperature, gold (III) oxide decomposes (turns back to gold and oxygen) at 160 degrees Celsius - and that's the most stable gold oxide we've found.
So what does react with gold? Mostly halogens. Halogens react with pretty much anything - fluorine is particularly nasty in this regard, but chlorine, bromine, iodine, astatine and ununseptium also react with other substances quite readily.
(Two things to mention with regards to that last sentence:
Astatine and ununseptium are radioactively unstable, which means that they're not likely to stick around long enough for you to really pull off chemical reactions.
This does not mean you have to worry about the fluoride in the water, the chlorine in the pool or the iodine in salt. What you're seeing there are either previously reacted elements - so that you're just ignoring part of the chemical name - or ions that aren't very reactive in and of themselves.)
Gold also reacts with acids. However, it doesn't react with oxygen, nitrogen, or really any of the gases present in the atmosphere. Which is why you won't see gold "rust." If we lived on a planet that stayed 170 degrees Celsius, then we might see gold "rust" - but we'd also die pretty soon after that.