Something else to consider is the amount of moisture in your interior air. In many homes, wintertime air gets overly dry due to central heating systems, fireplaces/woodstoves, and anything with a burning flame. Windows are rarely opened, which prevents outside humidity from replacing what these heat sources dry out.
The other end of the spectrum is too much moisture in the air. If you heat with electric baseboard heaters, radiator-type heaters or the like, then zero moisture is dried out due to heating. In this type of home you can actually have an excess of moisture, which may cause mold. The mold, in more severe instances, could grow to be a year-round problem. However, it is also entirely possible to have seasonal mold. This can happen in exterior walls, due to the buildup of condensation -- cold outside air meeting warmer, interior walls, and moisture condensing within the walls. In the summer, look to the plenum of your central A/C system as a possible problem area, assuming that the ENTIRE plenum area and all of its components are not made of metal. While it's still possible for bad air to grow on all-metal areas, it rarely turns into a problem.
The last wintertime air problem is only a problem when there are newer materials involved; new home construction, new carpet, new upholstered furniture, etc. This is due to off-gassing, or gasses given off during the airing-out period. Since winter air often doesn't get replaced/exchanged, many man-made products can cause off-gassing that some people are very sensitive to. This is why most newer construction in more urban areas now require the utilization of air exchangers to be used.
We live in a HORRIBLY wet area! Even now, with the ground totally frozen to at least 2' already (and deepening by the day!), our basement sump pump is still pumping out well over 1,000 gallons of water every day. This Spring, during the melt, it will pump out up to 7,000 gallons per 24 hours!! Our house has high humidity, yet we have no mold. So how can this be possible, you ask? Because this is a drafty old house in a very windy area. Every time the wind blows (which is MOST of the time!), the inside air is being replaced with clean, fresh outside air.