Switching frequency affecting needed duty cycle? – attiny85

I am driving an n-MOSFET directly from an attiny85. It is switching fairly high-power, but the MOSFET has been carefully chosen to handle this. I have been adjusting the switching frequency to try to minimize switching losses, which are my main source of heating in the FET (very low RDS_on). I originally planned on switching at 32khz because it is easily achievable, and it's well above audible frequency. The attiny85 nominally produces a 490hz pwm signal. 
My question is this: to achieve my target power to load at 32khz, the signal is about 80% positive duty cycle. However, when I reduce my frequency to 490hz, the same power is achieved at about 20% duty cycle. Why is this?I suspect it may have to do with a decreased gate drive current at increasing frequencies, but I can't seem to find information about this anywhere in the datasheet or online. My other idea was that I was coming closer to saturation in a large inductor that is present in the circuit, but the same effect is present when I change the inductance slightly. I would like to switch at a higher frequency because logically it should reduce losses, and for non-efficiency reasons, become inaudible.

Firstly, what mosfet are you using?  I assume it is  logic compatible, and is thus  fully enhanced by the logic high from the micro.Secondly, low RDSon devices have very high gate capacitance, both static and dynamic. This must be charged and discharged rapidly to maintain high switching speeds, .The transient current required to drive the gate can be approximated from the data sheets, as can the discharge current. In both cases, a very low  output impedance driver is needed to handle the current, a micro port does not normally have that sort of capability, you can be talking about transient currents of hundreds of mA and greater as a minimum.There is a technical document here,  if you are up to the maths and physics, if you do a search on mosfet drivers you should find a lot more.Driving inductive loads is not without its problems,  If you are switching an inductor at speed, the reverse voltage is conventionally clamped  with a silicon diode, this slows the turn off, and other means must be used, without knowing what you are doing  I can't comment further,cheers,Richard

Richard has covered most of the bases here, but re your difference in duty cycle between the frequencies....  Without a driver you may find your switching times are quite high so at 32KHz you're getting very triangular drive pulses.  Some scope traces might be useful.

That document link is really handy Mr. Gabric, thanks!