Actually, the output changes as fast as it can (slew rate), but it can't go beyond the supply rails (and usually to a voltage less than the rail voltage -- e.g. if the op-amp's supply lines are connected to a +/-15 supply, then a typical case is that it can only swing to about +13.5V and -13.5V).
But, in an ideal case, where the op-amp is powered by +/- infinity voltage, your scenario would be correct. But, you have to pay attention to polarities. In other words, if the op-amp has a gain of 1000 and there is 1V on the inverting input and 0 on the non-inverting input, then the output would go to -1000V as fast as it can. If there is 0 on the inverting input and 1V on the non-inverting input, then the output would go to +1000V. Likewise, if there is, say, -0.5V on the inverting input and +0.5V on the non-inverting input, then the output would be +1000V. In other words, it's the difference voltage that is amplified.
In the real-world case of an op-amp with a gain of 1000 and powered by a +/- 15V supply with outputs that can only swing to 1.5V less than the rails (e.g. +13.5V and -13.5V) then for the case where there is 1V on the inverting input and 0 on the non-inverting input, the output would try to go to -1000V, but will stop at -13.5V, because that is as far as it can go.
Now, the way feedback works is say you have an Rf of 2K and an Ri of 1K (where Rf is the feedback resistor from the op-amp output to the op-amp inverting input and Ri is the input resistor, also connected to the inverting input), and the non-inverting input is tied to ground (which, in this case, is zero volts). If you apply 1V to Ri (on the side that isn't tied to the inverting input), then the instantaneous (i.e. at the length of time it takes to overcome input capacitance) result is that the inverting input of the op-amp goes to 1V, causing a 1V difference between the inverting and non-inverting inputs. Because the inverting input is more positive than the non-inverting input, the output is going to drive negative. It's going to try to go to -1000V. On it's way there, it will reach -2V. When it does, the inverting input will be at 0V and thus there will no longer be a voltage difference between the inputs and the op-amp's output will stop at -2V.
That's the over-simplified view. Lets look at it in more detail. Lets say we start with zero volts on Ri. Thus, after a brief period of time, the output of the op-amp will at at zero (in the best case where the op-amp is offset adjusted and such). That is our initial condition. Then, lets say we apply -1V to Ri (again, to the side of Ri that is not connected to the inverting input). This will instantaneously apply, -1V to the inverting input. Let's say the slew rate of the op-amp is 1volt/microsecond.
Then at 0.0uS, the output will be at 0V, and the op-amp output will begin trying to get to +1000V.
At 1.0uS, the output will be at 1V, which will raise the inverting input to -0.33V (through the resistor divider network). Because the non-inverting input is tied to zero, the input difference is now -0.33V which will cause the op-amp to try to get to 333V (thus it will continue to slew in the positive direction),
At 1.5uS, the output will be at 1.5V, and the inverting input will be at -0.17V and the op-amp output will be trying to get to 170V.
At 1.9uS, the output will be at 1.9V, and the inverting input will be at -.033V and the output will be trying to get to 33V.
At 1.998uS, the output will be at 1.998V, and the inverting input will be at -0.002V and the output will be trying to get to 2V – 0.002V or 1.998V...wait, it's already there. Thus, the op-amp has found stability.
Now, as Earl pointed out, the output will probably overshoot this point and swing a little bit beyond 1.998V and then pull back and overshoot it a bit less in the negative direction and then back positive, etc, until it settles down -- thus "ringing".
This also illustrates the importance of gain. Notice how the output is not at exactly 2.000... Volts. Less than infinite gain is a source of error. If the gain of this op-amp were 10000, then the output would stabilize at 1.9998V (i.e. yet closer to the ideal 2.000...Volts).