ArcMan beat me to it while i was writing but posting anyway...
Lets say a voltage regulator provides 5V @500mA output .
sorry to be picky but there is a problem with your opening statement.
the voltage regulator you are talking about can provide up to
500mA @ 5V.
this regulator will always provide 5V.
how much current is drawn from it depends what you are connecting to it.
the maximum load of the regulator is 500mA but it is perfectly acceptable to draw less from it.
it's important to understand this bit if you are going to understand the next bit.
No, it will always draw more. It depends on the input voltage, too.
nope. not true.
in a typical linear regulator the current draw of the regulator is around the same as the current draw of the components drawing current from it.
i say "almost". the regulators Datasheet will tell you what the additional quiescent current loss is. it is safe to ignore this small amount for the sake of this discussion.
so, in answer to Airman's original question,
if you are using a linear regulator you can presume that the current drawn is the same all the way through.
for your example where you draw 25 mA from a regulator, it will draw 25 mA from it's source.
if you connect nothing to the output of your regulator it will draw nothing from your battery. (apart from the afore mentioned quiescent current...)
it's worth noting that if you connect a 5V regulator to a 10V supply and draw 25 mA from the regulator the regulator is providing 0.125Watts of power but it is using 0.250Watts of power so it is only being 50% efficient.
(Power = Volts * Amps)
obviously for other kinds of regulators things are different.