I'd like to know how to reliably check the voltage that these motors are? The car ran off of 4 1.5V AA batteries so I'm guessing maybe a 5v and a 3v? Is there any way to test them out without risking burning out the motors or the power supply I connect to them?
If your car is of the cheap(-ish) variety, the steering motor may just be a regular DC motor driven to some end stops (I've seen that in a couple of cheap R/C toy cars). Some have a poor quality potentiometer feedback an ca thus be called (crude) servos, but the motor can be used without the feedback pot if needed.
The drive voltage of a DC motor is not cast in stone and so they work over a range of voltages, from where they just move to where they get too hot.
Testing is easy, if you've got access to a variable supply (lab supply).
First run it at a moderate speed - perhaps give it 3V..4V for 5..10 minutes and feel the temperature of the motor housing. As long as it stays under 40°C it can handle it just fine. Then up the speed/voltage a little, run for another 5..10 minutes and check the temp. Repeat until fairly hot to touch, but not burning that should be around 40°C (If you have an IR thermometer it will be more precise of course).
A motor can be run above 40°C, but for every 10°C, the life is approximately halved and there's n upper limit depending on the magnets in the motor and the insulation of the windings inside the motor, so don't overdo it. For most instances, 50°C should be an acceptable max.
Bear in mind that this will only find the allowed max. voltage of an unloaded motor (unless you load it while testing) and the motor will get hotter much faster when it does actual work.
If this is too much bother, just run them with a similar 6V battery (in the target application) and check/feel the motors temp every now and then, when used as normal (for your app.) - A steering motor is not used very much, compared to a drive motor.
It's probably a little out of your league for now, but in critical applications, it's customary to install temperature monitoring, so the motors can be driven fairly hard and then slowed (or shut down completely) when getting above a set temperature.
BTW. A tiny drop of light oil (a Teflon loaded oil like eg. Super LubeTM
is ace here) in each axle bushing really does wonders for such motors if they are mechanically noisy - wipe off any excess, it will only draw dust and grime.