Connecting a pot to a motor will vary the speed, but you're burning up lots of energy in the potentiometer. If you get to a sizable motor (even a 1 amp draw is getting sizeable) you'll be looking at tons of energy waste.
The best way to control the speed of a motor is with an H bridge and a PWM signal. The H bridge acts as a switch that can turn power to the motor on and off, as well as reverse direction. The PWM signal commands the speed that the switches turn on and off, determining the speed. PWM runs at thousands of cycles per second, so the motor runs smoothly. For example, if you turn the switches on for 25% of the time, off for 75%, the motor will run at 25% speed.
H bridges and motor controllers can be found at many electronics stores, including ours.