The big question is 'build or buy'.
If you want to build all of the electronics then the price will depend on whether you have all the necessary tools like soldering iron, solder, multi-meter etc. These are a one-off cost but may break your budget before you even get started adding the cost of strip board, chips, resistors, capacitors, header pins etc.
Assuming you don't have the required tools then get a pre-built board. If you are looking at < $50 then something like an Arduino, Roboduino, or a Baby Orangutan B328.
The new kid on the block, from a SoR member, is the uBotino see
http://www.societyofrobots.com/robotforum/index.php?topic=12216.0This is like a mixture of the $50 robot design and Arduino but also comes with the ability to drive two small DC motors. If you have questions about it: then post them to the above link.
If you really want 'remote control' then look at eBay but IMHO the "fun" part of robotics is to make them autonomous.