After reading this thread I believe people here need some basic introduction to electronics an robotics with capacitors.
Capacitors have two functions:
DC circuit function: charge up and then realease a high current(like in camera flashes). However this fuction is not that popular in robotics. In this case the current dont flow trough the capacitors.
AC circuit function: in AC circuits the capacitor becomes a resistor. Its impedance(the resistance) is calculated as Z=1/(jwC), where j is the imaginary unit, w=2*pi*f where f is the frequency of the current and C the Capacitance. In other words, the higher the frequency of the AC, the lower will be the resistance of the capacitor. This function is widely used in electronic cicuits of any kind to remove noise(except, maybe, pure logical cicuits). Those are band pass filters. If it is a low band pass circuit the circuit will allow AC with low frequency to have high gain output. High bang pass filters are the opposite. You can also combine those two filters and have a hybrid filter. In this case current flows through the capacitors.
There is also a AC-DC cicuit function. This is used in biased transistor circuits. But that is just a mixture of the two functions above.
Once you know that you can understand why put capacitors between the pins of the regulators. They filter the noise that might damage your ICs. That is why it is also recommended to put some capacitors in parallel with DC motors. AND that is the reason you shouldnt NEVER, EVER, put a capacitor neighter in series nor in parallel between the PWM output of your microcontroller and the servo signal input socket/pin. The PWM signal is basicaly squareshaped AC. It has a frequency therefore the capacitor will react to it in the same way it will react to noise: the PWM signal will be filtered and your servo wont work.
Finally, about the "direction" you have to solder the capacitor. Capacitors are generally divided in two kinds: ceramic capacitors and electrolytic capacitors. Ceramic capacitors are the same ones you learn about in school: they are like resistors and can be soldered in any "direction". On the other hand there are the Electrolytic capacitors that have a direction. The capacitor you are using is Electrolitic if you connect the + side in the ground(which can also be considered the - side) your capacitor will just explode. Because of that you MUST connect the + with the + and the - with the -. (in a more precise way, you connect the + side of the capacitor with the highest electrical potential terminal and the - side with the smallest electrical potential terminal.)
The tutorials and above links are good but they more like cake recipes.(SOMETIMES) They dont teach you the reasons behind the instructions. So if want to experiment you are (SOMETIMES) ****ed(unless you study, of course).