1. if i power a 2A motor plugged in straight form the transformer (with the usual rectifier bridge and capacitors), to calculate the wattage, do i do 18x2=36W, or do i do 25.4x2=50.8W? In other words, do i calculate the power using RMS voltage output form the transformer, or using whatever comes out of the capacitors?
You should calculate with what the motor sees. When you bridge rectify and add a capacitor, assuming the cap is infinite large, you will get: AC x sqr(2) - 2 x diode voltage drop.
Unfortunately, you don't have room for this large a cap, so the voltage will sack in between each peak, the load and the value of the cap will determine how much.
To get accurate results, you should use the U_RMS of that but you probably have to eyeball an average off a 'scope, as it's not many instruments that reads true RMS of anything but a sine wave.
Before you calculate anything, you should verify the current draw, as this will change with voltage.
RMS values are the same as the DC values, if the DC is really a DC.
Btw. With 2x18V windings, you could make a center tap and use just two diodes for the bridge.
2. what happens when i draw 250W out of it? Is it calculated that at 250W, the power drop will be acceptable, or if i go over 250W, i get pretty lights and special smoke effects?
In a perfect world... And this if your transformer is from a reputable source, a 250VA (~W) marked transformer is spec'd at 250VA out.
What this means is, that when you draw 250VA it will give the spec'd voltage. With a lower drain, it will give a higher voltage - how much higher depends on the quality (and to some extent the power rating), better quality means less overvoltage unloaded (or partially loaded).
There should be a safety margin in the design as well, so if you push it and drain a little more than 250VA, the voltage will fall under the spec'd. but it won't blow up with small overloads that is not permanent.
The transformer design will have a certain temperature as the first limiting parameter and that is chosen with a margin, so that the manufacturer is sure they won't be returned with the lacquer isolation melted and the entire brick melted by short circuited windings.
That's all good with reputable manufacturers, but if you bought a Chinese (or from any other low wage country) transformer, you should measure it up to be sure.
Load it as hard as it takes to get the voltage to the nominal specs and note the current (that's your VA rating). It's best to measure with a true RMS voltmeter (= one that says "True RMS" somewhere on the meter), as a loaded transformer does not give a true sine wave out.
While testing it, measure the temperature of the core. This is best done (on a toroid) by placing a temp. sensor against the transformer in the middle hole and use a sponge or similar to press it against the transformer and isolate it from ambient temperatures.
To make sure the temperature follows the temperature of the core and windings, increase the load in small steps and keep it loaded for 20 minutes (or whatever it takes for the temperature rise to stabilize, nothing under 5 minutes will do) in each step, then (assuming it is still not too hot) increase the load eg. 5..10% or thereabout and repeat.
40°C/104°F is a safe temperature to touch and if you want to push it a little further, 50°C/122°F should be useable if nobody can touch it in the finished project (don't encapsulate it, it need air flow for cooling - use metal net or similar).
Above all, keep the mains from bodyparts, I hate funerals.
(And don't use your tongue to test for voltage on the low voltage side either).