[...] to transmit 4 data pins from the serial port of the computer [...]
Even though you haven't done it yet, beware that the serial port use +/-12V signals, so you need a level converter for that.
i've used the circuits shown in : http://home.att.net/~wzmicro/rf.html .. with the differences that i've connected +3v for the AD pins of encoder (HT12E) that i want to transmit
Are you aware that, in the schematic, it is coupled so the HT12E is powered continuously, but the transmitter is only grounded (through D1 to D4) when a button (S1 to S4) is pressed?
and that i've used the AM 433Mhz transmitter and receiver shown in [...]
Also, for the sake of testing, i had connected 4 LED's to the 4 output pins of the decoder (HT12D). Neither them nor the valid transmission pin (pin 17 of decoder) showed any signs of success.
I guess you don't have an oscilloscope, which would make it a 5 minute job to find the fault?
If you probe the DOUT-pin of the HT12E with a DMM (in the 20V range Try both AC and DC) what does it read?
You really have to approach your problem in a modularized way. First verify the encoder, then the transmitter, then the receiver and finally the decoder.