When you wired up the LED, you have one end connected to + of the battery.
The other end is connected to D4.
So if D4 is also high (+), nothing will happen to the LED since both ends are + and +.
When you set D4 to low (-), you then have a voltage difference across the LED (+ and -), turning it on.
If you instead wired one LED end to ground (-), and the other to the D4 pin, when D4 goes high it is powering the LED - potentially bad. Its better to use the above method so that the battery directly powers the LED and not the microcontroller.