The Due is a dead product IMO. It is much slower than the Raspberry Pi (by a factor of 10 or so) and it has much less RAM (by a factor of a few thousand.) Neither the Due or the RPi are 5V compatible, though.
The Arduino Uno is pretty nice for tinkering. If you want that same CPU on your own (Atmega 328p,) it's only $2.50 in quantity dozens, and it can run just fine with just supply voltage and a de-couping capacitor (which costs maybe a quarter) and perhaps a reset pull-up resistor for good measure. You'll need a USB programmer, rather than a bootloader, to program it, but that's pretty cheap. And you'll need a 6-pin ICSP header to actually program it, unless you have some socket that's already wired to plug it into. It will run on the built-in RC oscillator at 1 MHz or 8 MHz. If you want up to 16 MHz (the Arduino Uno speed) or 20 MHz (the rated speed) you need a crystal (50 cents) and a couple of small capacitors (a handful of cents.)
Breadboard, CPU, capacitors, crystal, programming header, and resistor is probably less than $6, somewhat depending on the cost of the breadboard. And the programmer is $20, and a one-time investment.
Finally, the reason the AVR has such a fan following is that it is very easy to program. The programming model makes sense, and peripherals are very straightforward. avr-gcc is a great compiler, and avr-libc is a reasonably C library (but, really, in an MCU, you don't want to be using too much of the C library...) avrdude makes uploading/programming super easy. And here's the secret: The Arduino IDE actually uses avr-gcc, avr-libc and avrdude under the covers. It just puts a newbie-friendly front on those tools.