You'll still need power to AVCC and AREF, yes, otherwise the ATMega won't work properly. Which ceramic capacitor are you talking about? Also, if you're going to be pulling out the ATMega regularly for programming, buy a ZIF socket instead of a normal one - there isn't much more annoying than pulling a socket out and killing the uC in the process. I'd actually advise you not to take the pins out though, its another ~5min of soldering that saves you a lot of time and effort later.