Yes, this is annoying. Your best bet is to unplug and re-plug the things connected to those pins when you re-program. If that ends up being a real problem, you might want to have a simple N-channel MOSFET between those external pieces and the pins, but have the programmer directly connected. Then, have a switch that turns the MOSFET on/off, and flip it when you program.
Another option (which you can combine) is to tie the programmer directly to the pins, but use a 1 kOhm resistor between the pin and whatever that pin is driving or reading. The programmer will then hopefully be able to drive the pins strongly enough.
Finally, if even that doesn't work, keep the same design, but also add a buffer circuit for the programmer pins, so that they have a very strong drive. Just beware that whatever is connected to those pins will also see the programmer signals, unless you implement the off-switch.