Man, it is a maze trying to find this information.
If You want to find step-by-step tutorial, then there aren't a lot of them available. If You are willing to extract information from what is actually available online - there are plenty of resources, such as YouTube videos, other robots which are not what You want, but they use same/similar control systems.
So far I understand I will need some kind of PID controller, likely code to control the motors and all the hardware.
PID is the one to be used in this case. You will most likely want to have gyro to get angular velocity and accelerometer to get current tilt angle. Data from Your sensors will have to be fed into PID loop to set angular velocity of flywheel.
I can write in C or shell scripts if I need to.
I'm not sure if You are aware, but C for microcontrollers is a bit different that for operating systems. Not that the language itself is different, but the features You use are different. Shell script in this case is pretty much useless.
The thing is that all these posts are so old that I'm wondering if there is a kit out there that I could buy or someone could help me put the parts list etc together.
I'm not aware of any kits of that sort, but hey, You are free to look around. Regarding part list - get parts used to build $50 Robot MCU - and You have brain of the robot.