[...] I've gotten a couple of brushless motor combos to test out. But, I can't help but think these are not the right thing to use. The ones that futaba brushless motors and the hobbyking ones use are different. They are smaller.
Just a guess, but I assume the ones you have are outrunners and I suspect that they use inrunners in servos.
However, you can get very small outrunners, but whichever, the smaller it is, the faster it will be and the harder you have to gear it.
The only economical feasible way of getting multiturn positioning is to reset to a known position at startup (and perhaps each nn timeunits, tests will tell) and then either count clicks up/down or, with an absolute 360° encoder count revolutions up/down (by index markings) plus the absolute position of the uncompleted turn (you can
get absolute position multiturn resolvers, but you really won't like the price).