Building your own adapter will have varying degrees of success based on your skill/craftsmanship, tools available, and precision required. The construction methods described above are all valid. You also might be able to find a piece of tube to make a spacer out of with minimal machining.
I did a little more searching for a shaft coupler but the significant size difference in your shafts makes finding an off the shelf solution challenging (without connecting several adapters together). So time to think a little more creatively...
I found the following 3mm shaft coupler with a 8mm OD.
http://www.technobotsonline.com/inline-coupling-3mm-to-3mm.htmlIm sure you could find other similar products if this exact one doesn't fit the bill.
Be warned that while the OD is listed as 8mm it is likely not very precise and may require a little modification (sanding or shimming) to fit nicely in a 8mm bore. Also a coupler such as this does not contain any features to attach a wheel to the outside but you could probably glue it. An epoxy with roughened surfaces would probably work well.