Telepresence Robot - Augmented VR

[XKForge]

Hi Everyone!

I finished my recent project and I'd like to share it.  It is a telepresence robot powered by 2 Beaglebones, 2 Arduinos and a Nokia phone, connected via wifi to a Mac.
Johnny-five is the framework used to coordinate the BBBs and the Arduinos.

It has two modes: autonomous and guided.  In guided mode, the robot's head is controlled by an Oculus Rift and driving is controlled with a LeapMotion gesture controller.  In autonomous mode, the robot can detect faces and track them.  More to come on this front! In both modes, the robot's eyes are streamed to the Oculus Rift for stereoscopic vision from the robot's perspective.

Have a look and let me know what you think!  Let me know if you have any ideas for cool HUD overlays or additional sensors I can bolt on.

! No longer available

[cyberjeff]

Hi Everyone!

I finished my recent project and I'd like to share it.  It is a telepresence robot powered by 2 Beaglebones, 2 Arduinos and a Nokia phone, connected via wifi to a Mac.
Johnny-five is the framework used to coordinate the BBBs and the Arduinos.

It has two modes: autonomous and guided.  In guided mode, the robot's head is controlled by an Oculus Rift and driving is controlled with a LeapMotion gesture controller.  In autonomous mode, the robot can detect faces and track them.  More to come on this front! In both modes, the robot's eyes are streamed to the Oculus Rift for stereoscopic vision from the robot's perspective.

Have a look and let me know what you think!  Let me know if you have any ideas for cool HUD overlays or additional sensors I can bolt on.

www.youtube.com/watch?v=EdjKXQLxtkM

Cheers!

Nice work so far. Thanks for posting.

I'll be back when you hit the OpenCV parts, or do you have another approach to computer vision? My only disappointment in your project in your project is that that part is offloaded to the Mac. This is just barely doable with the embedded hardware now easily available.

[XKForge]

I use OpenCV for the face recognition bits; the OpenCV part was surprisingly simple, probably no more than 25 lines of code!

 I do anticipate moving most of the software to the Beaglebones for autonomous mode and I suspect it will work just fine.  As the project is still in "prototype" mode, I don't mind having it running off the mac for now.  There are several programs that need to be consolidated before I move them to the BBBs. 

The only thing remaining on the mac will be code necessary for telepresence with the robot in manual mode..

[cyberjeff]

 I do anticipate moving most of the software to the Beaglebones for autonomous mode and I suspect it will work just fine.  As the project is still in "prototype" mode, I don't mind having it running off the mac for now.  There are several programs that need to be consolidated before I move them to the BBBs. 

The only thing remaining on the mac will be code necessary for telepresence with the robot in manual mode..

I have an unused BB black here. What do you think it is best used for? I have a DUE for the mechanics and a RasPi 2 for the Kinect for my catbot project.

[XKForge]

There are any number of projects you could use a BBB for.  Online, I read that the RaspPi didn't have enough horsepower for hi-def video streaming and that's why Babbage has BBBs. 

[XKForge]

Thanks, dissy363.  I posted my second video on youtube.  Have a look!  Hopefully they are helpful.  The new videos will be more detailed.

https://www.youtube.com/watch?v=KPNC3twaz6c