Author Topic: Telepresence Robot - Augmented VR  (Read 1680 times)

0 Members and 1 Guest are viewing this topic.

Offline XKForgeTopic starter

  • Beginner
  • *
  • Posts: 4
  • Helpful? 2
Telepresence Robot - Augmented VR
« on: September 29, 2015, 01:23:31 AM »
Hi Everyone!

I finished my recent project and I'd like to share it.  It is a telepresence robot powered by 2 Beaglebones, 2 Arduinos and a Nokia phone, connected via wifi to a Mac.
Johnny-five is the framework used to coordinate the BBBs and the Arduinos.

It has two modes: autonomous and guided.  In guided mode, the robot's head is controlled by an Oculus Rift and driving is controlled with a LeapMotion gesture controller.  In autonomous mode, the robot can detect faces and track them.  More to come on this front! In both modes, the robot's eyes are streamed to the Oculus Rift for stereoscopic vision from the robot's perspective.

Have a look and let me know what you think!  Let me know if you have any ideas for cool HUD overlays or additional sensors I can bolt on.

! No longer available

« Last Edit: September 30, 2015, 01:03:52 PM by XKForge »

Offline cyberjeff

  • Full Member
  • ***
  • Posts: 114
  • Helpful? 7
Re: Telepresence Robot - Augmented VR
« Reply #1 on: September 29, 2015, 12:30:50 PM »
Hi Everyone!

I finished my recent project and I'd like to share it.  It is a telepresence robot powered by 2 Beaglebones, 2 Arduinos and a Nokia phone, connected via wifi to a Mac.
Johnny-five is the framework used to coordinate the BBBs and the Arduinos.

It has two modes: autonomous and guided.  In guided mode, the robot's head is controlled by an Oculus Rift and driving is controlled with a LeapMotion gesture controller.  In autonomous mode, the robot can detect faces and track them.  More to come on this front! In both modes, the robot's eyes are streamed to the Oculus Rift for stereoscopic vision from the robot's perspective.

Have a look and let me know what you think!  Let me know if you have any ideas for cool HUD overlays or additional sensors I can bolt on.

www.youtube.com/watch?v=EdjKXQLxtkM

Cheers!

Nice work so far. Thanks for posting.

I'll be back when you hit the OpenCV parts, or do you have another approach to computer vision? My only disappointment in your project in your project is that that part is offloaded to the Mac. This is just barely doable with the embedded hardware now easily available.

Offline XKForgeTopic starter

  • Beginner
  • *
  • Posts: 4
  • Helpful? 2
Re: Telepresence Robot - Augmented VR
« Reply #2 on: September 29, 2015, 01:11:49 PM »
I use OpenCV for the face recognition bits; the OpenCV part was surprisingly simple, probably no more than 25 lines of code!

 I do anticipate moving most of the software to the Beaglebones for autonomous mode and I suspect it will work just fine.  As the project is still in "prototype" mode, I don't mind having it running off the mac for now.  There are several programs that need to be consolidated before I move them to the BBBs. 

The only thing remaining on the mac will be code necessary for telepresence with the robot in manual mode..

Offline cyberjeff

  • Full Member
  • ***
  • Posts: 114
  • Helpful? 7
Re: Telepresence Robot - Augmented VR
« Reply #3 on: September 29, 2015, 03:19:05 PM »

 I do anticipate moving most of the software to the Beaglebones for autonomous mode and I suspect it will work just fine.  As the project is still in "prototype" mode, I don't mind having it running off the mac for now.  There are several programs that need to be consolidated before I move them to the BBBs. 

The only thing remaining on the mac will be code necessary for telepresence with the robot in manual mode..

I have an unused BB black here. What do you think it is best used for? I have a DUE for the mechanics and a RasPi 2 for the Kinect for my catbot project.

Offline XKForgeTopic starter

  • Beginner
  • *
  • Posts: 4
  • Helpful? 2
Re: Telepresence Robot - Augmented VR
« Reply #4 on: September 30, 2015, 12:09:35 AM »
There are any number of projects you could use a BBB for.  Online, I read that the RaspPi didn't have enough horsepower for hi-def video streaming and that's why Babbage has BBBs. 

Offline XKForgeTopic starter

  • Beginner
  • *
  • Posts: 4
  • Helpful? 2
Re: Telepresence Robot - Augmented VR
« Reply #5 on: November 09, 2015, 05:08:37 PM »
Thanks, dissy363.  I posted my second video on youtube.  Have a look!  Hopefully they are helpful.  The new videos will be more detailed.

https://www.youtube.com/watch?v=KPNC3twaz6c

 


Get Your Ad Here

data_list