Behavioural Robotics

[Jon_Thompson]

Is anyone else here doing anything with behavioural robotics? My main focus of interest is in this area, specifically in the design of systems that adapt to previous experience of the environment.

[mstacho]

I've looked into it a while back.  My work is (vaguely) dealing with something that comes right before behavioural robotics: we are working on adaptation and rapid learning for a specific context at the moment.  The idea being: if the robot has a learned representation from experience, cool, it should use it.  but if it doesn't, it shouldn't keep failing while learning until it gets it right.

MIKE

[jwatte]

it shouldn't keep failing while learning until it gets it right.

So how would it learn? From what I know, that's actually exactly what humans do when learning something as children.

[Jon_Thompson]

I've looked into it a while back.  My work is (vaguely) dealing with something that comes right before behavioural robotics: we are working on adaptation and rapid learning for a specific context at the moment.  The idea being: if the robot has a learned representation from experience, cool, it should use it.  but if it doesn't, it shouldn't keep failing while learning until it gets it right.

MIKE

I get where you're coming from on that one!

I have a long term fascination with the Eliza Effect. Does a machine have to be genuinely intelligent, or do we simply have to perceive it to be so? After all, we readily anthropomorphise other animate objects.

My experiment involves building a kind of "pet" that people can play with. Though it is clearly a machine with tracks and bumpers and so on, if its range of reactions to situations is flexible, adaptable and natural enough, will they begin to imbue it with its own "personality" without it resembling a real animal? Above all, will it be fun to have around?

[mstacho]

@jwatte: Although I agree that we learn by making mistakes, that is because a lot of our neural networks aren't structured.  We are, in effect, learning what the model is along with that the model does, and that leads to a ton of mistakes.  I feel that robots learning new skills can be given a leg up by having really excellent reactive algorithms working in parrallel with learning algorithms.  This way, learning doesn't *enable* performance, it *enhances* it.

I come at it from an engineering background, though, so to me the best robot is the one that does the job, not the one that has the coolest software. :-P  Eventually, reactive algorithms fail and learning has to be done, I agree.  But we should give the robot the best abilities we can before it has to learn in order to maximize its usefulness.  At least, so my philosophy goes.

@Jon_Thompson: Definitely look up the (now old)  "Subsumption architecture" from Rodney Brooks.  I'm personally a fan of a hybrid approach.  While Brooks says that complex behaviour doesn't need complex control policies and I agree, I also feel that SOME complex behaviour does, or can benefit from more complex control. 

Also look up http://en.wikipedia.org/wiki/Braitenberg_vehicle.  They're neat, and really go to show you how far you can get by just wiring sensors to motors.

MIKE

[Jon_Thompson]

My approach has been to design specific subsystems to handle different parts of the overall design. Some use adaptations of techniques found in other approaches, while others I cannot find in the literature. This "pragmatic" approach has been remarkably successful, but it's a hell of a lot to think about!

[jwatte]

This way, learning doesn't *enable* performance, it *enhances* it.

I see -- so, if you know you need to walk, you already know you need to sequence the leg movements in a certain pattern, and you can use learning to fine-tune, and adjust to unexpected inputs, rather than having to first work out coordination. That seems like a good way to quickly get to useful behaviors that have some level of adaptation, with the likely acceptable trade-off that wild variations or off-spec environments probably will make it fail.

Looking forward to videos of your project :-)

[mstacho]

I agree, and we've already had some cool successes: our robot hand can hold a paper cup, for instance, and then we can fill the cup with water.  Using only an adaptive controller with a few sensors, the robot will increase its grip force only as much as is necessary to handle the increased load of the water.  It doesn't NEED to learn that water is being poured, it just accomodates an increase in load forces.

However, these ideas are still limited.  Learning is used when the problem you're trying to solve is fundamentally undefined (if it were defined, then why not just design a controller and be on your way?)  So as long as you can build a controller that is "good enough" and then adapt its parameter to be better, my work is great.  But what if you can't build that controller in the first place?  And just how much offline programming is too much?  If it's harder  to build the controller than to just toss in a neural network and call it a day, are there benefits to doing things the hard way?

MIKE

PS: The answer to the last question is actually a resounding YES.  For instance, no one in their right mind would put a neural network onto a jumbo jet.  You just can't ensure that it will work for every possible flight scenario, and training the network will require either an exceptional simulation or a few thousand crashed jets :-P