Robot Learning

[brandonman]

Hey everyone. First post here. I'm a freshman in High School who has been working with robotics for the past year or so. I'm an avid fan of the BEAM robotics movement, although it has been slow here lately. I feel most of the problems of robotics can be implemented very well without using a microcontroller, which seem overkill for most situations (Don't get me wrong, a lot of stuff may require it, but a lot doesn't need it.) Anyways, enough on my background.

I've been dreaming a lot about robot learning. With my mentality on BEAM robotics, I've come up with a few possibilities (Nothing very large scale *yet*)

1: RAM. I have considered using the state of all sensors and output devices on the robot as addresses to the RAM, which then write a 'good' or 'bad' bit to that location, saying "Is this response (the output bits) good for the input bits? If so, write a 1. Otherwise, write 0." This seems interesting, but the problem is that of course, this is very black and white, allowing NO room for any gray areas. Also, once something happens, drawing upon it will be definite. (Not really changing like true memory.)

2: Decoder -> Capacitors. The states are sent to the Decoder, which then turns on an output, allowing the 'good' or 'bad' bit to travel through to fairly large capacitors at the other end. (Nothing massive, but ones that will store for a fair amount of time.) This allows for a little bit of 'greyness', as well as slowly changing memory which is much less definite and black/white. The capacitor is slowly added up to positive, not automatically positive.

Curious if these seem to be good ideas, or are horrible. I am extremely interested in seeing what everyone else would consider good ideas for memory.

[bulkhead]

That's pretty interesting for a fresh take on robot learning.  I think though that most robotic learning occurs on a higher level, mainly in the code that the robot is running rather than in the hardware.  Wikipedia has a lot of interesting info on AI: http://en.wikipedia.org/wiki/Artificial_intelligence#Learning.  There are lost of interesting questions about where it will lead to--code that generates more code, etc.

[Webbot]

I've successfully used the capacitor approach in the past to simulate conditioning.

When sensor1 is triggered then it fires a monostable to generate a pulse of several seconds duration.
If sensor2 is triggered then it fires another monostable to generate another pulse of several seconds duration.
Feed these both through an AND gate to get a pulse whose duration is set by how close in time the two sensors fired.
Use this pulse to charge up a capacitor - so the closer together the two events happen then the more it charges
Use an FET transistor and Schmitt trigger to sense when the voltage on the capacitor hits a certain point and then the trigger will change to output of 1. This means that it has learned that Sensor1 firing means that Sensor2 should fire soon so it can behave as if Sensor2 has fired. You can do this with some more logic gates.

If neither sensor fires then the capacitor starts to naturally discharge. The hysteresis in the Schmitt trigger allows the voltage to drop quite a lot before the output goes low (ie it has 'un-learned').

You can improve on it by adding extra circuitry so that if Sensor2 fires without a preceding Sensor1 then it uses a resistor/transistor to briefly short circuit the capacitor so that it 'un-learns' more quickly.

[mbateman]

Although not directly applicable, I do something similar in software for my radar mapping. For every point on the map, I have a 4 bit number. Each time a sensor "sees" something at that point, it sets the sets the number to 0xF. Another process runs in the background that subtracts one from each location on a regular basis. This means that a given sighting fades over time unless it is seen again. This method could be modified slightly to do what you are proposing in software instead of hardware if that would be useful.

[brandonman]

Curious if anybody has any more thoughts on this subject? I'm having trouble thinking of any advantages to this though. Most robots I work with are on a smaller, survival oriented scale, rather than microcontroller based robots, with massive amounts of data. Mine are more along the lines of -> A few sensors, 2 motors, possibly some LED's, maybe some IR. I just am not seeing any advantages to a learning learing circuit in this type of situation, just yet. Does anyone have any ideas on this? Maybe work more with gray logic, rather than the pretty much digital values, or what? Thanks!

-Brandonman

[Webbot]

I always think that some kind of learning is just amazing to watch.

Have a small robot with a sonar running around missing all the obstacles and your kid will say 'Yeah- but what does it do'. Your robot is just an amoeba !

Make it run around, hit stuff, then learn - in front of their eyes - how to miss stuff then suddenly the robot is 'smart'. And you have a big smile.

You may think that this is just an extra step removed: ie 'I could tell it how to miss stuff', but this approach just teaches 'how to learn to miss stuff'.
But once you add enough sensors etc then the robot may take on a behavior that you would never had thought of - and that's the interesting part.

[airman00]

One approach I've seen to learning is pain and pleasure. You add on "pain sensors" - aka bumper sensors, etc. and then give it sensors to understand - like sonar , encoders,etc.
Then set the goal of the robot to avoid pain at all times  - lets call the absence of pain "pleasure". The robot seeks pleasure and avoids pain. By remembering situations where it felt pain , it can reduce the number of crashes or whatever.

[rogerfgay]

Brandonman - Perhaps it would be useful to describe what the robot will be learning and the advantage of being gray about it. Here are a couple of thoughts.

Let's say you have a robot that will learn to avoid bumping into something. If it needs to be convinced by pounding its head against a wall several times, one might say it's a slow learner. It could be better to be black and white about it - learning there's an object on the first bump.

Turn the logic around however and maybe there's an interesting idea. Let's say you have a robot with weak and uncertain sensor capabilities; like Lego Mindstorms Ultrosonic or Light Sensors. (My apologies to Mindstorms fans.) There is uncertainty in any single measurement and you can get chaos if you respond as though every measurement is exact and accurate. More generally - you have a perception problem. It seems to me that you're on the verge of inventing something that would trust a running sum or average of measurements over any single measurement. Perhaps the possibility of an object in the way would cause the robot to slow down, but not to avoid until further accumulated measurements confirm.