Society of Robots - Robot Forum
Software => Software => Topic started by: Commanderbob on March 29, 2009, 02:53:52 PM
-
I want to make a robot that takes input from its surroundings via ultrasonic and Sharp IR sensors. I then want it to use this information with the input of a three axis accelerometer, gyro, and temperature sensor (included in the gyro), to figure out how to avoid walls.
I have done a lot of reading on reinforcement and Q learning and I understand the outline of what needs to be done.
I am going to use the accelerometer to give negative and positive feed back. If there is a spike of deceleration then that is negative. I haven't quite decided what is positive yet.
I want to program into the robot as little as possible, ie not telling it that values less than 3000 for the motor will make it go backward. However this creates a problem of a massive amount of choices for it to make, the motor and turning values go from 2000-4000 roughly so around 4,000,000 possible combinations.
As this will require a lot more power then an AVR has I am using a Gumstix.
Does anyone here have experience with reinforcement learning and is willing to help me out?
I attached some pictures of the robot.
I have all the code for the AVR (on the robostix) done and now I just need to get the code going. All the stuff I have read about reinforcement learning is just theory and no actually code or how to implement it, so I need some help.
Thanks,
Justin
-
input of a three axis accelerometer, gyro, and temperature sensor (included in the gyro), to figure out how to avoid walls.
I'm a bit confused . . . accelerometers, gyros, and temperature sensors are useless for avoiding walls . . .
The wall avoiding algorithm is as simply as a photovore.
photovore:
if light found
turn to light()
wall avoider:
if wall found
turn away()
I want to program into the robot as little as possible, ie not telling it that values less than 3000 for the motor will make it go backward. However this creates a problem of a massive amount of choices for it to make, the motor and turning values go from 2000-4000 roughly so around 4,000,000 possible combinations.
Do you mean you want it to learn how to control its own motors? If so, I recommend reading a ton of AI theory before starting! Oh and learning can be done on a microcontroller. The limitation isn't the processor speed, its the time it takes for a robot to physically complete an action and memory to record what happens.
-
I'm a bit confused . . . accelerometers, gyros, and temperature sensors are useless for avoiding walls . . .
What I want it to do is use the accelerometer and gyro (more the accelerometer) to determine when it hit the wall. There should be a rather large spike of negative acceleration when the wall is hit. I then want it to figure out what it did that caused it to hit the wall and avoid that.
I know it is easy to avoid walls, but I want it to learn how to avoid walls. However, I don't want to code it like that. I want to code it as general as possible, so I could teach it things.
Oh and learning can be done on a microcontroller. The limitation isn't the processor speed, its the time it takes for a robot to physically complete an action and memory to record what happens.
The main reason I am using the Gumstix is for more memory. I imagine a lot of RAM is needed to store the knowledge gained, as the longer it runs the more it accumulates.
-
However, I don't want to code it like that. I want to code it as general as possible, so I could teach it things.
then what language do you want to code it in?
-
However, I don't want to code it like that. I want to code it as general as possible, so I could teach it things.
then what language do you want to code it in?
I was going to code it in C, but that is not what I meant. I meant that I don't want the algorithm to know it is learning to avoid walls, ie tell it before hand that something coming closer is bad. I just want to have it know that spikes of negative acceleration is bad and to avoid that.
-
spikes of negative acceleration on the accelerometer? if so it will only know that there is a neg spike once its hit a wall so it will keep smashing into walls?
-
No it will learn that whatever it did before smashing into the first wall is bad and will not do that again.
-
i can understand if it uses a wall sensor for that but if it uses a gyro/accel to read this it will only be able to see the negative spikes once its too late.
-
No it will learn that whatever it did before smashing into the first wall is bad and will not do that again.
There in lies the problem. I am not sure how to make it realise what led up to the event of hitting the wall. Was it something it just did, or something that happened a few minutes ago?
I was thinking of storing the scans form the past few minutes, finding trends in them, saving those trends as bad, then if it hits a wall again compare the trends to see if some are similar.
-
i can understand if it uses a wall sensor for that but if it uses a gyro/accel to read this it will only be able to see the negative spikes once its too late.
As you can see in the pictures it also has an IR sensor and ultra sonic sensor. I want it to learn what the inputs from those sensors mean, by using the accelerometer as a base of knowledge.
-
No it will learn that whatever it did before smashing into the first wall is bad and will not do that again.
There in lies the problem. I am not sure how to make it realise what led up to the event of hitting the wall. Was it something it just did, or something that happened a few minutes ago?
I was thinking of storing the scans form the past few minutes, finding trends in them, saving those trends as bad, then if it hits a wall again compare the trends to see if some are similar.
but then there lies a problem as most of the time events before hitting a wall are the same as regular driving.
-
That is the whole point of the program, to pick out which events are different, which ones lead to crashes.
-
ohhh, so will it record the events and if there is a crash store it and if those events start happening again know it may crash again?
-
That's the basics of it. The problem is recognising what in the recorded events led to the crash. If anyone has any suggestions on how to do this I might actually make some working code ;).
-
That's the basics of it. The problem is recognising what in the recorded events led to the crash. If anyone has any suggestions on how to do this I might actually make some working code ;).
Take a look at artificial neural networks. They are very good at recognizing patterns in complex data sets. But I don't know if you will be able to use this kind of tools for real time data processing on a MCU...
-
Well, I'm using the Gumsitx, 600MHz.
I've read some about neural networks, but I don't quite seem to understand how they help. I am going to read some more tonight. If you have any info that would help me code one that would be great.
-
Commanderbob: Perhaps you would want to simulate algorithms on a PC before using them on your robot. You can simulate and repeatedly set up scenarios for your algorithm testing.
(Something like robotbasic.org might help)
-
Ok, so basically your refusal in wanting to use any sort of visual sensors (and only using touch) means you want to build a blind robot?.?.?
-
I want to make a robot that takes input from its surroundings via ultrasonic and Sharp IR sensors. I then want it to use this information with the input of a three axis accelerometer, gyro, and temperature sensor (included in the gyro), to figure out how to avoid walls.
I'm a bit confused . . . accelerometers, gyros, and temperature sensors are useless for avoiding walls . . .
What I want it to do is use the accelerometer and gyro (more the accelerometer) to determine when it hit the wall. There should be a rather large spike of negative acceleration when the wall is hit. I then want it to figure out what it did that caused it to hit the wall and avoid that.
I know it is easy to avoid walls, but I want it to learn how to avoid walls. However, I don't want to code it like that. I want to code it as general as possible, so I could teach it things.
There are visual sensors used (the ultrasonic and IR rangefinders). It's just that this robot is essentially learning how to avoid obstacles. If I'm not mistaken, it will relate what occurs when it collides with something (using a gyros and accelerometers) to what occurs visually (using the Ultrasonic and IR sensors) and to not let it happen again.
Imagine it like we are just learning the different textures (smoothness/roughness/softness, ect.) an object posseses. When we first touch the object, we will relate visually to what we sense on our fingertips, so if it's a hard, flat surface that we view, and it feels smooth.
The next time we see a surface similar to that, we don't need to reach out and touch it before identifying the texture of the object. We can just say its texture is smooth because of the similarities it possesed with the first object.
BTW, Cool project CommanderBob!
-
Comanderbob, I'm thinking that you'll first have to clearly define the "problem" your robot is going solve and use it's performance as a positive feedback and hitting a wall as a negative one. Otherwise, if you program your robot just to avoid walls, it will quickly solve this problem and devise world's best wall avoiding algorithm - staying on the spot.
-
Comanderbob, I'm thinking that you'll first have to clearly define the "problem" your robot is going solve and use it's performance as a positive feedback and hitting a wall as a negative one. Otherwise, if you program your robot just to avoid walls, it will quickly solve this problem and devise world's best wall avoiding algorithm - staying on the spot.
In other words, the robot has to have a goal / target. For example, it could be to go to a certain spot, while avoiding obstacles on the way.
At the very least, the goal should be to always move in a straight line, I think. That way the robot should never stop and do nothing.
-
can it stop to turn?
-
Sorry for not replaying in so long. For some reason all my email were going into the spam folder, so I didn't know anyone posted.
I have not done any more actual work as school has kept me busy. However with only 15 days left I will have more time. What I did do was find http://leenissen.dk/fann/ (http://leenissen.dk/fann/) it is basically what I want.
At the very least, the goal should be to always move in a straight line, I think. That way the robot should never stop and do nothing.
Ha ha, that is what I was thinking, if it's goal was only to avoid walls then the best was to do that would be to never move and it should eventually just stop.
Commanderbob: Perhaps you would want to simulate algorithms on a PC before using them on your robot. You can simulate and repeatedly set up scenarios for your algorithm testing.
(Something like robotbasic.org might help)
Thanks for the suggestion but I find it a lot more work to do that than to just dive into the real thing. I already have the robot built, I just need the code and to simulate the real input it will receive will be harder than just letting it run.
Canabots, that is exactly what I am trying to do.
guncha, that again is what I was thinking but the only real problem I can think of is avoiding walls. I have a GPS that I could hook up later and try to make it learn how to get to places sent wirelessly to it.
I am getting a new computer in a few days so I'll wait for that before I start the coding, as I will need to rebuild the gumstix stuff.
Thanks for all your feedback!
Justin Rajewski
-
I know it is easy to avoid walls, but I want it to learn how to avoid walls. However, I don't want to code it like that. I want to code it as general as possible, so I could teach it things.
Here's some food for thought.. http://www.societyofrobots.com/robotforum/index.php?topic=7.msg30869#msg30869 (http://www.societyofrobots.com/robotforum/index.php?topic=7.msg30869#msg30869)
-
Some other interesting places to visit.
FANN documentation/tutorial
http://softlayer.dl.sourceforge.net/sourceforge/fann/fann_doc_complete_1.0.pdf (http://softlayer.dl.sourceforge.net/sourceforge/fann/fann_doc_complete_1.0.pdf)
Google Tech Talk on ANN
The Next Generation of Neural Networks (http://www.youtube.com/watch?v=AyzOUbkUf3M#noexternalembed-lq-lq2-hq-vhq)
I definitely recommend checking out FANN, I haven't tried it yet but it looks promising.
-
I was messing around with the FANN library on my computer and it is really cool. I made a training file with 10 inputs and 2 outputs (speed and turning). The training file contained stuff like 9 8 5 3 2 0 1 0 0 0 (larger number = closer) the output would be something like .94 .75 meaning turn 94% right and forward at 75% max speed. Once it was trained off the sample data I gave it, it really seemed to work well.
Now I need to figure out a way to train it on the fly, without a training file.
My new computer does not have a serial port :'( so I have to go buy a USB->Serial adapter before I can work with the Gumstix.
Anyone know where I can get a good cheap one? Radio Shack sells them for $30 which seems really over priced to me.
-
I bought a usb to serial adapter on ebay for $2 shipped ;D
-
does that online store that has free shipping have them?
y'know the one that's really cheap...
-
I would be inclined to use a combination of a hash table or bloom filter with locality sensitive hashing.
I would get the sensor readings every .1 sec for example. If I got some bad news from my accelerometer I would go back through the last 10 sensor reading (ie the last 1 sec) . Do the locality sensitive hash of all the sensor reading for each of the past 10 sensor reading in turn. Eg int x=LShash(speed X, speedY, light level, ......). Then I would put an entry into a hash table for each of the 10 values of x indicating something bad had happened.
Then it is a matter of constantly monitoring your input sensor readings, doing the locality sensitive hash to get x, then putting x into the hash table to see if anything bad happened before with approximately those sensor readings.
There is an overly technical Wiki page on LS hashing:
http://en.wikipedia.org/wiki/Locality_sensitive_hashing (http://en.wikipedia.org/wiki/Locality_sensitive_hashing)
-
Chapter 16 in the book Robot Programmer's Bonanza does EXACTLY what you are trying to do
with a program and all.....
http://www.amazon.com/Robot-Programmers-Bonanza-John-Blankenship/dp/0071547975/ref=pd_bxgy_b_img_b (http://www.amazon.com/Robot-Programmers-Bonanza-John-Blankenship/dp/0071547975/ref=pd_bxgy_b_img_b)
it does it on a simulated robot instead of a real one but the ALGORITHMS are what you need and
the program shows all the algorithms you want. The chapter also explains the theory and
how to improve on the given program.
You can see the program in action in this video
Simulator for Robotics (http://www.youtube.com/watch?v=rSXroWfnWfw#lq-hq)
Also you can download the program source code from
www.RobotBASIC.com (http://www.RobotBASIC.com)
Samuel
-
Well I got my Serial to USB converter from Amazon for $15.25, I wanted one that I knew would work with Linux. I also just got libfann to work with the gumstix and ran the first program on it, sadly the battery died so I have to wait for it to charge.
Thanks for all the suggestions! The current problem I have is how many angles should I take measurements at? I have the ultrasonic and IR sensors mounted on a HS-50 servo, which is fairly fast. I need a good trade off of scan points and speed. Does anyone have experience with this? I think the standard would be 5, full left, 45 left, forward, 45 right, full right.
Justin
-
>:( Another set back...
The regulator in the ESC does not have the power needed to power the robostix/gumstix/netstix/wifi configuration (the wifi pushes over the limit). Whenever the wifi is in moderate use the gumstix resets. The ESC also gets pretty toasty just running the gumstix, not even the motors (stupid linear regulators). I ordered a switching BEC with 10A rating.
-
If you want the robot to really "think" and "learn" you should look into neural networks. A good introductory tutorial can be found at http://www.ai-junkie.com/ann/evolved/nnt1.html (http://www.ai-junkie.com/ann/evolved/nnt1.html).
Regards,
- Moshe
-
If you want more RAM then I've just bought some of these http://yhst-27389313707334.stores.yahoo.net/micaad.html (http://yhst-27389313707334.stores.yahoo.net/micaad.html) which look pretty cool. Add a 4Gb micro SD card for a few bucks and you have 4Gb of RAM via an SPI interface. Will let you know how I get on - once they arrive from the USA
-
If you want the robot to really "think" and "learn" you should look into neural networks. A good introductory tutorial can be found at http://www.ai-junkie.com/ann/evolved/nnt1.html (http://www.ai-junkie.com/ann/evolved/nnt1.html).
Regards,
- Moshe
You must of not read the thread. libfann is "Library for Fast Artificial Neural Networks". That is what I am using.
If you want more RAM then I've just bought some of these http://yhst-27389313707334.stores.yahoo.net/micaad.html (http://yhst-27389313707334.stores.yahoo.net/micaad.html) which look pretty cool. Add a 4Gb micro SD card for a few bucks and you have 4Gb of RAM via an SPI interface. Will let you know how I get on - once they arrive from the USA
I think you mean Flash not RAM. MicroSD cards use flash memory. Also the gumstix already has a microSD slot on it.
-
If you want more RAM then I've just bought some of these http://yhst-27389313707334.stores.yahoo.net/micaad.html (http://yhst-27389313707334.stores.yahoo.net/micaad.html) which look pretty cool. Add a 4Gb micro SD card for a few bucks and you have 4Gb of RAM via an SPI interface. Will let you know how I get on - once they arrive from the USA
I think you mean Flash not RAM. MicroSD cards use flash memory. Also the gumstix already has a microSD slot on it.
Yes - I know all of that but thank-you for the lesson. ;D Of course by mentioning that it requires an SPI interface then I'd assumed everyone would realise that its not ordinary RAM in the mcu address space
But the only difference between RAM and Flash is that one is persistent and the other isn't, and one is more complex/slower to access than the other. But at the end of the day its still a persistent memory - hard drive or call it what you will.
Just thought that if you ONLY need more memory for storing maps/etc then this board may be smaller and need less power than a whole gumstix board.
So glad I took the time to give a comment but good luck
-
I see what you were saying, thanks for the suggestion.
I think speed/RAM is more a problem than Flash. Training and running a neural network takes a decent amount of power. Also you need to run Linux to use libfann, of course you could try to write your own that would fit in an AVR, but why reinvent the wheel?
-
Just to add my experience with SRAM and a low level microcontroller (meaning without integrated memory support).
I have a 32 mb sram ic running on a board with a pic18f. the sram needs 32 I/O lines in order to use it (24 for addressing and 8 for data) as well as 1 for write enable. most low level microcontrollers dont have this amount of I/O available. especially not just for ram.
to get around this, i used 2 I/O expansion IC's. the expansion IC's use an I2C interface so the microcontroller sends I2C commands to the port expanders in order to control the SRAM IC. I2C isnt the fastest protocol in the world, and think about having to send 32 commands through I2C just to provide a read/write operation on 1 byte....
In short, SRAM might be faster but unless you have integrated support and and plenty of spare I/O you might find its actually slower than flash solutions.
(also you have to create a software storage architecture for reading/writing aswell as creating a small adressing system (mine means that 8bytes have to be read/written before the data byte can be accessed). Plus create an address stack to manage locations and free space)......
Its a lot more involved than it at first seems.
say we have a byte stored in a variable on the microcontroller. we want to transfer this into external ram so we transfer it to address FF2. now when we want to retrieve that byte of data, how does the microcontroller know where to get that byte from? if we keep the address stored in another variable then all thats happened is that weve freed up 1 byte from the microcontroller but used up 3 bytes in order to reference it.
Thats why a lot of thought has to go into creating an addressing system at the beginning of the external ram so it can store its own references
-
If anyone is still interested I started working on this project again, except now I'm doing it for school so I'll be forcred to work on it ;).
I am using libfann on the gumstix. So far I did some test code and it works fine on the gumstix. For anyone who wants it, I attached the bitbake file so you can use it in your projects.
Right now my code is very basic, it sets up a timer so the accelerometer is read 200 times per second. The latest 50 measurements are stored in an array. If the first two are 50 apart (5/8ths of a g) then it is considered a collision and the red LED on the gumstix is flashed.
Other than that it's not doing much else at the moment, it does create a log file on a microSD card.
Tomorrow I plan on writing a better collision detection function. Any suggestions on how to detect a sudden drop in data that only a collision would cause?
-
UPDATE: I now have the ultrasonic sensor scanning about once a second. It takes three measurement (-45, 0, 45 degree positions). The data collected is fed into the ANN which outputs two numbers, one for the motor, the other for the turning servos. For now the motor is not powered, but I can push the car around and it turns itself.
It is not learning yet, but I added some sample data that the ANN is trained off of just to test how well it will work. It surprisingly works really well with just 5 sample sets of data.
Do you think that three measurements with the ultrasonic sensor are enough to detect most objects? Has anyone had any experience with with? I was originally going to use 5 (-90 -45 0 45 90) but it took too long.