my omnidirectional robot with artificial vision

[vixctor]

Hello, first of all, sorry for my english, it's not my native languaje.

  This is mi first post, I find this forum so complete and with a lots of interesting projects.

   I saw the robot Taurus 2. What surprises me, is the fact that it was builted on 2003.  My second robot, born at ends of 2004, is very similar to Taurus 2,  but I didnt get the idea from here... lol

So, this is my robot and how it's evolved since 2004, it has had partipating in several contest.   But today, I used it for fun and to demostrate his capabilities.

Lets see.  It's a onmidirectional plataform, such as Taurus 2, but, in 2004, I didn't have money to buy a CMUCAM, so I decided to design my own artificial video system, because a didn't like many things of CMUCAM:

a) CMUCAM, doesn't broadcast video, so you never can see what your robot sees.  It doesn't matter because the robot must be autonomous, but, it is so usefull seeing what the artificial vision is doing for debbuging purposes
b) CMUCAM it's slow, you can achieve 30fps but with lower resolutions
c) CMUCAM is big and expensive

So I decided start from 0 and design an artificial video system, based on a analog NTSC cam.
Here is the results:

a) You can see the original video plus an overlay of what is going on
b) It runs in real time, 30 FPS @ 256 x 240, It can run on 320 x 240, but I don't like having 9 bits =O)
In fact, it can run at 60 fps, but I used 1 field to do the overlay
c) It so small, everything is inside the lens holder, it uses a PIC18F452 @ 12 mips
d) The algorithm only uses 32 bytes of RAM, so, frame buffer is no needed
e) As the video is analog, it can be broadcasted wireless
f) The cam, can learn more than 256 diferent colors, all done by simple commands (like CMU)
g) finally, Its 4 times faster than CMUCAM3, but half the cost =O)

Well, these are the picture and videos, I hope the comunity enjoy it, greatings...



Last program loaded, seeking for a ball in real time, like a doggie
(everybody can pick and static object, can your robot follow an object moving like that?)
[youtube]wl3ab92Tix0[/youtube]

Here is how the robots "see" the world (the overlay gives you and idea)
[youtube]WJGc6Rdzbtc[/youtube]

My robot at IEEE latin american robotics contest in 2005
[youtube]1IJkaEYJLA0[/youtube]

More on how the cam see the world
[youtube]SPtVwI9NXb8[/youtube]

IEEE latin american robotics contest in 2007, (YES, it can swim, in fact, the robots floats and its hermetic)
[youtube]ZdVqkcWNq1I[/youtube]

IEEE latin american robotics contest in 2008, disabling a "bomb"
[youtube]GQ78aCtYNAk[/youtube]

And finally, how the cam learns colors (This video was for version 1.0, the last version is 2.5 and I working on version 3.0)
Overlay in version 1.0 has 64 colors, 2.5 version has 16 million colors 
[youtube]spILs0mTo7M[/youtube]





   

[Metal Slug 2]

Thats Incredible!  Congratulations!

Man, you came soo close to finishing the robotics comp in 2007.. its too bad your robot toppled over 

Are you by any chance going to make this project open source?  I would love to learn how you were able to do this.

[voyager2]

This is amazing!
Have you used the on board USB on some pics to interface with a USB cam?
Give us details man!

[vixctor]

This is amazing!
Have you used the on board USB on some pics to interface with a USB cam?
Give us details man!

Thanks you liked it.

No, there is no USB CAM, instead off,  I start from 0, thinking about how a video system recongnition can be done...
I used to have one of these economic wireless cameras.  I opened it and got out de CCD sensor, it was an omnivision OV7910...
Next, I had to find the datasheet in internet of this cam.  What I found is that this image sensor is 100% analog, but, it can be configured by I2C, to output analog RGB in three pins instead of NTSC.

I had read how CMUCAM does the job, it compares, pixel by pixel (3 bytes each) vs the range of color (window color) you are looking for.   
It takes so long and too much power procesor, because comparations are in digital domain. So I though "why not compare the analog RGB values, using analog comparators, such as LM339... It's the same principle but in analog domain...."  So, I used 3 analog window comparators, each one conected to an ouput color (R G or B), and to digital potentiometers...   What I got is the same way procesing colors, but in real time, there is no "cycles clock". 

The AND ouput of the 3 windows comparators tell me two things: 0 = no color wanted, 1 = this pixel has the color that you are looking for. 
This single bit, is the input of the pic, so, the microcontroler, just have to sincronize with HSYNC and VSYNC and process this only bit.
It is tricky, doesn't it?

But in fact, every CCD sensor, is analog by nature, then they convert it to a digital domain, so this is not a loss process, thinking that all images in the sensor are analog by nature, I only simplified very much the job of the microprocesor...

So, I got this RGB path ok? then, I had to "reencoded" it to NTSC using an AD725 from analog devices... but, by the way, I put 3 muxes in the RGB path, so, it's very easy to do a video overlay, wich I use to show what de camera is seeing

I hope you understand this explanation, sorry again for my english, but it's usefull to me to practice my writing
regards.

[voyager2]

This is amazing!
Have you used the on board USB on some pics to interface with a USB cam?
Give us details man!

Thanks you liked it.

No, there is no USB CAM, instead off,  I start from 0, thinking about how a video system recongnition can be done...
I used to have one of these economic wireless cameras.  I opened it and got out de CCD sensor, it was an omnivision OV7910...
Next, I had to find the datasheet in internet of this cam.  What I found is that this image sensor is 100% analog, but, it can be configured by I2C, to output analog RGB in three pins instead of NTSC.

I had read how CMUCAM does the job, it compares, pixel by pixel (3 bytes each) vs the range of color (window color) you are looking for.   
It takes so long and too much power procesor, because comparations are in digital domain. So I though "why not compare the analog RGB values, using analog comparators, such as LM339... It's the same principle but in analog domain...."  So, I used 3 analog window comparators, each one conected to an ouput color (R G or B), and to digital potentiometers...   What I got is the same way procesing colors, but in real time, there is no "cycles clock". 

The AND ouput of the 3 windows comparators tell me two things: 0 = no color wanted, 1 = this pixel has the color that you are looking for. 
This single bit, is the input of the pic, so, the microcontroler, just have to sincronize with HSYNC and VSYNC and process this only bit.
It is tricky, doesn't it?

But in fact, every CCD sensor, is analog by nature, then they convert it to a digital domain, so this is not a loss process, thinking that all images in the sensor are analog by nature, I only simplified very much the job of the microprocesor...

So, I got this RGB path ok? then, I had to "reencoded" it to NTSC using an AD725 from analog devices... but, by the way, I put 3 muxes in the RGB path, so, it's very easy to do a video overlay, wich I use to show what de camera is seeing

I hope you understand this explanation, sorry again for my english, but it's usefull to me to practice my writing
regards.

Yeah that sounds good
Might break open my little digital camera now, and hook it up to some comparators and a micro...
I can imagine that with this setup if the Micro-controllers clock is out by 1/100s of a microsecond, it would be disastrous!   

[vixctor]

Not working with digital cameras, ITU605 or ITU601, runs at 27mhz, where each pixel needs 4 clocks (Y, Cb Y Cr).

It never happen that micro controller loss signal video, that is because the cam has it's owm microcontroller, wich calculates the coordinates of the object.  On the other hand, there is a master "brain", with another PIC, who ask the cam for coordinates using I2C.   You might think that the micro on the cam can lose sincronization when the master PIC "talks" with cam, but.... no, there is a trick....

The PIC inside cam is I2C slave, it receives the interruption from the master PIC but doesn't take care off... Just put low the clock signal in order to put in strecth clock the I2C (a feature of microchip I2C slave mode), then, the master has to wait until slave answers...
meanwhile, the PIC in cam, continue to process video signal, and calculates coordinates.  In the blanking lines, the pic checks is there was an I2C interrupt, if then, send to the master the coordinates and release the clock signals.  Since the transfer takes 3 horizontal lines... once the PIC in cam finishes the transmition, it rest 3 to the counter lines, in order to never afect the sincronization with video cam signals.

Also, I put a pin on the PIC of the cam, that set if during a frame, the cam sees the object.  If not, clear this pin, because is useless asking "all" the time via I2C if there is an object in sight.  So, the master, just pool that pin and if set, then proceed to ask the coordinates.