Linking up a camera

[ejohns85]

Hi there,

For a university project I am doing this year, I will be creating a robot that uses a camera to navigate itself around an environment. However, I have little past experience on such a matter and would really appreciate some advice on how this can be achieved.

Mechanically and electrically, my robot is already fully functional, but it lacks its brain, and the camera interface. It will not be required to perform high level computer vision techniques, but will need to sense brightness thresholds - will a microcontroller be powerful enough to do this, or would I need to linke up the robot to a PC? Any suggestions for suitable microcontrollers?

As far as the programming is concerned, I have been warned that Java is too slow for the necessary real-time analysis that will need to be performed, so I would like to use C if possible. Any ideas on how I can capture frames, access pixel data and analyse it, and then send appropriate instructions to the robot's microcontroller, using C?

I'm pretty much a beginner in robotics so any help at all would be much appreciated! Thanks.

[Admin]

What sampling rate do you need?

Java is fine for computer vision - I have used it for an old webcam on a robot. Unfortunately, I do not have any of my sample code to give you since I did it for a class assignment a few years back and lost it. Search the web for webcam processing (not sure what the best word to use for this is) source code . . .

You probably dont want to use a microcontroller, unless you have a camera/circuitry that does all the processing externally - such as the CMUcam. Image processing is very intensive.

What kind of vision quality do you need? There are many lower level sensors (like photoresistors) that can also do brightness thresholding . . . If you dont need good quality, lower resolution cameras will give you higher sampling rates.

[ejohns85]

Thanks for the help. My robot is basically required to detect beacons of light and hence determine its position and navigate around. Photoresistors would not give me a high enough resolution I'm afraid. It won't be moving particularly quickly, so I reckon a sampling rate of at least 10 Hz should be ok.

I have had a look at the CMUcam, but it is not within my budget. From what I gather, it just uses another microcontroller to extract the data from the camera, which can then be sent to the robot's controller. I think this involves reading in a string of serial data containing pixel voltage levels, into an ADC, and then storing it as an array of data to build up each frame. Would it not be possible to buy a microcontroller and program it to do this myself?

Another student doing a similar project used Quicktime for Java to analyse video, but this required almost one second to analyse each frame. Is there a quicker way this can be done in C? Thanks!

[Admin]

are you sure 2 or more photoresistors wont work?
http://www.societyofrobots.com/algorithms_photoresistor.shtml

A CCD camera is basically a matrix of photoresistors . . . with one you can determine the light brightness, with 2 photoresistors you can locate the light, with three you can determine the distance away by triangulation. Most microcontrollers have 8 ADC ports, so you can make a 4x2 matrix of photoresistors . . . Considering your budget, thats damn cheap 

the time it takes to analyze a frame depends entirely on the resolution, the information you need extracted, and the accuracy desired. for extremely fast processing, convert all the images to greyscale (black and white), reduce the resolution (delete every other line, etc.), trim edges, etc - basically delete any info not required.

if all you are doing is locating the brightest light, what you do is save the image into a matrix, scan the matrix for the brightest pixel, then record the x and y location in the matrix of that brightest pixel. there are more reliable ways to do this, but this is the simplest fastest method.

[dunk]

ejohns85,
interfacing a camera directly to a microcontroller is possible but not easy.
i did a lot of reading on the subject and even went as far as looking at video signals on an oscilloscope.
from what i have seen on line there are 2 methods people have had success with.

1. some cameras have a digital output. although the cameras tend to be more expensive there is *far* less work to do.
take note that microcontrollers only have a limited amount of memory so you probably won't be able to capture every pixel.
the CMU cam uses this method.
here's a link to this sort of solution:
http://www.it.lth.se/digp/sammanfattning/2004/lp-4/grupp3/Index.html

2. most cheap cameras give the out put as composite video. there are links out there to people who have successfully interfaced microcontrollers to this sort of camera but this solution is not for the faint hearted.
here's an example:
http://www.seattlerobotics.org/encoder/200110/vision.htm

although it's possible both these methods are a big undertaking.
i'd suggest you get one of these wirless video cameras and send the output back to a PC TV card and do your processing on a seperate computer.
alternatively, if wirless is not a requirement, use a $10 USB webcam from ebay.

most modern programming languages have modules to decode mpeg images.
i have successfully used python as a programming language for video decoding (with similar speed restrictions to your friend using Java) but it should be considerably quicker using C.

good luck!

dunk.

[Admin]

Something I found by accident today:
camera frame grabber modules
http://www.rtdusa.com/PC104/PC104_peripheral%20Module.htm#frame%20grabber

Definitely out of your budget, but potentially useful for someone else . . .

[Nyx]

Kinamax nightvision webcam. I have one, it's great. 640x480 resolution, color view in daylight, black and white  in low light:

http://www.tigerdirect.ca/applications/searchtools/item-Details.asp?EdpNo=1567081&sku=M501-1160

As for the computations, you could carry a laptop. You may already have one, or you could get a cheaper one:

http://www.tigerdirect.ca/applications/SearchTools/item-details.asp?EdpNo=2366112

Benefits include free wireless communication (wifi), alot more power than any microcontroller can possibly pack, alot of memory, the comfort of a real OS and the ability to use any programming language you want.... Plus the laptop has its own power source and can power USB devices such as webcams on its own.

[ejohns85]

1. Thanks for the help - I'm now looking into a camera that provides a digital output. (I don't know how analogue cameras work and I would probably need to buy some extra fast ADCs for it.) In order to record every pixel in each frame, I am considering installing an additional RAM chip that would be loaded with data from the camera when each frame is taken, whilst wiping out the previous frame. Whilst the frame data is in the memory, I could then use the microcontroller to analyse the frame, and after each anlysis, tell the camera and memory to record a new frame. Does this sound feasible?

2. In order to make the frame rate as large as possible, I would have to know how long it takes for my C program in the microcontroller to perform its analysis. How can this be done? I guess I could set a test pin high when the "analyse frame" loop has been done once, and use an oscilloscope (which I don't have!) to determine the time it took, but is there an easier method by inspecting the code, or is it just trial and error? I think that programming in assembly code would make this easy, but I'll have lots of if/else statements and I'd rather use C.

3. I would need to use the microcontroller's clock to time the frame capturing of the camera. But how can a C program use the clock? For example, if I know it takes 0.1 seconds to capture an image, store it in memory, and then analyse it with the microcontroller progam, my program needs to wait at least 0.1 seconds between every time it tells to camera to record a frame. I know there is a sleep() function in C, but how does this work? Does it use the onboard clock, and would it be accurate to the microsecond such that this method would work?

Phew! Any help would be great!

[dunk]

1. so it sounds feasible but maybe it's not necessary.
rather than capturing every pixel and storing it in ram why not just capture as many as you have space for.
this will lead to lower picture resolution but from the description of your project you don't need great resolution right?

2. rather than buying an oscilloscope for this you could use a 2nd microcontroler with a timer module built in. enable an interupt on an input pin. every time  the input is detected, display the value of the timer on LEDs (or via serial UART connection) and reset the timer. you'll find the value for each "tic" of the timer on your micro's data sheet.

when ever i have needed time critical loops i have used machine code and just counted the number of instructions.
the time for each instruction to execute is constant and can again be found in the datasheets.
so i don't have any experience programming in C on microcontrollers so there may be a better way than this:
when you compile the program prior to transferring onto the microcontroller you have the code in machine code.
i know on microchip PICs there is a tool to convert this HEX file back into assembly language.
from here you could count the instructions. (a laborious task but possible if you can follow assembly language.)

3. again, i've never used C on microcontrollers but i don't really understand why you need to do this.
the camera will output data for each pixel. presumably it gives you some sort of output on a separate pin when each pixel is ready? your program will take the data for each pixel and store it. just implement a counter. when your program has stored X pixels the image is complete so the program moves on to the "analise frame" section.
am i missing your point here?

dunk.

[ejohns85]

I have been getting a lot of advice from others that a microcontroller simply wouldn't be powerful enough to process images in the way I am requiring. So I am now looking into using some sort of wireless link between a microcontroller and a PC, so that the PC can be sent data from the camera's output, analyse it with some software that I would write, and then send back instructions to the microcontroller.

Any ideas on how this could be done? Would it actually be necessary to have a microcontroller on the robot as well as a remote PC?

Thanks!

[dunk]

i think that's sound advice. it is possible to get a microcontroller processing video but it's a *huge* task and only suitable for some one with a lot of microcontroller, electronics and programming experience.

here is how i see your simplest video solution:
so if you are going to use a remote PC to process the video you want to use a wireless CCTV camera and connect the receiver output to a TV card on your PC.
here's an example of a unit that transmitts the output from any CCTV camera. there are quite a few on the market:
http://www.maplin.co.uk/Module.aspx?ModuleNo=27680&criteria=Covert%20and%20Wireless&doy=8m8
and the receiver:
http://www.maplin.co.uk/Module.aspx?ModuleNo=32647&criteria=Covert%20and%20Wireless&doy=8m8

when you feed the output from the receiver into the PCs TV card you will then be able to see what ever your bot sees on your computer's TV software.
the next step is to fins a piece of software that can capture single images from a PC TV card.
i used camE for Linux but there's probably a windows equivalent out there.

most modern programming languages have tools to decode jpeg images.
obviously with a wireless camera there will be some interference so you are going to have to apply some sort of software filter.

so using this method you don't need any sort of microcontroller involved in getting the video signal to the PC.
i'd still imagine a microcontroller will be the easiest method of controlling your robot's movments though.

dunk.