Running OpenCV on a uc

[stopgo]

I'm interested in color and motion detection.  I want a more reliable way to detect color than photoresistors, and I want to use the camera for motion detection in combination with other sensors for extra reliability (using a Kalman filter or somesuch).

I'm new to the microcontroller world.   Do you guys think running OpenCV on a microcontroller is out of the question?  Can anyone point me to a good uc to use for vision?  I would imagine I want a DSP for this, but I know very little about DSPs.

Edit:  I can't imagine I want to do more than the Mars Rovers, and they apparently operate at 25Mhz, with 128MB RAM and 256MB Flash.

[hgordon]

It would seem possible, as OpenCV runs on Linux, and there are linux distributions available for ARM7, ARM9 and Blackfin processors.  See -
    http://opencvlibrary.sourceforge.net/InstallGuide_Linux

The main issue you will encounter is that OpenCV uses floating point math, and most microcontrollers only have fixed point multipliers, so you will either have to convert some OpenCV code or run floating point emulation on those particular processors, which is probably quite slow.  Also, I don't know how large the library is, though it can probably be trimmed down to a few megabytes.

Bottom line - if you are familar with embedded programming, code size optimization, floating point emulation, etc, this is probably an interesting project to undertake.  If not, you probably want to use a full-size Intel compatible processor that has floating point and lots of memory.

In any case, to get an idea of what vision algorithms you will need, try www.robotrealm.com - that is a good starting point for image processing software.

[Steve Joblin]

If you are looking to add "image processing" as a sensor, I strongly suggest you check out www.roborealm.com

[stopgo]

It would seem possible, as OpenCV runs on Linux, and there are linux distributions available for ARM7, ARM9 and Blackfin processors.  See -
    http://opencvlibrary.sourceforge.net/InstallGuide_Linux

Ah - this will be fun.  I've never played around with an embedded Linux before.  It will be interesting to see how expensive in terms of memory resources something like uclinux is.

The main issue you will encounter is that OpenCV uses floating point math, and most microcontrollers only have fixed point multipliers, so you will either have to convert some OpenCV code or run floating point emulation on those particular processors, which is probably quite slow. 

This is a good point, but I think I can avoid emulation.  There are apparently DSPs specially tailored to handle floating point ops (usually in 1 cycle).

Also, I don't know how large the library is, though it can probably be trimmed down to a few megabytes.

Yeah - I was thinking I would just hack out everything I don't need.  Now I'm actually wondering how difficult it would be to hack out the library's reliance on an OS altogether.  I guess I'd have to run through the code and replace all the syscalls.  That seems like a pretty daunting task.

Bottom line - if you are familar with embedded programming, code size optimization, floating point emulation, etc, this is probably an interesting project to undertake.  If not, you probably want to use a full-size Intel compatible processor that has floating point and lots of memory.

You know, I think it's possible to extend the amount of memory the uc has access to by adding external RAM and flash, but I haven't looked into how one does that yet.

In any case, to get an idea of what vision algorithms you will need, try www.robotrealm.com - that is a good starting point for image processing software.

As Steve Joblin added, I think you meant roborealm.com.  It looks quite useful.  Thanks for all the feedback guys!  I'll post back once I figure out which uc I choose.

[hgordon]

I suspect that the OS interface in OpenCV is pretty minimal.  There might be some memory alloc calls, or the library may require that the host program handle buffer allocatons.  You'll just have to look at the code, but given that it's used in Windows and Linux, this probably isn't a big issue. 

The floating point will be more of a challenge.  You might want to check the Texas Instruments Da Vinci processor - I believe it has an ARM9 core that runs linux and a TMS320 DSP core that does floating point.  I personally have never been fond of TI architectures, but maybe this one is easier to program.  Also, I have never found the lack of floating point to be a fundamental obstacle, instead using lookup tables for transcendentals or careful fixed point math to keep track of the decimal.

[dunk]

there are a few libraries out there that use far less resources than openCV and are more portable.
have a search. i haven't seen much info on libraries specialising in embedded processors though.
i'm currently using http://cimg.sourceforge.net/ on a Intel XScale ARM processor with good results.

if you want to do image processing on a more basic microcontroller i think any standard libraries will be too large to be of much use.
have you seen the http://www.jrobot.net/Projects/AVRcam.html for an example of doing image capture on an AVR?

the down side with roborealm as i see is it is you would have to either run windows on your robot or do your image processing remotely on a windows PC somewhere.
there's been a lot of buzz on the forum about roborealm but i don't really get what people are using it for (apart for airman00 who has bolted wheels to his PC....).
maybe i should install windows somewhere and have a play...

dunk.

[hgordon]

there are a few libraries out there that use far less resources than openCV and are more portable.
have a search. i haven't seen much info on libraries specialising in embedded processors though.
i'm currently using http://cimg.sourceforge.net/ on a Intel XScale ARM processor with good results.

Good point.  Another useful library is CVIP Tools - http://www.ee.siue.edu/CVIPtools/ .  In the past, I've been able to pick through the source code to find the specific functions I need.

[stopgo]

Thanks for all the feedback guys!

hgordon, may I ask more specifically what you've disliked about TI DSPs?  In what ways have you found them difficult to program?  Poor GCC support?  Poor documentation?

If I decide to go with fixed-point, the Blackfins look pretty snazzy.  http://blackfin.uclinux.org/ also looks like a rather convenient source of information for a newb like me.

I guess I should really map out what algorithms I expect to use so that I can decide how much floating-point emulation would matter.

[hgordon]

hgordon, may I ask more specifically what you've disliked about TI DSPs?  In what ways have you found them difficult to program?  Poor GCC support?  Poor documentation?

If I decide to go with fixed-point, the Blackfins look pretty snazzy.  http://blackfin.uclinux.org/ also looks like a rather convenient source of information for a newb like me.

It's an old bias.  The TI processors always had interesting features and powerful processing, but it always felt like the features were kludged together.  A good example is the new TMS320DM355, which has all kinds of whiz-bang capabilities, but they're scattered all over the chip, and I have yet to find documentation in one place that shows me how it all goes together.   

In contrast, all of the features in the Blackfin are fully integrated into the processor core, the functions are easy to access, and the architecture makes sense.  We have developed a board around the Blackfin BF537 (see http://www.surveyor.com/blackfin/ ), and I have found that this processor is a joy to program.  In a lot of ways, the Blackfin is similar to the ARM7 in the elegance of the architecture, but there's a LOT more horsepower.

[Admin]

Edit:  I can't imagine I want to do more than the Mars Rovers, and they apparently operate at 25Mhz, with 128MB RAM and 256MB Flash.

I believe the time between sensing and acting on the mars rovers is like 20 minutes (I can't remember the exact time, but it was really high) . . . so don't be too impressed

I guess I should really map out what algorithms I expect to use so that I can decide how much floating-point emulation would matter.

I would say don't even bother with floating point, and just accept the rounding errors. Basically trade accuracy for higher speed processing . . . unless you want to build a mars rover

[stopgo]

You guys make a good point - I'd take a big clock speed hit if I went for a DaVinci.  At this point, I'm thinking I want a Blackfin.  The BGA is a pain, though.  I've been planning on building a PCB router, but I think something with that many traces in such a small area seems like it would be pretty difficult to do, unfortunately.

Anyway, I've been scoping out the surveyor cameraboard, and it's looking pretty optimal

Now I just need to do some reading up on cameras.

[hgordon]

You guys make a good point - I'd take a big clock speed hit if I went for a DaVinci.  At this point, I'm thinking I want a Blackfin.  The BGA is a pain, though.  I've been planning on building a PCB router, but I think something with that many traces in such a small area seems like it would be pretty difficult to do, unfortunately.

We use the sparse-BGA version of the BF537 and were able to route everything in 4 layers.  I believe the the higher density BGA chip needs at least 6 layers.

As regards cameras, we use the Omnivision AA board format, which is a fairly standard configuration for camera modules.  Besides the OV9655, we've tested with the OV9620 and OV7640 cameras.

[stopgo]

Do you know of any programmers cheaper than the ICEbear?  Or any DIY schematics for making one?