Choosing a Microcontroller for Image Processing

[samurai7]

Hi everyone,

Just want to say I've been reading the tutorials/forums on this site for a while, and I find them very helpful. However, I find myself in a bit of a pickle. Hopefully, somebody can point me in the right direction. 

My partner and I are working on a school project which will involve using a camera to detect and recognize crosswalk signals (e.g. white walking person symbol, or orange hand). We'll have to do some interesting shape detection here (which I need to look into more), and possibly colour detection. The issue is that the solution will need to be mobile, and we won't be able to use a laptop (not even close by via wireless communication). We have experience working with sensors, motors, microcontrollers, etc., but no experience with image processing.

We've been researching it quite a bit in the last few weeks (also read Admin's Computer vision tutorial) and it's apparent you need something powerful to do the job. It became apparent that we can't use an Arduino (by searching the forum), as it doesn't have the processing power to do the job. So one option we've looked into is the Mini2440 (http://www.friendlyarm.net/products/mini2440). It's able to run Windows CE 5/6 and Linux 2.6, which could be pretty handy, because it's basically a scaled down notebook. This guy looks like it can do the job, but there doesn't seem to be too much support for it. I was also doubting my ability to use Matlab on this device (I know how to use Matlab, but the Mini2440 doesn't meet minimum req.). We'd have to use/figure out OpenCV and work it from there. We've looked into other boards as well, but again, not sure if they would do the job (e.g. the Jackrabbit BL1800). We have a limited time to do this project, so we wouldn't want to take up too much time learning how to use the board we choose (and money buying it if we find out it doesn't fit our needs).

Also, would the Blackfin Camera be useful for our needs? I know you query the camera/board and it does the processing for you, but does that mean it can find more complex shapes like the crosswalk signals?

But then we stopped and thought, "How is Admin doing it with the Axon?" I'm not really sure what defines "enough processing power" to do image processing on an MCU, but if the Axon can do it by itself, I'd be curious to find out.

Also, what is the general way to interface with image data? Do RoboRealm/OpenCV allow you to extract pixel matrices and play with that data? I'm sure it has library functions that help with that, but I'm just curious.

If anyone has any advice on a board we can use, or can explain what is needed in a board to carry out image processing, it would be greatly appreciated! I'm very excited to be working with computer vision, but just need a good push in the right direction. 

Thanks for all your help everyone! (sorry for the super long post)

[klims]

I have always thought the RoBoard and a webcam (or two) would be a dynamite combination for a robot project. You would be running windows or linux so you could use OpenCV or RR.

I have messed around with OpenCV a little and I think your biggest problem will be compensating for changing lighting conditions because you will be outdoors. You should start your image processing as soon as possible and get it working on the bench in a well lit room before you go out doors.

But then we stopped and thought, "How is Admin doing it with the Axon?" I'm not really sure what defines "enough processing power" to do image processing on an MCU, but if the Axon can do it by itself, I'd be curious to find out.

I believe admin uses a blackfin to do the image processing and the Axon just interprets the output from the blackfin.

Also, what is the general way to interface with image data? Do RoboRealm/OpenCV allow you to extract pixel matrices and play with that data? I'm sure it has library functions that help with that, but I'm just curious.

OpenCV is just a library. You need to write code to use the library and do useful things with the functions.
RR is a GUI that allows you to do image transformations and do things with that information. GUI will use more system resources.

Personally I think a Blackfin and a microcontroller would be the best option.

How long do you have to do the project?

[samurai7]

Hi klims, thanks for the response!

The RoBoard + webcam sounds awesome, but perhaps a bit too expensive for our sponsor to approve.

I have messed around with OpenCV a little and I think your biggest problem will be compensating for changing lighting conditions because you will be outdoors. You should start your image processing as soon as possible and get it working on the bench in a well lit room before you go out doors.

That's a good point. Just testing and making sure it works indoors will save a lot of headache and debugging versus going straight outdoors.

I haven't done any serious coding (excluding simple C for Arduino or Autoit) for about 2 years now, but if I were to utilize OpenCV with the Mini2440, I assume I'd simply grab an IDE (e.g. I have a free license for Visual Studio), and start by writing code to interface with the camera (later moving on to process the image data). I know C++ as well as C, which would you recommend? Also, I'm assuming we should be able to develop the code on a laptop, and later build/deploy it onto the Mini2440 with any libraries we'll need. Does RoboRealm allow you to deploy programs or modules in this fashion?

Personally I think a Blackfin and a microcontroller would be the best option.

How configurable is the Blackfin camera? Would I be able to utilize it to identify complex shapes such as crosswalk signals, or maybe to extract portions of an image/matrix for comparison with something in memory (such as pictures of crosswalk signals)? As for the "microcontroller", is that something like an Arduino?

Sorry, I tend to babble.. 

How long do you have to do the project?

I have until the end of March to do the project. Hopefully it's enough time.

Thanks again

[klims]

I don't know how capable you and your buddy are but if this is a uni project  and you have till march I think you would be lucky to even get anything working using the Mini2440 and a camera, let alone having a completed project. You need to remember other classes and assignments.

This is why I was saying to go for a simpler option that might cut down the amount of stuffing around you need to do. And there is a lot of stuffing around getting OpenCV working properly if you aren't an experienced programmer.

I don't own a Blackfin yet so I can't give you too much info on what it is capable of. Maybe someone who owns one will be able to help. Worst case get onto their forum and ask the question.

C++ is definitely the way to go if possible.

I haven't used RR much or even seen what it is at these days, but last I saw no it cannot. I don't even know if it's compatible with windows CE.

Maybe if you tell us a little more about your project (hardware and software) we might be able to offer some more help.

Good luck!

[samurai7]

You make a really good point about the time constraint. Even in my own experience, things don't always work out on the first try, and you end up wasting time in between.. hmm..

It seems like the Blackfin Camera is the way to go, hopefully someone who has used one can add some comments about it (I'll look into it more as well).

As for my project, there's not too much more to it. The camera is to be placed on a white cane, and the aim is to aid people with vision impairment at intersections. We plan to implement some sort of audio or speech output to notify the user when it is safe to cross the street. We will also be using the camera to detect drop-offs (e.g. stairs or a curb) and warn the user accordingly. I'm sure whatever controller option we choose for the first objective should be able to handle drop-off detection.

The hardware would likely just consist of the camera module and microcontroller we choose (though I already own an Arduino). We were initially planning to use ultrasonic sensors for drop-off detection, but our professor advised against it because the accuracy and consistency would be difficult to achieve.

Thanks for the advice, let me know if there's anything else I can clarify. 

[superchiku]

well...for image processing you need a pretty powerful processor...mostly what i do is that i run the code in my laptop and attach a wireless cam to the robot, remember the wireless cam should have good range...then what i do is that , i run the code in the laptop after receiving the images and then transfer back commands to the robot's microcontroller using either bluetooth or wireless...

it sounds messed up when u first read about it but when you actually implement it, ull realise that its quite useful...because the processor of the laptop can process the images much faster than any microcontroller......

the only disadvantage is that the robot can only work within a specified distance from the laptop because the wireless cam has a fixed range....

another solution is using a netbook and attaching it with your robot....., the netbook is small and portable so it wont take much space on your robot........


and yes if you use a laptop or a netbook then try to use opencv coz its fast and powerful...i use that... also vc++ gives you an advantage of communicating with your robot via the parallel or serial port...

[samurai7]

Hi superchiku!

That sounds like a really good idea, and I'd love to try and implement that. Unfortunately, I don't think it's feasible for our situation, because we're not actually making a full-fledged robot. The body is literally a white cane for people that are legally blind. Nobody would walk around carrying a netbook, and it would be difficult to walk around and swing a cane with a netbook attached to it .

The only alternative I can think of is using a cellphone. We could either transmit data to it for processing, or use the built-in camera and just do everything on the phone. Alas, I have no clue on how to develop an application for a cell phone that could do anything like that (I wouldn't have the time, but check out this link!=>http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2856957/).

Also I've looked into the Blackfin Camera more (e.g. Admin's tutorial on it=>http://www.societyofrobots.com/electronics_blackfin_camera.shtml), but I couldn't seem to find out more information on the features/functions it provides. Are the functions in Admin's tutorial only available when using the Axon? Can I manage to use the same functionality using an Arduino, or would I have to write some functions like the ones described by Admin to get it all to work? I'm trying to look into it, but can never seem to find the right piece of documentation from Surveyor Corp. I already have an Arduino Duemilanove, so I'd prefer not to buy an Axon to get it done (tight budget).

Ah well, as I said, I tend to babble. Thanks for the responses guys, I really appreciate the help.

Cheers

[klims]

The only alternative I can think of is using a cellphone. We could either transmit data to it for processing, or use the built-in camera and just do everything on the phone. Alas, I have no clue on how to develop an application for a cell phone that could do anything like that

I was just thinking that exact same thing!
Buy yourself and Android phone, or a small tablet running linux and your set! Something like this http://www.dealextreme.com/details.dx/sku.31523 but with a camera, or maybe look into getting a webcam to interface.
I have tried to code a little in Android and it's not as easy as I thought it would be. It's been a good 5 years since I did any coding in Java so that probably doesn't help

[samurai7]

hey klims,

That sounds like an idea! At this point I'd really prefer to use Matlab or Octave, because it would probably minimize our time on "stuffing around". So if I could run Octave/Matlab on something like that or the Mini2440, and get pictures from a wireless camera, everything should work out. It's either this option, or the Blackfin + Microcontroller.

Hopefully someone knows if the Blackfin can handle Image Correlation + Edge detection.

Thanks!

[klims]

At this point I'd really prefer to use Matlab or Octave, because it would probably minimize our time on "stuffing around". So if I could run Octave/Matlab on something like that or the Mini2440, and get pictures from a wireless camera, everything should work out. It's either this option, or the Blackfin + Microcontroller.
Thanks!

I've done a little matlab coding and from what I remember it was not very efficient. Not sure about Octave. I think you might struggle to get it running well on a micro, but I could be wrong.

You can definitely make a start on edge detection and object recognition in the lab. Plenty of algorithms out there (even some in Open CV) to do it. I think you might have a hard time determining what is a ledge compared to just any other horizontal line using only image processing.

[Soeren]

Hi,

Nano-ITX is a PC in 7cm x 10cm, which runs most of what other PC's can, so you probably have a lot of experience already to help further your project in a way that won't happen if you have to fight a microcontroller.

[klims]

I really think you should just use a netbook and a webcam for this project. You can tell your professor some story about using a netbook because its something they could carry around conveniently and most people already do.
That is unless there is some kind of hardware requirement for the project...

[samurai7]

Hi Soeren, that sounds like a good idea! Some of those motherboards aren't too expensive, and they're more powerful than a mini2440. After some initial research, it seems others have definitely taken this route, but usually using RoboRealm. I'm assuming it isn't too big an issue to boot Windows (e.g. XP) onto a nano.
Thanks for the tip!

Hi klims, I agree with what you're saying. We're first going to work out our solution using a laptop + the webcam. Then we will attempt to work it out on the Mini2440 (or nano-ITX board) if we are successful and have time (*knock on wood*). This way we have a safe fallback plan. Our professor said it is okay to go with the laptop, just to show that our idea is feasible, but we know it would be impressive to do it without the laptop.
Thanks for the recommendation!