[ASK] Neural Net for Robot Vision

[marteenjwiil]

Hai all,

Now I'm working my final project about robot vision using neural network (back-propagation method).
I'm using CMUcam3 camera for the robot and for neural net training i using off-line training (using my PC computer to get the weight).
The purpose of my project is to detect color object.

My question are:
> Is there anyone of you had experience using neural net for robot vison?
> If there was, what kind of problems I must face?

 I using RGB color image, and for the input of neural net i grab every pixel value of using 10x10 pixel resolution. 

Thx

[robotcoder]

Not sure what Neural Net is, but I have been researching using opencv to do facial recognition.  I believe there are examples that allow you to follow certain colors.

[marteenjwiil]

Neural Network is like a system that work like human neuron, it will get input and process it to get some output. We get input and then inform the system what kind output we desire for the input. Like human neuron this system can learn and determine what kind of output that  match with the input.

[guncha]

In high school I did this project-paper-thingy about NN, but it wasn't about robot vision. It was about image compression. A la take a typical 128-32-4-32-128 feed forward network and train it so it would output the same thing as was input to it. After it has converged, first three layers (128-32-4) make the encoder and 4-32-128 is the decoder and activation from middle 4 neurons represent the compressed image. Here are a couple of things I learned:

1) NN is no magic tool. It will do whatever you ask for it as best as it can, but no more. Most of the time it will give you an average solution compared to using conventional techniques.
2) It is VERY computationally intensive. Especially when we are talking about image processing with any reasonable resolution. Floating point math and nonlinear transfer functions is a never-ever on a MCU.

Still, it is pretty awesome that you can teach a robot to do certain things without changing a line of code. Feels kinda the right way to go. Also I would recommend sending the image from camera to a computer who does the learning/computation and then sending back the commands to the robot.

So, yeah, it's definitely an interesting project to do. It would be real nice if you could keep us posted on your progress.

Amm, out of curiosity, what kind of output are you training it to give you ?

[guru]

Computer Neural Nets are like math function approximators. The NN forms a mathmatical function for each output based on the inputs. The advantage of NN is that they figure out the mathmatical function when the you dont know what it is yourself, but you have a known set of inputs and outputs to build it on. Some people tack a trainer on thier NN to make a closed loop and thus continue to "train" the NN while it is in operation. Other NNs may only require initial training and then once its done its done. For example, an NN could implement inverse kinematics, and once its trained there is no reason to keep training it.

Comparing artificial NNs to the ones in our brain there is no comparison. Our brains neurons are still *way* more advanced, lots of other processes involved. For example, our brain neurons are also temporaly encoded (so to speak), the specific relative time a neuron fires is important.

C

[marteenjwiil]

This is description of my project:
1. First, the camera will track color object (I use 2 color yellow and blue to tell the camera that there are 2 possibility of object, yellow for furniture and blue for doll)
2. I using 2 servo for the turret so the camera can move Right to Left and Up to Down.
3. If the camera detect yellow or blue object, it will stop in the direction which have biggest number of pixels and density.
4. After that the camera will grab 10x10 pixels that place near the centroid of x and y of color object. (from: (centroid_x - 5) pixel to (centroid_x + 5) pixel and same for y direction)
5. after grab the RGB color, I convert it to grayscale and that divide with /255 and than put it for neural net input
>> I using back propagation method

All of the training process will do off-line, but i plan to get the training weight result and put in the camera so the input will process using neural net directly in CMUcam

[guncha]

Soo, what purpose neural networks serve in your project? Do they try to recognize what kind of object the camera is pointing at ?

[marteenjwiil]

yes, the neural network is for object detection.
The neural network will try to recognize what the shape of the object.

[guru]

The quality of the NN depends on the quality of the input too. So I wonder if it would be of more benefit to have the inputs in the frequency domain instead of pixel space? This means taking the 10x10 pixel matrix and doing a fast fourier transform. (Convert FFT using B&W image.) FFTs on a pixel matrices recognizes edges and region of continuity. You may find this input and the input from the pixel matrix produces better recognition.