how to find the positon of a object

[roboticwater]

Hi,

I am doing a program to move a robotic arm to the destination only with one camera. Now I can process image to extract three color blobs sticked on the joints of arm and ignore the background . I also got the center (x,y) position of each color blob on the image. How can I get the actual position of each blob? (Is my step correct?) I have read through the tutorial of Stereo Vision here, but I still cannot understand the equation for one camera. I hope someone can help me out. I am really a beginner on COMPUTER VISION.
x_camL = focal_length * X_actual / Z_actual
y_camL = focal_length * Y_actual / Z_actual what is the x_camL and y_camL here (the position of the camera in which coordinates)

Thank you very much!!!!!!
jerry

[sdk32285]

Im not sure i understand your question. But is sounds like you need to calibrate your camera so that you can relate pixels to real world distances.

[roboticwater]

Oh, sorry for the confusing question. I need to know the actual positions of three key points (the elbow joint, the end-effectors and the destination). Because I can find the positions in the image coordinates. What is the next step I need to follow for making the robotic arm to move to the destination automatically and correctly by using only one camera.

Thank you very much anyway!!!!!

[Admin]

only with one camera . . . I have read through the tutorial of Stereo Vision here, but I still cannot understand the equation for one camera.

So the problem is that you are using equations for stereo vision (which requires two cameras) with only one camera

You will not be able to get depth reliably with one camera.

As for the X/Y locations . . . get out a yard stick and measure where the object is, and compare that to the X/Y pixel position that the camera says.

So you might get 5 inches with a camera reading of 2 pixels. So thats 5/2. I usually just fudge this cause I never need it to be accurate.

So if one day your camera says the object is at 8 pixels, then:
5"/2 pixels * 8 pixels = real location of the object in inches.

This is a robot Im working on that sounds similar to yours:
http://www.societyofrobots.com/robotforum/index.php?topic=1218.0
It might give you some ideas . . .

As for your robot arm questions . . .
http://www.societyofrobots.com/robotforum/index.php?topic=1403.msg9235#msg9235
(try not to double post your questions)

[roboticwater]

Thank you very mucn Admin. I have read through links you gave me, and it really helps me.
Another quick question
Is there any method that can get more accurate result from a 2D image?  How to  "calibrate the camera so that I can relate pixels to real world distances" ?

Thanks a lot

[Admin]

Is there any method that can get more accurate result from a 2D image?  How to  "calibrate the camera so that I can relate pixels to real world distances" ?

Yeap. The basic method I use is to get a big white poster board and draw a grid of 1"x1" squares on it. Then using a camera image, hand count the number of pixels per grid square to get your ratio. Just zoom in on the image to do this.

The above method isnt perfect because it doesnt take in to account the 'fish eye' effect of the camera lens, but its good enough for a beginner.

For 3D, using 2 cameras in stereo, I use lego blocks positioned in a special order. The size of a lego block is exact and known, then I just use special calibration software to calculate the transform for me.

You may be interested in this post:
http://www.societyofrobots.com/robotforum/index.php?topic=1677.0
(you both seem to be doing the same thing)

[roboticwater]

Thank you so much Admin!!!! I think I am a upper-level beginner who knows the direction now , but still a beginner .

I am trying to use two camera to get a stereo vision instead. I have two non-Parallel Cameras (rotation about Y-axis). In your tutorial, I have to caltulate Zo which  eaquals b / tan(phi) and then Z_actual = (b * focal_length) / (x_camL - x_camR + focal_length * b / Zo). Does the equation work when Zo is (0,0,0) or angle "phi" is 0  (I want to use the other camera (cR) to check whether the target and robotic arm are in the same plane).
could you please also recommend some "calibration software" to calculate the transform?

Thanks a lot!!

[Admin]

(rotation about Y-axis)

Do you mean X or Z? If its the Y axis, your cameras are crooked for no reason at all . . .

Does the equation work when Zo is (0,0,0) or angle "phi" is 0  (I want to use the other camera (cR) to check whether the target and robotic arm are in the same plane).

Assuming you meant Z axis . . .
Well Zo = b / tan(phi), so what happens when phi is 0?
tan(0) = 0, meaning you will divide by 0 causing the world to end.

The same for Zo=0.

If you meant X axis, just plug in numbers to see what happens yourself.

could you please also recommend some "calibration software" to calculate the transform?

Im using custom software developed by Peter Madden of the Lauder Laboratory at Harvard University. I recommend just googling around.

[RoenZ]

Wew.. i think we same roboticwater.... i'm doing the stereo vision too..... but in my stereo i used the parallel axis.....with the baseline 10cm.... why u try the non-parallel axis??? I think it's more difficult...... and what camera did u buy??? I'm using the Logitech Quickcam Pro 4000.....

 

[roboticwater]

Yeah RoenZ! we come together right here finally ! My situation is to know how many degrees that the base of the robotic arm need to turn to make the robotic arm and the target in the same plane, and then calculate the positions of each joint together with the target point in their plane in unit of centimeters. I choose the base of the robotic arm as the original point(0,0,0).

So you mean parallel axis of cameras should work, right?

How did you solve the problem as you asked in your thread about the unit of (x_camL - x_camR). I think x_CamL and x_CamR are the position values of target point P in left image and right image respectively, right? I just change the pixels coordinates of x_camL and x_camR to cm units for keeping them consistent as Admin told us. (I use photoshop to find the coordinates in cm unit). Am I right?
I'm using Philips ToUcam pro

Thanks

[RoenZ]

Yeah too roboticwater..... but we are in different applicarition but same in stereo vision......
I've still no idea how to change the pixel coordinates of x_camL and x_camR because it's need the fundamental matrix....i mean i must know the intrinsic parameter of the camera. The Logitech don't give me the parameter that i need to calibrate my camera (the CCD chip form factor) Does anyone know it???

I've tried the experiment and i found that the disparity vs distance will be in the exponential function.......
e.g : If the disparity = 52 pixel so the distance will be 70 cm from the camera but the stereo triangulation equation will be not valid more.....

And if u change the pixels coordinates of x_camL and x_camR to cm units for keeping them consistent as Admin told us....like this:
I captured the image from the camera with 320x240 pixel and i measured it in Photoshop. The measure in cm will be 11.32 cm x 8.5 cm.
I set up the object with the distance of 70 cm from the camera and i checked in the photoshop --> xCamL=6.35 cm | xCamR=4.55cm so the disparity will be xCamL-xCamR=1.8 cm.
So Z=(f*b)/(xCamL=xCamR)

[RoenZ]

So Z=(f*b)/(xCamL=xCamR)
      =(0.45*10)/(1.
      = 2.5 cm

U can see the Z must be 70 cm not 2.5 cm.........

Btw what progamming language did u use?? I used Delphi..... 

[roboticwater]

U can see the Z must be 70 cm not 2.5 cm.

You are right. I am confused now
I use Java.

Does anyone can help us?? Please......

[Admin]

(rotation about Y-axis)

Do you mean X or Z? If its the Y axis, your cameras are crooked for no reason at all . . .

I cant help you unless you tell me which axis the non-parallel cameras are rotated around.

I suspect you are using the wrong equation, or using the correct equation incorrectly . . .

[roboticwater]

Thanks Admin!!!! I think I have to summarize my queitions as below now.

1,

I need to know how many degrees that the base of the robotic arm need to turn to make the robotic arm and the target in the same plane, and then calculate the positions of each joint together with the target point in their plane in unit of centimeters.

In my application above, the parallel axis of cameras should be ok, right? As RoenZ said parallel axes is simple, so I choose parallel axes at last.

2, The equation for parallel axes should be below:
Z_actual = (b * focal_length) / (x_camL - x_camR)
X_actual = x_camL * Z_actual / focal_length
Y_actual = y_camL * Z_actual / focal_length, right?

3, Does the value of focal length influence the result much or not? Since I can only use the minimum focal length of my camera provided in the user manual.

4.  "x_CamL" and "x_CamR" are posions of point P in left image and right image right? For keeping them consistent, we can change the pixel units to e.g. cm units, right?

5. In RoenZ's reply #10 and #11, could you please explain why the result seems making no sense. Where actually should the original point be?


Thank you very mcuh!!!!
 

[Admin]

As RoenZ said parallel axes is simple, so I choose parallel axes at last.

That makes life easier

Does the value of focal length influence the result much or not? Since I can only use the minimum focal length of my camera provided in the user manual.

Yeap it has a huge influence actually. You can calculate the focal length if you have the numbers to everything else in your equation.

For example: focal_length = x_camL * Z_actual / X_actual

4.  "x_CamL" and "x_CamR" are posions of point P in left image and right image right? For keeping them consistent, we can change the pixel units to e.g. cm units, right?

x_CamL and x_CamR are pixel locations while X_actual is in normal units (such as cm)

5. In RoenZ's reply #10 and #11, could you please explain why the result seems making no sense. Where actually should the original point be?

The reason for the error is that he used 'xCamL-xCamR=1.8 cm'. Its not a subtraction of distances, but instead should be a subtraction of pixels.

[roboticwater]

Thank you very very much Admin for your patience!!
I' v got another problem! Because of the requirement of my application, I have to use single camera to get robot localization.
So the problem is how to set up an experiment to construct a lookup table with a discrete set of known sample points in image which represent the real joints of the robotic arm, and then interpolate a unkonwn point between them to get the approximate position in real world. The distance between robotic arm and the camera is fixed and the arm can rotate 360 degree.
Is that possible??? 

[Admin]

I have to use single camera to get robot localization.

I assume you meant robot arm end-effector (grippers) localization?

Why dont you want to program in inverse kinematics?

Hmmmm if you had only one camera, then how would you know the depth of the object your arm wants to grab?

how to set up an experiment to construct a lookup table with a discrete set of known sample points in image which represent the real joints of the robotic arm, and then interpolate a unkonwn point between them to get the approximate position in real world.

Ive been considering doing this for one of my robots too. But Im not sure how well it will work . . . First you must manually create the lookup table. Make your robot go through a bunch of different positions in a grid pattern, then record the angles of each position. Then to interpolate, first find the object location, then try to find in your lookup table four arm positions that are the closest. This is because that object is somewhere in a grid with four corners. In 3D space, you would need to find the closest 8 points (a cube). Then average the robot arm joint angles of each grid corner (in the lookup table) weighted by the closeness between each grid corner.

But Ive never done this yet, so not sure how well it will work . . .

And . . .

You can find the location of the arm like this:

I would put a red ball on the end of the robot end-effector.

Do a blob measurement. By measuring the size of the ball you can determine the z-axis (depth away from camera). And by finding the x and y axis you can find those coordinates too. But finding the z-axis of the object you wish to grab will be really hard with one camera. Probably still hard with two . . .

[RoenZ]

Wew i finally found the depth....
First--> u can't measure the pixel with the photoshop because the photoshop have a default constanta to convert it to cm. e.g: 72 pixel/inch
Second --> the coordinate (0,0) of the image located in (160,120) in 320x240 pixel.
Third --> After u measured it the disparity in cm will be know ( i mean the constanta in pixel/cm)

But after i got the depth, i confused how to detect what is the background and what is the object.... 

-RoenZ

[Admin]

But after i got the depth, i confused how to detect what is the background and what is the object....

This is actually an active area of research among experts . . . its not easy, nor a fully solved problem.

My computer vision tutorial goes over the basics . . . but it only works for simple shapes and colors.

I recommend cheating by simplifying either the object or the background. Or preprogram just a few shapes (with dimensions) in to memory and have the microcontroller decide which is probably the correct object that it sees.

[roboticwater]

I would put a red ball on the end of the robot end-effector

Yeah! Admin. Your method sounds a clever way!!!! I think it is much  better than recording so many positions (It is really hard to keep measurement precise during the process of the experiment).

then record the angles of each position

But in your explain, what do you mean the angles? I have a joint on shoulder, and a joint on elbow. 

I assume you meant robot arm end-effector (grippers) localization?

Yes, I do mean robot arm end-effector locallization. I also use inverse kinematics learn from your tutorial. That is great because I only need to know the position of the target and the end-effector right?

[Admin]

But in your explain, what do you mean the angles? I have a joint on shoulder, and a joint on elbow.

The angle of each DOF (at each joint) that results in a certain end effector location.

That is great because I only need to know the position of the target and the end-effector right?

yeap!

[RoenZ]

Yeah i make it simple... for all of the obstacle i colored with black and for the shape is cube and the wall is white.... but i didn't get the smooth segmentation... i mean i didn't get a shape of cube after i segmented it??? Have any simple algorithm to get a smooth segmentation??

[Admin]

i mean i didn't get a shape of cube after i segmented it??? Have any simple algorithm to get a smooth segmentation??

What do you mean? Are you trying to 'see' 3D shapes? What software are you using?

Its easier to just see the object, identify it by color, and 'guess' the shape knowing what you preprogram into your robot.

In reality humans do the same thing. We have a large memory database to make assumptions about what we see.

Try this:
http://groups.google.com/group/sci.image.processing/browse_thread/thread/4bf8e5d78e2fede3/ff8e34a67cf0716c%23ff8e34a67cf0716c

[RoenZ]

Thx for the link... before that i want to ask about the stereo triangulation.... u can see at the picture below





i'm confused about the similiar triangle equation, it marked in blue colour. Why xL/f=(X+(b/2))/Z
Thx

[Admin]

The images you posted are too tiny for me to read . . .

What part are you confused about?

I did my own write up of stereo vision, this might help:
http://www.societyofrobots.com/programming_computer_vision_tutorial_pt3.shtml#stereo_vision

[RoenZ]

Sorry for the tiny image.....this is the bigger one....


i marked it with red one.... thx

[Admin]

i'm confused about the similiar triangle equation, it marked in blue colour. Why xL/f=(X+(b/2))/Z

Hmmmmm your englarged image doesnt look like the smaller images, nor does it have a blue circled equation that you asked about . . .

Those are simple equations, just plug in the distances as shown in the image. For example, look up the datasheet of your camera to find f, the focal distance. And simple measure the distance between cameras to find T. Although I think they made a mistake in that red equation because they misdefine T . . .

Just use my equations because they are well explained

[RoenZ]

Hmmmmm your englarged image doesnt look like the smaller images, nor does it have a blue circled equation that you asked about . . .

Sorry for the different image....

Although I think they made a mistake in that red equation because they misdefine T . . .

yup ur right, i thinked they made a mistake.....