Society of Robots - Robot Forum

General Misc => Misc => Topic started by: paulstreats on May 28, 2008, 07:34:35 AM

Title: Universal Objects Database
Post by: paulstreats on May 28, 2008, 07:34:35 AM
Eventually everbody meddling with robotics wants to be able to detect objects such as cans of coke etc.. in order to do stuff with them.

This requires a lot of thought such as storing known physical dimension, colours, decals, weights, all of the possible properties and attributes of the objects. Aswell as this it is also a good idea to think about a way of storing methods for that object (like the method of opening a can and pouring it).

This is time consuming work to actually make yourself a standard way of defining all of the above and implementing an easily searchable database structure for it all.

You also have to think about what sensor might be used to detect it, so the database has to work with cameras or rangefinders etc (rangefinders can extract object shapes meaning that they can search a database and get a shortlist but with no color data, the object search isnt going to be precise but is still a functional tool).

Once you have the database structure made, youve then got to populate it(this could be done autonomously by a robot programmed to detect things with as little intervention as possible or wholly by the human user). This whole process is also going to be very time consuming for any one person alone.

Also we need to think of other storage methods so aswell as building a 3d cartesian style map of the object dimensions, maybe we can also add a 3d kohonen feature map for an additional way of storing the data

Im suggesting that to be a progressive community withiin robotics that we should all look to collectiovely make an robotics standard of object storage and retrieval and also create a basic object library so when people do reach this stage, they have a basic object set to work with.

There seems to be a lack of robots around the household to help with daily life and i think that the whole object definition and methods etc.. are the primary reason behind why there hasnt been much progress in the way of domestic robotics.

Any thoughts/ suggestions?

anybody like to lead such a project?
Title: Re: Universal Objects Database
Post by: silo_xtreme on May 28, 2008, 08:35:50 AM
Hi Paul,

I agree with you because vision is developing and will eventually need such a database.  I'd really be interested in leading such a project, and provide a home for it as well. 

Some additional thoughts:

It could be easily accessed and searched via web services.
We could eventually upgrade it to identify the objects and provide the information back to the "user".

All I would need is some more expert advice :) and help from the interested community.

Title: Re: Universal Objects Database
Post by: AndrewM on May 28, 2008, 10:32:26 AM
A standardized camera would definitely be needed, and likely a standard mcu.  I had thought about playing with camera vision for a while now, but have too much on my plate for the time being.  However, my thought process was to use multiple mcus for object detection.  Basically something like this:

(layer1)camera feeds mcu#1 -> mcu#1 cuts the image size down to something useable (depending on the camera resolution of course) -> mcu#1 feeds image to next layer of mcus simultaneously -> (layer2)mcu#2 performs edge detection algorithms -> mcu#3 performs blob detection -> mcu#4 etc etc -> all layer2 mcus feed layer3 -> mcu#5 uses feeds to lookup data and object tracking -> mcu#6 computes distances and size determinations on all shapes -> mcu#7 lookup data and object tracking for "part breakdown".

Basically a lot of seperate functions passed off to a lot of mcus to allow as close to real time processing as possible, allowing image processing to become very granular in nature.  Another layer might be added between current layer2 and layer3, to allow individual part breakdown lookup in a table, which then would get fed to the last layer.  Basically a break down that says "we have objects a, b, c, & d that in our database correspond to an eye, a human type nose, a human type mouth, etc", the last layer then says "we know all these objects types and layout are part of a human face, so this overall object must be a human face".
Title: Re: Universal Objects Database
Post by: paulstreats on May 28, 2008, 10:51:24 AM
I strongly disagree with needing a standardised camera, mcu, and/or programming language.

Look at internet databases as an example. All they do is hold data. You can access them with any browser or a databasing utility on any computer platform with whatever processor and also using different programming languages.

This is why there is a need to create a standard system so that it can be used by many more setups. Otherwise by the time you have your structure made and have data in it, the cameras you based it around will have ceased to be manufactured.

What you do with the data is up to the end user, if you want to use an elaborate 7 mcu processing system then fair enough (im likely to use a system like that myself!) but a data repository isnt the complete system its only a part of a system that can be plugged into and used.

For example:-

A rangefinder detects an object which it gets shape data from (lets say a shampoo bottle). It can then search the data base for a match in that data and it is likely to find that it is or resembles a shampoo bottle.

At the same time a system that uses a 640x480 pixel camera can also get that exact same shape data and also search the database, this time the camera can also search the results by colors or patterns on the bottle so it can better refine its results.

As long as the shape data etc.. is stored and searched for in a standard way then both types of senors above can utilise the database. Maybe the rangefinder will come back with 3 types of objects that have the same shape because it is limited in only having to search for the shape but it can still know that the object is a bottle or a container.

I hope there is a way to unify a data storage sytem like this. And i dont fancy tackling it alone!
(all i get from colleagues when i explain the concept is "Good Luck!". Everbody seems to want to do it but its such a big task that people shy away from it)
Title: Re: Universal Objects Database
Post by: silo_xtreme on May 28, 2008, 11:01:05 AM
I agree with Paul here.  We don't need a standard computing platform because all we're doing is accessing information.  I think by using standard web services and other real-time internet standards we can allow this information to be accessed by multiple systems.

Andrew, I do like your robust design for image processing.... perhaps, you'd be interested in just using a vision chip like the ARM9 CAP series to do that heavy lifting? 

Ok.  Now unless otherwise directed I am going to start planning the database schema for the information to be stored.

So far I'm thinking a 2D Cartesian style map?  My concern with 3D is that visual processing might not be able to detect the "depth" of an object. So what we typically can end up with is the 2D representation and our "distance" to the object (assuming that we're facing the obstacle at 0 degree angle).

Thoughts?
Title: Re: Universal Objects Database
Post by: Steve Joblin on May 28, 2008, 11:13:57 AM
RoboRealm has a module that does shape matching where you store pictures of items and then the system can scan a scene and find objects that match that shape... pretty cool stuff... how does this fit into the UOD (Universal Objects Database)? http://www.roborealm.com/tutorial/Shape_Matching/slide010.php (http://www.roborealm.com/tutorial/Shape_Matching/slide010.php)
Title: Re: Universal Objects Database
Post by: paulstreats on May 28, 2008, 11:25:16 AM
Exactly the type of thing that i would like to see. Ive not used roborealm but it seems that there is a database there to do this after all. Maybe we could kind of elaborate on that system? see how it works and wether it could be improved, such as adding an attribute for how any sounds the objects make and known methods of using them etc...

I would also like to see a database that everybody has on their systems which doesnt necessarily do a live internet search but rather maybe once a day connects up to a shared repository and downloads any updates and also uploads any new objects that it has discovered or maybe find a new method or attribute for an existing object, a system like this would be highly desirable and an extremely rich source of information
Title: Re: Universal Objects Database
Post by: silo_xtreme on May 28, 2008, 11:29:54 AM
Paul,

I don't think that the "database" that you're thinking of exists as a part of  RoboRealm, it's a machine vision capability that can be part of a database.

I like the idea of a share repository and I'd be concerned about the size of the database for robotic applications.  Maybe it's a good idea to have both 1) a central repository and 2) a internet search-able version?
Title: Re: Universal Objects Database
Post by: AndrewM on May 28, 2008, 09:32:50 PM
Not to be argumentative, but if you are not going to use a standard camera (not meaning specific model, just resolution), then you will have to store your object shape data using vectors.  And while I haven't seen any hobby level robot code to produce vector graphics from a robot captured image, it would provide for your best level of universal objects.

Forgetting about things like "this is a 1967 Cobra Streetjet" and "this is a 1967 Cobra Streetjet 1/10th scale model", you will have to have some type of standard object classification system otherwise you just have a database of data that is only useful to the person who put it there.  With standard resolutions you have "a coke can is 92px across and 143px tall from X distance".  With vectoring you can "a coke can has these proportions".  I don't see another way you can have useful object shape classifications without using one or the other, standard resolution or vectored images.

The reason I brought up speciifying an mcu as a possibility is level of detail.  An 8 bit processor, uses, well 8 bits.  You could not realistically use an 8bit processor for the same level of colors and resolution that you can with a 32bit processor.  Even the Internet databases have very specific formats (RFCs) that determine how each type of data is stored and delivered, as well as what that data may contain.  HTML, RSS, XML, SQL, etc, etc.  All have some type of standard in place to prevent the data from becoming a trashheap.
Title: Re: Universal Objects Database
Post by: AndrewM on May 28, 2008, 09:46:22 PM
Andrew, I do like your robust design for image processing.... perhaps, you'd be interested in just using a vision chip like the ARM9 CAP series to do that heavy lifting? 

Sorry to stray off topic, but I just read the specs on the CAP, very robust.  If I ever get around to a vision system it definitely would warrant further investigation especially with the ability to add custom logic.  Nice.
Title: Re: Universal Objects Database
Post by: AndrewM on May 28, 2008, 10:23:16 PM
One more thing before I head off to get some much needed sleep.  Before you get too far into this, I would recommend coming up with an object classification system very early on.  It will make things much easier for not only people to upload their objects into the database correctly, but for people to be able to easily find the types of objects they might need for their own robot project when they download from the db.  Something like the following:

A. Container
   1.  Liquid Container
       a.  Soda Can
           1) 12 Oz Coke
           2) 12 Oz Mt Dew
       b.  Shampoo
           1) Head and Shoulders

4 maybe 5 levels deep.  Granular enough, but not too complex to make it no longer user friendly.
Title: Re: Universal Objects Database
Post by: silo_xtreme on May 29, 2008, 06:27:19 AM
Andrew,.

You make some very good points and all should be considered.  As I'm a Systems Engineer I'll look at all of the aspects that everyone mentions, thinks of , contributes, derives requirements , and see how they can be put together. :-)

 I'm keeping notes...  ::)

Title: Re: Universal Objects Database
Post by: Tsukubadaisei on May 29, 2008, 08:56:52 AM
Nice discussion, I quickly read it but I would like to mention a few things:

1)Such image database would take PETAbytes of data. Each virtual object should have a 3d model with textures. I doubt the robot will the object from the same angle every single time.

2)Search through this data base will require very fast processors. That means find files and doing all the comparations.

Also I disagree with standards. I think you are talking about protocols. Each robot is a different robot and demands different hardware. If all robots have the same standard hardware then all robots would be the same and would be limited to just a certain set of tasks. Protocols are like languages between machines. Even radically different machines, if they comunicate with the same protocol, then they will be able to transfer data. And this already exists: file formats are protocols, TCP/IP, USB, rs232-c and so on.
If such an searchable image database was possible with current technology, then Google would have already implemented it on its image search. And Google has several neural networks built on powerful sever workstations, so if they cant do that then good luck doing this on a single ARM.

I think this kind of stuff will only be plausible after:
1)the current computer arquitectures get extremelly fast and powerful.(lets face it: binary logic is very inneficient when it comes to intelligent tasks)
2)a revolutionary hardware that works like the human neuron is invented. And after we understand how the human brain works as well, specially the vision and visual memory centers of the human brain.

Sorry if I sound pessimistic, but I really wishe such technology exited. But I already did my own reaseach in the past and that was my conclusion which, unfortunatelly, is very sad.
Title: Re: Universal Objects Database
Post by: AndrewM on May 29, 2008, 09:13:40 AM
1)Such image database would take PETAbytes of data. Each virtual object should have a 3d model with textures. I doubt the robot will the object from the same angle every single time.

2)Search through this data base will require very fast processors. That means find files and doing all the comparations.

While I think the master database might wind up being pretty huge, I don't think the intention is for a robot to contain all that data.  I might be wrong with the intent, but I see it like this:
1) Office full of programmers like to drink Mountain Dew, but leave their cans floating around the office.
2) Ingenious programmer who is into robots decides to create a robot to go around the office and clean up the cans using a vision system.
3) Robot builder downloads the object information for a few items such as Mountain Dew cans, a trashcan, etc and stores this data in a lookup table on the robot.
4) Robot builder saves a ton of time spent trying to record object information and classify it themself with the robot.
5) Now that the cans are cleaned up, robot builder notices all the snickers wrappers floating around
6) Robot builder accesses the object database to download the object information for candy bar wrappers and adds it to the robots lookup tables.
7) Managment is happy that the programmer's area is now kept clean and gives the Robot builder a huge raise

 ;D

I use the term "standard" because it is what the IETF calls such things
Title: Re: Universal Objects Database
Post by: silo_xtreme on May 29, 2008, 09:20:31 AM
I agree with Andrew, and I hope to alleviate some excellent "issues" mentioned by Tsukubadaisei. 

1)Such image database would take PETAbytes of data. Each virtual object should have a 3d model with textures. I doubt the robot will the object from the same angle every single time.

Can visual processing by a robot currently determine that the object is 3D?  IMO 2D is a good working issue from visual imagery - please correct me if I am wrong.

2)Search through this data base will require very fast processors. That means find files and doing all the comparations.
Unless you use distributed processing.  ;D


Maybe this helps. ?
Title: Re: Universal Objects Database
Post by: bukowski on May 29, 2008, 10:56:58 AM
Quote
Can visual processing by a robot currently determine that the object is 3D?  IMO 2D is a good working issue from visual imagery - please correct me if I am wrong.

Affordable 3d cameras are just around the corner (check out the zcam by 3dv systems, supposed to release to the public sometime this year), it would probably pay off to do models, but simple ones. Say, a simple cylinder for a coke can, it takes 170k for an autodesk inventor file.
Title: Re: Universal Objects Database
Post by: silo_xtreme on May 29, 2008, 11:10:27 AM
Quote
Can visual processing by a robot currently determine that the object is 3D?  IMO 2D is a good working issue from visual imagery - please correct me if I am wrong.

Affordable 3d cameras are just around the corner (check out the zcam by 3dv systems, supposed to release to the public sometime this year), it would probably pay off to do models, but simple ones. Say, a simple cylinder for a coke can, it takes 170k for an autodesk inventor file.

according to their website:

Quote
3DV Systems has developed a unique video imaging technology and camera for sensing distance in real-time between an imaging sensor and the objects in its field of view (i.e. the objects' depth), at high speed and high resolution. The technology, which is based on the Time-of-Flight principle, is described thoroughly in several publications by the company's founders and engineers, and is well protected by international patents

This means that they don't do anything different than what machine vision does using a camera w/ an IR sensor, Sonar, or an additional camera.  This means that the "depth" is only representing the "distance" between your robot and the object.  So, again, if you're looking at that coke can, you'll only be able to detect 1/2 of the can, or 180 degrees - revolute about the can (i.e. a slice) at that instance in time.

A full model is 360 degrees.  So at most the database might want to contain "sections" of objects?

Maybe I'm wrong?  ???

Title: Re: Universal Objects Database
Post by: paulstreats on May 29, 2008, 12:56:05 PM
Camera systems can determine 3d with more than one camera. Its also possible to make assumption based on other factors (look at the vision tutorial on this site).

Maybe when objects are being created by the robots etc... they can make the basic entry and add to it.

Also searching etc.. even through a small sub database like this will become very slow which is why I suggested adding other systems such as a kohonen feature map for the objects which is in itself an implementation of a software based neural net.

Also i think that implementing a good kohonen style map for the objects will also allow searches for small sections of objects a lot easier or for example comparing a small section of the pattern rather than rotating an entire 3d object in every possible direction to see if it might fit
Title: Re: Universal Objects Database
Post by: silo_xtreme on May 29, 2008, 01:29:01 PM
Camera systems can determine 3d with more than one camera. Its also possible to make assumption based on other factors (look at the vision tutorial on this site).

I'll only believe you on this one if the robot is looking at more than one viewing plane of the object.  This is because your human eye can't see through objects either, and you can only visually see 360 degrees of an object if you take multiple views.

A single view won't reveal anything more than a section; which may in fact be a 2D or 3D section of that object.

To prove this:  Look at the coke can from the front; and then with out moving yourself, your eyes, or the can - try to read the back of the can.  ::)

Either way, I think we're all mostly saying the same / similar thing.   

Now back to building the database.  Who wants to start providing raw visual sensor data that your robot sees?  We need Content!  :-X


Title: Re: Universal Objects Database
Post by: JesseWelling on May 29, 2008, 03:54:59 PM
...wouldn't a web crawler be just as good for finding random images of things. But you would need some one to tag what kind of objects they are... Google was working on this...
Title: Re: Universal Objects Database
Post by: paulstreats on May 29, 2008, 07:46:37 PM
The 3d thing is nt to do with seeing the bak of an object......
Its to do with seeing a depth of an object, therefore determining wether an object exists depending on wether it protrudes from its environment and therefore gaining some kind of  shape to search for.

if you just haved a 2d image then its very difficult to determine a difference between 1 object and another, this is where the depth perceptions come in.

Of course, the database doesnt need these perceptions, it just needs the overall shape or to be more precise the dimensional graph. Its up to the end system or user to implement the detection systems.

It needs to be made clear now that this is just a database and that it is up to the system or user to interpret theri own data in order to use the database
Title: Re: Universal Objects Database
Post by: paulstreats on May 29, 2008, 07:51:49 PM
Also another thing...

I think somebody mentioned that google worked on a neural net system? or was it somewhere else?

What ever.... Maybe its worth seeing how that system has been deveoped and interfaced.......

(Im full of these types of ideas, but my commitments wont give me enough time to follow them up, so thank you to all that are contributing and willing to progress them :))
Title: Re: Universal Objects Database
Post by: Webbot on May 29, 2008, 10:28:07 PM
Surely XML is a way forward. Databases are great for constantly changing data - but a Coke can doesn't change - although your view of it might. XML would also allow you to add all sorts of header tags that may help you to scan across lots of DTD/XML schemas. ie 'Its red' so it could be Coke but its 'Not a Pepsi'.

Oh No its the 'soda wars' all over again  ::)
Title: Re: Universal Objects Database
Post by: JesseWelling on May 29, 2008, 11:10:21 PM
I would think that XML can't be sifted through as fast as a Relational (http://en.wikipedia.org/wiki/Relational_database) or Object (http://en.wikipedia.org/wiki/Object_database) databases because of internal representation...

But that's just my hunch... :-\
Title: Re: Universal Objects Database
Post by: Tsukubadaisei on May 30, 2008, 01:12:15 AM
Quote
Can visual processing by a robot currently determine that the object is 3D?  IMO 2D is a good working issue from visual imagery - please correct me if I am wrong.

Affordable 3d cameras are just around the corner (check out the zcam by 3dv systems, supposed to release to the public sometime this year), it would probably pay off to do models, but simple ones. Say, a simple cylinder for a coke can, it takes 170k for an autodesk inventor file.

I knew you guys would missunderstand me. You dont need a 3d camera at all. 3d cameras are good with you want depth and that is not the point. Without the 3d model database, you wil need lots of pictures of the same object from different angles otherwise your robot wont be able to idetify it. For example lets suppose your robot has a picture of a car in the database. This picture shows the right side of the car. But then your robot aproaches the car from the front. The image of the car will be frontal side  of the car. The right side and the frontal side of car are shaped very differently(usless you heva a cubecar). If you have a 3d model in the database, the coputer will try rendering this mode in different angles until it finds one 2D image that maches the actual view from the robot camera. By the way, you guys should learn why and how 3d cameras are used before making an argument, otherwise your arguments would sound silly(that applies for any serious or academic discussion).

However I would like to mention that I liked the idea of downloading a just a set of items from a bigger database in some server. That is actually a nice idea. It is good to be able to dynamically change the settings of the machine. I thought the database would be inside the robot.

Finally I would like to mention you guys are focusing too much on the database problem when that is, in my opinion, the smallest one. I think that the actual image processing and recognition algorithm. is the real issuue here. Recogizing a ball or a line or even a can of soda(talking about only the shape) is relatively an easy and very specific task. If you are planning on building a database, that is because you are going to have lots of objects. And most objects in real world have colorfull, complex, assymetrical shapes. Programming for those shapes is going to be a nightmare. And you also have to program lots of filters because the conditions will also change from time to time. Finally for a complete robot you would like to write the  following three programs:
1)The easiest program, but hard. It would be selecting an (random) object from the database, and searching for it.
2)The even (much) harder program. Finding random objects in the real world and looking for them in the database.
3)The hardest: combining the above two in a single program.
Title: Re: Universal Objects Database
Post by: Webbot on May 30, 2008, 01:54:12 AM
So for problem (2) match a real world object against a database - then this would benefit from some high level indexing routines to quickly filter out wrong data. ie 'its red so it could be Coke', 'its purple so it could be Pepsi'. Sorry to labour the soda gag  ::)

This means taking some kind of sensor data and using it, or many of them, as a quick high level filter. This could be done on the 'robot' and then going to the database for the filtered results. e.g. no point checking if its a red hexagon, red pentagon etc if you know it 'aint red'.

Its a bit like using 'keywords' for searching web pages. You could also make this adaptive - so that if you took a long time to find the actual 'thing' in the database then you could automatically associate more 'keywords' against it based on your initial sensor input. So it would be found quicker next time.

Which brings me to Jesses point:
Quote
I would think that XML can't be sifted through as fast as a Relational or Object databases because of internal representation...
If the 'answer' is something concrete and unchanging - say a CAD diagram of the Coke can -  then that data isn't going to change (its read only). Equally: the individual points that make up the diagram probably aren't that useful for searching puroposes. Its more likely that you want to find something thats a 'red cylinder' rather than it has a 'point at (5,1,6) in 3D coords'. In which case there's no point putting the CAD diagram into a database other than as a BLOB - in which case you might as well just store it as a file - and XML is great for that. XML is also a good format for you to run a process on to create the high level indexes.

So the problem is how to build the high level indexes that allow you sift out the relevant results and what sits on what machine. I would think the bulky, read-only data could sit elsewhere (eg on a server) but you could always load the indexes onto the robot.
Title: Re: Universal Objects Database
Post by: JesseWelling on May 30, 2008, 03:00:46 AM
I think you are missing the point that XML is text and needs to be parsed. Every time. It's a strength and weakness because it has ultimate portability, and readability, but sucks for very very large amounts of what could be binary data.

I'm imagining a 3d model, or 3d point cloud, stored in the data base for each object and then the model being looked at could be matched to objects in the data base with a baysean filter, but you would need access to your almost your whole data base to do that  :-\
Title: Re: Universal Objects Database
Post by: silo_xtreme on May 30, 2008, 06:45:12 AM
I think you are missing the point that XML is text and needs to be parsed. Every time. It's a strength and weakness because it has ultimate portability, and readability, but sucks for very very large amounts of what could be binary data.

I'm imagining a 3d model, or 3d point cloud, stored in the data base for each object and then the model being looked at could be matched to objects in the data base with a baysean filter, but you would need access to your almost your whole data base to do that  :-\

Unless you thought of the system as a bipartite (2 part) system.  The database can do the heavy processing and object matching, the robot can send in the "visual data" and just get back from the database "the answer".  If the database just kept updating every time it got new data, then it would be "current" and hopefully "Fast enough" for the robot.   ;)
Title: Re: Universal Objects Database
Post by: Nyx on May 30, 2008, 07:33:36 AM
Well....... As someone who is now doing an M.Sc. in computer science, and has taken both grad level AI and vision courses....

You could have a database of images containing specific objects in focus, with their outline marked in some file, and some description of the contents of the image at those coordinates marked in some format of your choice... These images would not have to be taken from the same camera, or in the same environment.... In fact, the more variablility the better.... Why would this be useful? To train learning algorithms for vision. Many algorithms in AI require training data (artificial neural networks being the most obvious example). Others require some data from which to build a database of characteristics common to certain types of objects (eg: eigenvector based approaches, eigenfaces for facial recognition).

Anyways, the whole point of such algorithms is that they learn by themselves, by seeing many examples of a coke can, and many examples of a lamp, what would be the common characteristics to coke cans or lamps. What you can hope to do is provide such examples, along with some useful accompanying information (such as what object is in focus, and what its rough outline is).

You can't really hope to just lookup a specific image you just filmed into a database of images taken with a "standardized camera". That is never going to work... Never has, never will. And no, you won't get everyone to use the same camera... Especially because, you know, technology progresses, and whatever is your "standard camera" now, will suck in 10 years.
Title: Re: Universal Objects Database
Post by: AndrewM on May 30, 2008, 08:26:44 AM
You can't really hope to just lookup a specific image you just filmed into a database of images taken with a "standardized camera". That is never going to work... Never has, never will. And no, you won't get everyone to use the same camera... Especially because, you know, technology progresses, and whatever is your "standard camera" now, will suck in 10 years.

I really don't get how people continually relate a standard in technology to a specific model.  It is specific minimum functionality.  http://en.wikipedia.org/wiki/Internet_standard

Think about how a robotic hobbyist might go about sharing image capture object data between robots.  If you were to build two robots that you wanted to share the same type of data between the two, you would probably want to ensure the cameras used on each robot supported the same kind of data, otherwise you are comparing apples to concrete blocks (metaphorically speaking).

That being said, I still believe the better approach is to have all captured data be in some type of vector format, that would have to be... standardized. (I love that word  ;D )  One possible solution would be to require all vector data in the database to use a minimum point to point length of 1 unit, allowing all robots that will use the data to have a set reference on how the data will be presented.  Here's an example:
We have two robots with two different cameras.  Robot A has a camera with a resolution of 640x480.  Robot B has a camera with resolution of 320x320 (just to make it oddball).
Robot A captures an image of a rectangular white box that is 4 inches wide and 8 inches tall.  The captured image is 200px across and 400px down.  Robot A converts this to a vector where the width of the image is 1 unit, the height is 2 units.  This gets sent to the database.
Robot B downloads this image into its lookup table.  Robot B then comes across a white rectangular box.  The captured image is 50px by 98 px.  Robot B converts this image into a vector making the shortest post (the width) 1 unit, the height comes out to be 1.96 units.  It looks for an object in its database and finds a probably match (around 98%) with the white rectangular box (based on dimensions alone).
Next might come color recognition or actual size determination based on distance or internal composition recognition (the vector objects that make up the object as a whole), or any of a number of other things.

Without the standard of minimum vector length of 1 unit, each robot will have to use up a lot more processing time to find matches as it upscales and downscales images to match.  With the standard the robot builder/programmer knows how the data will be presented from the global object database, so can write his code to match.

The vector standard would also allow an easier integration of 3D objects in the database (not necessary, but can be added in the future).  Once a full 3D object is in the database using the vector standard, the robot builder can slice off sections of that 3D object to store in a lookup table at say every 15o rotation providing a higher "probability" match.  Or, with a fast enough mcu, can dump the entire 3D object into the robot and do rotational comparisons on the fly.

The idea is to make the database useful for robotists around the globe.
Title: Re: Universal Objects Database
Post by: AndrewM on May 30, 2008, 08:33:46 AM
It just dawned on me (I am slow sometimes) but in the above example, even if the white rectangular box was laying on its side with a height of 1 unit and width of 1.96 units, the program could still match it easily enough because it is comparing relative dimensions and not "actualities".
Title: Re: Universal Objects Database
Post by: Nyx on May 30, 2008, 08:54:46 AM
Quote
I really don't get how people continually relate a standard in technology to a specific model.  It is specific minimum functionality.  http://en.wikipedia.org/wiki/Internet_standard

You obviously didn't really read my comment, but anyways... It never ceases to amaze me how people with some engineering background think they can improvise themselves AI experts. There's a very active subfield of computer science called COMPUTER VISION (http://en.wikipedia.org/wiki/Computer_vision). This field has been concerned with issues like object and scene recognition for decades... And they've come up with many interesting approaches which... You should look at. Some of them are very simple to implement and computationally efficient... Like the eigenvector based approach I just mentioned. You could also think of applying Hidden Markov Models (HMM) and neural networks to such problems, or work from spectral representations of images, which are less susceptible to changes in orientation, etc.

All of these approaches make ALOT more sense than trying to send images accross a network and match them to some raw database... Which, like I said, is never going to work, and isn't very practical (requires a fast connection) no matter the format of the image you send/receive. If you use eigenvector based approaches, you only need a database for training. Once this is done, images are "compressed". They're represented by essential components only... So you can represent an image by its features, using less than 1KB of memory, or whatever you can afford to use in your system.

Lookup tables haven't been the focus of AI researchers for um, say, 60 years?

So yeah uh... The computer scientists of the world would like to tell you that:

(http://blogs.indiewire.com/reverseshot/archives/you_got_served.jpg)

Now sit down and learn at least a *little bit* about AI, before you start trying to reinvent us a square wheel.
Title: Re: Universal Objects Database
Post by: silo_xtreme on May 30, 2008, 09:03:56 AM
LoL - just the approach I was getting at.  ;D
Title: Re: Universal Objects Database
Post by: Nyx on May 30, 2008, 09:11:04 AM
LoL - just the approach I was getting at.  ;D

Well my advice to you, if you do design such a database, is that too much information is always better than not enough. You should design lots of standardized tags that people can put on images... Allow them to upload matching stereo pairs, etc... If they don't have some of the info, they can omit it... But when someone wants images that have all the tags, they can query the DB for only images that have all the information their system needs. You definitely want lots of info about the camera(s) that took the picture(s), the lighting in the scene, etc to be stored, if available.

*Legally speaking*, if you want to make this research friendly... You probably want some licensing terms that say whatever you upload to the database must be public domain...
Title: Re: Universal Objects Database
Post by: silo_xtreme on May 30, 2008, 09:28:50 AM
I'm with you on that ;D

I'm also expecting to have other forms of visual data as well.  For example, say you don't have a CCD / CMOS camera.  You can "scan" the object with your sharp IR as well and still get a visual representation of the object.
Title: Re: Universal Objects Database
Post by: Nyx on May 30, 2008, 09:31:16 AM
If you have some kind of ontology to describe the scene and/or objects in the scene, it would be nice to be able to tag multiple pictures as being of the same object in the same scene, or from the same scene, but taken from a different angle. People in vision often like taking pictures of an object from 100 different angles when trying to build 3D reconstructions of it, for example.

I also think you want to allow as high of a resolution as possible, to allow as much detail as possible... If you want to "future proof" your design. What you can do, however, is provide a standard fixed resolution representation as well.
Title: Re: Universal Objects Database
Post by: AndrewM on May 30, 2008, 05:26:05 PM
Lookup tables haven't been the focus of AI researchers for um, say, 60 years?

I'll avoid the whole pissing contest it seems you want to get into, but want to point out that lookup tables are still very viable for DNA programming and genetic algorithms.  I guess we will have to find Chef and do a dance off now.  Just make sure butter's tap shoes are tied off well...   ::)

The lookup tables were for images inside the robot, as opposed to having the entire database stored inside the robot itself, or having the robot call out to the internet for object recognition.  Smaller portions of the whole data for the individual task the robot is responsible, as opposed to knowing what everything in the world is.
Title: Re: Universal Objects Database
Post by: Nyx on May 30, 2008, 07:16:39 PM
Eigenvector based approaches are kind of like a lookup table, but the lookup is much faster than going through a table and trying to find the nearest sample, because each sample is like a point in n-dimensional space... So the samples can be put in a kd-tree, and lookup takes time log(k), where k is a constant.
Title: Re: Universal Objects Database
Post by: AndrewM on May 31, 2008, 10:38:57 AM
I'm also expecting to have other forms of visual data as well.  For example, say you don't have a CCD / CMOS camera.  You can "scan" the object with your sharp IR as well and still get a visual representation of the object.

Is the sharp IR accurate enough to get that kind of data?  I purchase a few about a month back, but have yet to use them for anything, so I'm curious on that.  All my IR in the past has been home built.

I haven't started down the road to robotic vision yet, but I know I will be in not too distant future, so I am starting to get excited about having a database to tap into when the time comes.  Slightly off the subject, but is there any intent to possibly having sample code for a few different setups for saving images for the DB and/or using the images from (I know getting the DB set comes first, but figured I would ask)?
Title: Re: Universal Objects Database
Post by: paulstreats on May 31, 2008, 02:23:20 PM
Sharp IR are not exactly an accurate sensor, but maybe an option in the database to search for a closest match, or even the reverse if your robot is looking in the real world for an object then it can look around till it finds something within maybe a 10% error margin. (this feature must be included anyway or a slightly dented can etc... wont match up correctly)

Im sure that when the database begins it will be important to let people know how to use it so i should think that there will be examples and documentation.... Of course the main part is to decide on a structure which i think that everybody should express an opinion on (if there is more than 1 idea then maybe a stronger better storage system can work by meging more solutions). It would be really helpful if people could link to example of the data storages that they suggest. So if somebody thinks neural net systems are best then post a link with theories/examples etc.. on so we can all see and express opinions and also make the job easier for the person who is eventually going to make the database.

silo xtreme: since you are leading the project, have you thought of any systems that you are going to use yet? (maybe somebody can help with specifics, there are some experienced programmers on this site who will probably be willing to help create structures/algorythms or even just help optimise codes if needs be :))

I would like to voice my opinion on seeing kohonen feature maps (again) as a way of storage physical object data. Also it could be worth looking into existing 3d graphics systems to see how they store shapes - strip count arrays etc... (there are lots of ways of optimising 3d vertexes all aimed at reducing storage space. think about those 3d games that you occasionally come across with a single .exe file which is only like 25kb long. Maybe the graphics wont be brilliant, but theyve got a lot of 3d object data in that 25kb)
Title: Re: Universal Objects Database
Post by: paulstreats on May 31, 2008, 02:50:53 PM
sorry for this follow on post but anyway.....

If somebody decides that they dont like the final storage structure. Then i dont see why there is a problem in using your own structure on your own robot but downloading the data from the UOD. Surely if you are capable of making an effective and efficient storage system of your own then you are capable of converting data from the UOD into your own system. Maybe you could release your own system as a real world system and also realease cross convertion tools with it....

Somebody above mentioned that if it was possible then google would have done it. THEY HAVE - its just not correctly linked. ie. search google for a can of coke and you get loads of data about it. search google images for coke and you get loads of images. search google sketchup and you get a 3d model of a can of coke what more could you need.
The difference is that google is made for people to search through and not robots.
Title: Re: Universal Objects Database
Post by: Tsukubadaisei on May 31, 2008, 08:39:53 PM

Somebody above mentioned that if it was possible then google would have done it. THEY HAVE - its just not correctly linked. ie. search google for a can of coke and you get loads of data about it. search google images for coke and you get loads of images. search google sketchup and you get a 3d model of a can of coke what more could you need.
The difference is that google is made for people to search through and not robots.
I mentioned that. I am not talking about the current image search methods using tags or related words with some keyword, like google does right now. I was talking about using a image for searching other images. And that is what I've been talking about through all the posts. For example the robot sees an object throgh the camera, and then search for this object in the database. Asking the robot to seach for something inputing a keyword in it is actually an already solved problem(for simple shaped objects).
Title: Re: Universal Objects Database
Post by: AndrewM on June 01, 2008, 12:34:20 AM
Searching for an object is not all that difficult.  Of course it depends on the data contained in the database and how it is all stored.  But to give one example using a coordinate system (I think the initial suggestion was cartesian coordinates, but it really doesn't matter).

Without breaking out a tape measure, we'll assume a coke can is 2 inches wide and 4 inches tall.  The primary coloring of the can is red (I don't drink soda anymore, but last I checked I think this is accurate).  So we snap off a picture of a coke can using a color camera.  Poof, we have a 2D rectangle with coordinates of {0,4}{2,4}{2,0}{0,0}.  A keyword (numeric actually) search of objects containing only 4 outside points with those same coordinates returns say 500 objects.  We add to that a primary blob of color range in red, might be an RGB color range etc.  Down to 10 objects.  Then maybe do a search on object shapes internal to the outside shape, such as a logo using the same coordinate system.  Poof, we have a 90-something% match.

And that is how we *might* search for an object.  (I like the word "poof" btw).  As I said, it depends on what data is stored and how it is stored.  But looking for coordinates, or vectors, or color ranges from a database really is as easy as typing in "Coke Can" in google, if the data is stored in the right format.

In answer to how 3D objects are stored for games, there are a couple ways, but mostly it is just coordinate pairs or vector pairs.  Either way gives you the same thing, it is just mathematically easier to rotate a set of vectors than a set of coordinates, although not by much if using a cartesian (or even polar) coordinate system and setting the point of rotation at 0,0.  Although I haven't written a game in ten years or so (God I feel old), but I used to do a lot of 3D graphic work and programming of same.
Title: Re: Universal Objects Database
Post by: silo_xtreme on June 13, 2008, 07:39:28 AM
Ok Everyone.

I just wanted to provide an update.  I've created an distributed image processing capability that can run on any machine and be controlled by a server.

All we have to do now is agree on how we should make that "standard" vector data  ::)
Title: Re: Universal Objects Database
Post by: silo_xtreme on June 16, 2008, 10:33:04 AM
hmm ... maybe the project has died?  ??? 
Title: Re: Universal Objects Database
Post by: AndrewM on June 16, 2008, 10:56:18 AM
I think getting a consensus on the style and format for storing data will be a tough one to do seeing as there is a wide variety of methods that can be used.  Anyone who is capturing image data now will want to see a format similar to what they are already using, which will make a mess trying to adopt so many different styles.

My opinion would be that seeing as you created the database and will be maintaining it, that you should just set the format "standard" (there I go with that word again :P )
Title: Re: Universal Objects Database
Post by: Admin on June 21, 2008, 12:01:23 PM
The AI/vision community currently just uses Google Image search.

Tell the robot to find coke, it looks it up and runs various classical AI algorithms to find a match in the environment.

Now to do the inverse is much harder . . . it sees something red . . . the algorithm can't just search the entire internet for it. Instead, it breaks the object into components, eliminating anything that isn't round, that isn't cylindrical, that isn't some particular size, etc . . .
Title: Re: Universal Objects Database
Post by: R.O.B. 2.0 on June 21, 2008, 01:02:10 PM
I think that along with focusing on the image processing, we should also focus on the how the robot would react correctly.  Say you had a red index card that was very important to you somehow and the robot picks it up and throws it away.  If it had million dollar info on it, you would not be too happy with your robot.