Society of Robots - Robot Forum

Software => Software => Topic started by: EDV on December 14, 2009, 12:56:46 AM

Title: Autonomous robot's navigation
Post by: EDV on December 14, 2009, 12:56:46 AM
Hello,

Now I work at associative video memory. The method still in developing (now it version 0.5)
but it gives good results already today.

I am dealing with research of computer vision in parallel with my main job
at "Impulse" more than three years (it is my hobby).

About me: http://edv-detail.narod.ru/EDV_resume.html (http://edv-detail.narod.ru/EDV_resume.html)

In the beginning my achievements were insignificant and little part of ideas has worked properly.
But I did not surrender. I generated big quantity of hypotheses and then test it.

The most ideas did not work indeed but those that worked were similar to particles of gold
in huge quantity of dross. My associative video memory method is working indeed.

============================- Common information -==========================

Algorithm AVM uses a principle of multilevel decomposition of recognition matrices,
it is steady against noise of the camera and well scaled, simply and quickly
for training, also it shows acceptable quick-action on a greater image resolution
of entrance video (960x720 and more). The algorithm works with grayscale images.

The detailed information about AVM algorithm can be looked here:
http://edv-detail.narod.ru/AVM_main.html (http://edv-detail.narod.ru/AVM_main.html)

AVM SDK v0.5 with examples of using and tests for comparison
of characteristics of the previous and new versions:
http://edv-detail.narod.ru/AVM_SDK_v0-5.zip (http://edv-detail.narod.ru/AVM_SDK_v0-5.zip)

Demonstration video how to train AVM:
http://edv-detail.narod.ru/Face_training_demo.avi (http://edv-detail.narod.ru/Face_training_demo.avi)

AVM demo with the user interface (GUI), installation for Windows:
http://edv-detail.narod.ru/Recognition.zip (http://edv-detail.narod.ru/Recognition.zip)

Connect the web-camera and start AVM demo after installation of "Recognition.exe".
After starting the program will inform that there is not stored previously data
of training AVM and then will propose to establish the key size of the image
for creation of new copy AVM. Further train AVM using as an example Face_training_demo.avi.

========================- Robot's navigation -=========================

I also want to introduce my first experience in robot's navigation powered by AVM.

Briefly, the navigation algorithm do attempts to align position of a tower
and the body of robot on the center of the first recognized object in the list
of tracking and if the object is far will come nearer and if it is too close it
will be rolled away back.

See video below:
[youtube]EcXl0mQcPxs[/youtube]

[youtube]AvQMi_DdF94[/youtube]

I have made changes in algorithm of the robot's control
also I have used low resolution of entrance images 320x240 pixels.
And it gave good result (see "Follow me"):
[youtube]HTxNlOpm11U[/youtube]

Robot navigation by gate from point "A" to "B"

See video below:
[youtube]R4Ogq2YUfrU[/youtube]

[youtube]1w2bMlTsLUI[/youtube]

First an user must set the visual beacons (gates) that will show direction where robot has to go.
Robot will walk from gate to gate. If the robot recognize "target" then he come nearer and stop walking.

Navigation application (installation for Windows):
http://edv-detail.narod.ru/Recognition.zip (http://edv-detail.narod.ru/Recognition.zip)

Installation also contains source code of robot control driver (InstDir\RobotController_src).
So you can adapt it to your robots and repeat my navigation experiments by yourself.

Also available source code of navigation application (but all comments in Russian):
http://edv-detail.narod.ru/Navigator_src.zip  (http://edv-detail.narod.ru/Navigator_src.zip)

Source code of "Navigator" program was prepared for Russian community.

But if you would show interest to this topic then I could adapt it to English community too.
Title: Re: Autonomous robot's navigation
Post by: maitri on December 15, 2009, 11:12:08 AM
A more detailed description of your algorithm and english comments would indeed be useful...
Title: Re: Autonomous robot's navigation
Post by: EDV on December 22, 2009, 11:33:01 AM
Navigator package description

The package consists of three parts: the robot control driver, the pattern recognition application (GUI), and a dynamic link library "Navigator".
Compilation of pattern recognition application will need wxWidgets-2.8.x (http://"http://sourceforge.net/projects/wxwindows/files/wxMSW/2.8.10/wxMSW-2.8.10-Setup.exe/download") and OpenCV_1.0 (http://"http://sourceforge.net/projects/opencvlibrary/files/opencv-win/1.0/OpenCV_1.0.exe/download"). If someone has no desire to deal with the GUI, then the project already has compiled recognizer (as EXE) and you will be enough to compile Navigator.dll, which contains the navigation algorithm. Compilation of Navigator.dll needed only library OpenCV_1.0 (http://"http://sourceforge.net/projects/opencvlibrary/files/opencv-win/1.0/OpenCV_1.0.exe/download"). You can build project by compiler Microsoft Visual C++ 6.0 (folder vc6.prj) and by compiler Microsoft Visual Studio 2008 (folder vc9.prj).

After installation (and compilation) libraries wxWidgets-2.8.x and OpenCV_1.0 need to specify additional folders for the compiler:

Options / Directories / Include files:
<Install_Dir> \ OPENCV \ CV \ INCLUDE
<Install_Dir> \ OPENCV \ CVAUX \ INCLUDE
<Install_Dir> \ OPENCV \ CXCORE \ INCLUDE
<Install_Dir> \ OPENCV \ OTHERLIBS \ HIGHGUI
<Install_Dir> \ OPENCV \ OTHERLIBS \ CVCAM \ INCLUDE
<Install_Dir> \ WXWIDGETS-2.8.10 \ LIB
<Install_Dir> \ WXWIDGETS-2.8.10 \ LIB \ VC_LIB \ MSW
<Install_Dir> \ WXWIDGETS-2.8.10 \ LIB \ VC_LIB \ MSWD
<Install_Dir> \ WXWIDGETS-2.8.10 \ INCLUDE
<Install_Dir> \ WXWIDGETS-2.8.10 \ INCLUDE \ MSVC

Options / Directories / Library files:
<Install_Dir> \ OPENCV \ LIB
<Install_Dir> \ WXWIDGETS-2.8.10 \ LIB
<Install_Dir> \ WXWIDGETS-2.8.10 \ LIB \ VC_LIB

Source code of the "Navigator" (for English community) can be downloaded here:
http://edv-detail.narod.ru/Navigator_src_en.zip (http://"http://edv-detail.narod.ru/Navigator_src_en.zip")

For compilation of the source code you can use the "MS Visual C++ 2008 Express Edition" (http://"http://www.microsoft.com/express/download/"). It is all official and free.


For connection of robot to Navigator program you have to adapt the control driver (.\src\RobotController) to your robot.

It's simple: the application Recognition.exe interacts with the robot driver "through shared memory (gpKeyArray). And all you need to do - it is a timer (method CMainWnd:: OnTimer) to send commands from the "gpKeyArray" to your robot.

The chain of start commands will be transmitted to robot for "power on" (cmFIRE, cmPOWER) when you start navigation mode. Respectively command "power off" (cmPOWER) will be transmitted when navigation mode will be disabled.

And most importantly: the commands cmLEFT and cmRIGHT should not activate motion in itself but only in combination with the commands "forward", "back" (cmFORWARD, cmBACKWARDS).

If you have adapted control driver to your robot then you are ready join to navigation experiments.
Title: Re: Autonomous robot's navigation
Post by: EDV on December 22, 2009, 11:44:34 AM
So, let's have a fun together :)
Title: Re: Autonomous robot's navigation
Post by: EDV on September 22, 2010, 01:15:22 AM
RoboRealm company begins distribution of "Navigator" plugin (http://www.roborealm.com/help/AVM_Navigator.php) within RoboRealm software (http://www.roborealm.com/).

(http://forums.trossenrobotics.com/attachment.php?attachmentid=2124&stc=1&d=1284968393)
Title: Re: Autonomous robot's navigation
Post by: EDV on May 04, 2011, 01:55:20 AM
Now AVM Navigator v0.7 is released and you can download it from RoboRealm website (http://www.roborealm.com/help/AVM_Navigator.php).
In new version is added two modes: "Marker mode" and "Navigate by map".

Marker mode

(http://www.roborealm.com/help/Navigator_wnd_4.png)

Marker mode provides a forming of navigation map that will be made automatically by space marking. You just should manually lead the robot along some path and repeat it several times for good map detailing.

Navigation by map

(http://www.roborealm.com/help/Navigator_wnd_5.png)

In this mode you should point the target position at the navigation map and further the robot plans the path (maze solving) from current location to the target position (big green circle) and then robot begins automatically walking to the target position.


(http://www.roborealm.com/help/Navigation_map_2.png)

For external control of "Navigate by map" mode is added new module variables:

NV_LOCATION_X - current location X coordinate;
NV_LOCATION_Y - current location Y coordinate;
NV_LOCATION_ANGLE - horizontal angle of robot in current location (in radians);


Target position at the navigation map
NV_IN_TRG_POS_X - target position X coordinate;
NV_IN_TRG_POS_Y - target position Y coordinate;

NV_IN_SUBMIT_POS - submitting of target position (value should be set 0 -> 1 for action).

Examples

Quake 3 Odometry Test (http://www.youtube.com/watch?v=9opAZYRsDxU#ws)

Navigation by map (http://www.youtube.com/watch?v=5qKQg5e4wbc#ws)

Visual Landmark Navigation (http://www.youtube.com/watch?v=WdfWt66rEsk#)


Title: Re: Autonomous robot's navigation
Post by: EDV on June 04, 2011, 12:17:37 PM
Quake 3 Mod

(http://www.roborealm.com/help/AVM_Navigator/Quake_Thumb.jpg)

Don't have a robot just yet? Then click here (http://www.roborealm.com/help/AVM_Navigator/avm_q3mod_help.html) to view the manual that explains how to setup RoboRealm
with the AVM module to control the movement and processing of images from the Quake first person video game.
This allows you to work with visual odometry techniques without needing a robot!

The additional software needed for this integration can be downloaded here (http://www.roborealm.com/help/AVM_Navigator/Setup_avm_q3mod.exe).
Title: Re: Autonomous robot's navigation
Post by: EDV on June 06, 2011, 01:29:30 PM
Is it possible to play with virtual robot in "Navigation by map" mode?

Yes!

(http://www.roborealm.com/help/AVM_Navigator/q3mod_route_training.png)

Just look into documentation (http://www.roborealm.com/help/AVM_Navigator/avm_q3mod_help.html) and download the "AVM Quake 3 mod" installation (http://www.roborealm.com/help/AVM_Navigator/Setup_avm_q3mod.exe).
Title: Re: Autonomous robot's navigation
Post by: EDV on July 10, 2011, 11:48:38 PM
Next modification of AVM Navigator v0.7.2.1 (http://www.roborealm.com/help/AVM_Navigator.php)is released.

Changes:
Visual odometry algorithm was updated:
AVM Nav v0.7.2.1 testing (http://www.youtube.com/watch?v=Ggv8MaeJcTo#)
Title: Re: Autonomous robot's navigation
Post by: EDV on August 09, 2011, 02:10:36 PM
I have done new plugin for RoboRealm (http://www.roborealm.com/help/EDV_DVR.php):

(http://www.roborealm.com/help/EDV_DVR_thumb.jpg) (http://www.roborealm.com/help/EDV_DVR.php)

Digital Video Recording system (DVR) (http://www.roborealm.com/help/EDV_DVR.php)

You can use the "DVR Client-server" package as a Video Surveillance System in which parametric data
(such as VR_VIDEO_ACTIVITY) from different video cameras will help you search for a video fragment
that you are looking for.

You can use the "DVR Client-server" package as a powerful instrument for debugging your video processing
and control algorithms that provides access to the values of your algorithm variables that were archived
during recording.


Technical Details

 - ring video/parametric archive with duration of 1 - 12 months;

 - configurable database record (for parametric data) with maximal length of 190 bytes;

 - writing of parameters to database with discretization 250 ms;

 - the DVR Client can work simultaneously with four databases that can be located at remote computers.


(http://www.roborealm.com/help/dvr_principle.gif)
Title: Re: Autonomous robot's navigation
Post by: EDV on October 03, 2011, 12:02:46 PM
Quote from: Mel
Hey EDV!
 I finally got my hands on a Roomba robot that I could try with the Nav programs. I went through all of the items and tutorials. When I placed the robot in the NAV mode, it moved. All others did not move unless I used the arrows to train them. The Nav by map mode showed the progress, but I could not get it to move by clicking on Left mouse. When I clicked on Left mouse it did not do anything. I would like to make that work. Can it work on it's own, or do I have to train it in one of the other modes?

I prepared simple video tutorial "Route training and navigation by map":

[youtube]qVz9iBazqug[/youtube]

See more details (http://www.roborealm.com/forum/index.php?thread_id=4246#) about tuning of "Marker mode" and "Navigation by map" modes.

(http://forums.trossenrobotics.com/attachment.php?attachmentid=3507&d=1317583050&thumb=1) (http://edv-detail.narod.ru/trend_empty_cells.png)

(http://forums.trossenrobotics.com/attachment.php?attachmentid=3508&d=1314812632&thumb=1) (http://edv-detail.narod.ru/dvr_client_trend_2.png)
Title: Re: Autonomous robot's navigation
Post by: EDV on October 19, 2011, 12:38:54 PM
It's test of new algorithm for AVM Navigator v0.7.3. In this video was demonstrated how robot just try to go back on checkpoint from different positions of the learned route.

[youtube]wj-FKhdaU5A[/youtube]

First in video the robot has received command: "go to the checkpoint" and when robot arrived to the checkpoint then I brought robot back (several times) in different position of learned route. When robot noticed the changes then it indicated that robot was displaced because any commands were not given by robot to his motors however changes were seen in the input image.

Then robot started looking around and localized his current position. Further the robot just calculated path from current position to the checkpoint and went there (and so forth).
Title: Re: Autonomous robot's navigation
Post by: EDV on January 06, 2014, 08:42:51 AM
Hi guys,

I'm still working over AVM technology. Now I've founded my own company that is named Invarivision.com (http://invarivision.com).
We are small but passionate team of developers that are working over system which would be able watch TV (http://invarivision.com/help.php) and recognize video that interests user.

And we need your help!

It seems that interface of our search system is good enough because we try to make it to be simple and user friendly but from other point of view it could be a total disaster.

Could you  please take a look to our system (http://invarivision.com) and then tell us about good and bad sides of this?

The constructive criticism is welcome.

With kind regards, EDV.
Title: Re: Autonomous robot's navigation
Post by: jwatte on January 06, 2014, 10:40:24 AM
How does your algorithm deal with offset, rotation, scale, and mirroring of the "trained" object/source ?
Title: Re: Autonomous robot's navigation
Post by: EDV on January 07, 2014, 05:45:28 AM
Algorithm AVM (http://edv-detail.narod.ru/AVM_main.html) uses a principle of multilevel decomposition of recognition matrices also it is steady against noise and can work with images that were offset or scaled but it does not invariant to rotation or mirroring.

Quote from: Cbenson
This is quite different from autonomous robot navigation.
Our search system also is the robot that watches TV like a human but does it simultaneously on several TV channels and this robot can watch TV nonstop and do not to get tired  :)

Quote from: Cbenson
Can you explain more about how the system would determine what video interests the user?
The user just uploads video (http://invarivision.com/help.php) which interests him to the search system and then system would search it on all channels that are scanned.

Also user can add own TV channel that interests him if system still does not have such channel.

So, in other words: AVM Video Search system (http://invarivision.com/) provides service that allows customers to make audit of TV channels or searching of forbidden or copyrighted video in file with help of automatically recognition of the video content.

The main advantage of this system is direct recognition of separate images into analyzing of video content that is provide searching of very small video fragments about two seconds long.
Title: Re: Autonomous robot's navigation
Post by: jwatte on January 07, 2014, 11:22:57 AM
Quote
it does not invariant to rotation or mirroring

That makes it kind-of hard to find video that's "similar" to what I like, rather than "exactly" what I like.

As I said on the other forum you also posted this: You should go for finding YouTube and Netflix that the user might like. Broadcast/Cable TV is dead (it just doesn't know it yet.)
Title: Re: Autonomous robot's navigation
Post by: EDV on September 06, 2019, 02:47:15 AM
I've prepared InvariMatch presentation:

[youtube]Gp-0CtAMdRo[/youtube]

I'm sorry for my English in this presentation I was try my best :)

It's hard to believe that all this stuff has grown up from robot navigation (there is the same Associative Video Memory algorithm that is used for matching of video content).
Title: Re: Autonomous robot's navigation
Post by: EDV on September 06, 2019, 02:49:16 AM
As I said on the other forum you also posted this: You should go for finding YouTube and Netflix that the user might like. Broadcast/Cable TV is dead (it just doesn't know it yet.)

Unfortunately but now we can't get reach such big companies (it's not so easy to get the decision makers there).