The great advantage of doing Computer Vision in ROS is the fact that we do not have to re-invent the wheel. A lot of third-party software is available, and we can also connect our vision stuff to the real robots or do some simulations. Here, we are going to enumerate interesting Computer Vision tools for the most common visual tasks, but we will only explain in detail one of them later, including all the steps to set it up. We will do it for visual odometry, but other packages are also easy to install and it is also easy to start playing with them; simply follow the tutorials or manuals in the links provided here:
- Visual Servoing: Also known as Vision-based Robot Control, this is a technique that uses feedback information obtained from a vision sensor to control the motion of a robot, typically an arm used for grasping. In ROS, we have a wrapper of the Visual Servoing Platform (ViSP) software (http://www.irisa.fr/lagadic/visp/visp.html, http://www.ros.org/wiki/vision_visp). ViSP is a complete cross-platform library that allows prototyping and developing applications in visual tracking and visual servoing. The ROS wrapper provides a tracker that can be run with the
visp_tracker(moving edge tracker) node as well asvisp_auto_tracker(a model-based tracker). It also helps to calibrate the camera and perform hand-to-eye calibration, which is crucial for visual servoing in grasping tasks. - Augmented Reality: An Augmented Reality (AR) application involves overlaying virtual imagery on the real world. A well-known library for this purpose is ARToolkit (http://www.hitl.washington.edu/artoolkit/). The main problem in this application is tracking the user's viewpoint to draw the virtual imagery on the viewpoint where the user is looking in the real world. ARToolkit video tracking libraries calculate the real camera position and orientation relative to physical markers in real time. In ROS, we have a wrapper named
ar_pose(http://www.ros.org/wiki/ar_pose). It allows us to track single or multiple markers where we can render our virtual imagery (for example, a 3D model). - Perception and object recognition: Most basic perception and object recognition is possible with the OpenCV libraries. However, there are several packages that provide an object recognition pipeline, such as the
object_recognitionstack, which providestabletop_object_detectorto detect objects on a table, for example; a more general solution provided by Object Recognition Kitchen (ORK) can be found at http://wg-perception.github.io/object_recognition_core. It is also worth mentioning a tool called RoboEarth (http://www.roboearth.org), which allows you to detect and build 3D models of physical objects and store them in a global database accessible for any robot (or human) worldwide. The models stored can be 2D or 3D and can be used to recognize similar objects and their viewpoint, that is, to identify what the camera/robot is watching. The RoboEarth project is integrated into ROS, and many tutorials are provided to have a running system (http://www.ros.org/wiki/roboearth). - Visual odometry: A visual odometer is an algorithm that uses the images of the environment to track features and estimate the robot's movement, assuming a static environment. It can solve the 6 DoF pose of the robot with a monocular or stereo system, but it may require additional information in the monocular case. There are two main libraries for visual odometry:
libviso2(http://www.cvlibs.net/software/libviso2.html) andlibfovis(http://www.ros.org/wiki/fovis_ros), both of which have wrappers for ROS. The wrappers just expose these libraries to ROS. They are theviso2andfovisstacks respectively. In the next section, we will see how we can do visual odometry with our homemade stereo camera using theviso2_rosnode ofviso2. Thelibviso2library allows us to do monocular and stereo visual odometry. However, for monocular odometry, we also need the pitch and heading for the ground plane estimation. You can try the monocular case with one camera and an IMU (see Chapter 4, Using Sensors and Actuators with ROS), but you will always have better results with a good stereo pair, correctly calibrated, as seen so far in this chapter. Finally,libfovisdoes not allow the monocular case, but it supports RGBD cameras, such as the Kinect sensor (see Chapter 6, Point Clouds). As regards the stereo case, it is possible to try both libraries and see which one works better in your case. Here, we show a step-by-step tutorial to install and runviso2in ROS andfoviswith Kinect.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.