Object Classification and Image Labeling using RGB-Depth Information

Sammanfattning: This thesis is part of research for the vision systems of four robots in the EU funded project, CROPS[l], where the robots should harvest apples, sweet peppers and grapes, and explore forests. The whole process of designing such a system, including the software architecture, creation of the image database, image labeling and object detection, is presented.The software architecture chapter of this thesis provides a review of some of the latest frameworks for robotics. It describes the structure of robetics components and the communication systems between them. The vision system is a subsystem of the robetics architecture studied in conjunction with other components of the robetics system. To build a vision system, three main steps should be taken. First a collection of images that are similar to what the robots are expected to meet should be created. Second, all the images should be labeled manually or automatically. Finally, learning systems should use the labeled images to build object medels. Details about these steps make up the majority of the content in this thesis. With new widely available low-cost sensors such as Microsoft Kinect, it is possible to use depth imagesalong with RGB images to increase the performance of vision systems. We particularly focus on various methods that integrate depth information in the three steps that are mentioned for building a vision system. More specifically, the image labeling tools help to extract objects in images to be used as ground truth for learning and testing processes in object detection. The inputs for such tools are usually RGB images. Despite the existence of many powerful tools for image labeling, there is still a need for RGB-Depth adapted tools. We present a new interactive labeling tool that partially automates image labeling, with two major contributions. First, the method extends the concept of image segmentatian from RGB to RGB-Depth. Second, it minimizes the interadion time needed for object extraction by using a highly efficient segmentatian method in RGB-Depth space. The ent~re procedure requires very few elieks campared to other already existing tools. In fact, when the desired object is the dosest object to the camera, as is the case in our forestry application, no click is required to extract the object. Finally, while we present state of the art in object detection with 2D environments, object detection using RGB-depth information is mainly addressed for future work.