PCL Developers blog

Gioia Ballin

email:gioia.ballin@gmail.com
project:Additional functionalities and improvement for PCL modules
mentor:Federico Tombari and Alexandru-Eugen Ichim

About me

I am a recently graduated student from University of Padua, Italy, with a Master’s degree in Computer Engineering. My research interests include computer vision and robotics, with particular attention to tracking and recognition tasks and I am looking for an appealing PhD position on these subjects. From the spare time side, my hobbies are doing sports and travelling.

Project Description

My project aims to improve the existing tools in PCL, while adding at the same time new functionalities at the state of the art of the research. The focus is on the following modules:

  • pcl_keypoints
  • pcl_recognition

Roadmap

In what follows, I show an outline of my GSoC roadmap. The boldface writings represent my current activities.

  • Module keypoints:
    • First approach to the object recognition research field: documentation on techniques at the state of the art.
    • Test functionalities for the new 3D object recognition tutorial and the existing detectors in PCL.
    • Implement a framework for simple keypoint detection evaluation.
    • Extend the framework in order to perform tests on complete datasets.
    • Porting of the ISS detector in PCL.
    • Test the performances of the ISS detector.
  • Tools:
    • Develop a new point type to handle monochrome images.
    • Develop a PNG to PCD converter.

Click here for a detailed roadmap about my GSoC activities.

Recent status updates

A new point type to handle monochrome images is now available on the trunk!
Saturday, August 18, 2012
../../_images/gsoc124.png

I’ve developed a new point type to handle monochrome images in the most effective way. It contains only one field, named intensity, of type uint8_t. Consequently, I’ve updated the pcl::ImageViewer so that it is able to manage the point clouds related to the new point type, named pcl::Intensity. Finally, I’ve changed the PNG2PCD converter by adding more functionalities: now the user can choose if the written cloud should be based on pcl::RGB or on pcl::Intensity. For more information, please see the documentation related to each single class or file.

A new tool for PNG to PCD conversions is now available on the trunk!
Friday, August 10, 2012
../../_images/gsoc124.png

I’ve developed a simple utility that enables the user to convert a PNG input file into a PCD output file. The converter takes as input both the name of the input PNG file and the name of the PCD output file. It finally performs the conversion of the PNG file into the PCD file by creating a:

pcl::PointCloud<pcl::RGB>>

point cloud. Now, the PNG2PCD converter is available on the trunk version of PCL under the tools directory.

Fair tests on the Stanford dataset 2
Sunday, August 05, 2012
../../_images/gsoc124.png

I’ve finally completed the section Detectors evaluation: repeatability and time performances and it is now available for consulting.

Fair tests on the Stanford dataset 2
Saturday, August 04, 2012
../../_images/gsoc124.png

In what follows I’ll show the main results of the tests on the Stanford dataset. The complete results will be given in Detectors evaluation: repeatability and time performances. The fairness of the comparison is ensured by fixing properly some common parameters among all the detectors. ISS has clearly the best relative repeatability among all the PCL detectors that have been under testing. With regards to the legend, the same considerations made in the previous blog post apply this time.

../../_images/absolute_repeatability_all3.png
../../_images/relative_repeatability_all3.png
../../_images/time_performances_all_6cr1.png
../../_images/iss_time_performances_6cr1.png
Fair tests on the Kinect dataset
Thursday, August 02, 2012
../../_images/gsoc124.png

In what follows I’ll show the main results of the tests on the kinect dataset. The complete results are given in Detectors evaluation: repeatability and time performances. The fairness of the comparison is ensured by fixing properly some common parameters among all the detectors. NARF and ISS are clearly the two best PCL detectors among the ones tested.

I think it is required a brief explanation of the graphs legend. First the harris 3D detector is tested with all its possible response methods, so the abbreviations HA, NO, LO, TO, and CU respectively refer to the response methods: HARRIS, NOBLE, LOWE, TOMASI, CURVATURE. The abbreviation ISS - WBE refers to the execution of the ISS 3D detector without the border extraction, while ISS - BE refers to the execution of the detector with the extraction of boundary points. Finally, the abbreviations 1 th, 2 th, 3 th and 4 th stand for 1, 2, 3 and 4 threads and they are related to the OpenMP optimization of the code.

../../_images/absolute_repeatability_all2.png
../../_images/relative_repeatability_all2.png
../../_images/time_performances_all_6cr.png
../../_images/iss_time_performances_6cr.png
News on tests
Wednesday, August 01, 2012
../../_images/gsoc124.png

While developing the evaluator for the ISS 3D detector, I realized that I set the harris 3D and 6D normal estimation radius to:

multiplier * cloud_resolution

where multiplier is set to 6, 9, 12 and 15 cloud_resolution each time. Instead of this setting, I should have set the normal estimation radius to:

4 * cloud_resolution

in order to obtain fair tests. The previous tests are valid but the reader should take into account this consideration. While testing the ISS 3D detector I re-run the tests to obtain the desidered fairness. I have also decided to collect this final tests in a specific blog page, so as the user can immediately reach the results without looking for it in all my posts. I have just completed the evaluation on the Kinect dataset and I will post it soon both on the blog and on Detectors evaluation: repeatability and time performances.

First pictures from the ISS detector
Saturday, July 28, 2012
../../_images/gsoc124.png

In what follows I’ll show some snapshots related to the behaviour of the ISS detector. They have been obtained by not setting the border radius.

Shapshots from the Kinect dataset.

  • Model:

    ../../_images/model_screenshot041.png
  • Scene:

    ../../_images/scene_screenshot095.png
  • Results:

    • Absolute repeatability: 8
    • Relative repeatability: 0.195122

Shapshots from the Stanford 1 dataset.

  • Model:

    ../../_images/model_screenshot058.png
  • Scene:

    ../../_images/scene_screenshot070.png
  • Results:

    • Absolute repeatability: 413
    • Relative repeatability: 0.769088
ISS is available on trunk now!
Saturday, July 28, 2012
../../_images/gsoc124.png

Now, ISS is available on the trunk and it is properly documented. In the section how to use the ISS 3D keypoint detector of this blog I will post some code snippets useful to the user who wants to exploit the ISS detector. Currently, the ISS detector is under testing. It will be tested for different configurations:

  1. using 1 thread and disabling the boundary estimation.
  2. using 1 thread and enabling the boundary estimation.
  3. using 1, 2, 3 and 4 threads (4 is the maximum number of threads allowed in my system) and chosing the best configuration among that described previously in 1. and 2. .

The tests will show both the repeatability results and the time performances of the detector. The results related to the configurations 1. and 2. can be compared with the results already obtained for the other detectors tested at the beginning of my GSoC work.

Coming soon: the test results for the ISS detector.

Working hard on ISS refinement
Saturday, July 21, 2012
../../_images/gsoc124.png

Some time has passed since my last post. My recent and current activities regard mainly the ISS detector. First of all, I completed the detector so that it has a basic functionality (e.g. it works ! ). In order to be sure that it really works, I developed a simple evaluation framework at the same time. This framework looks like the basic frameworks I developed at the beginning of the GSoC code sprint and it allows the user both to compute the repeatability of the detector based on a pair of files (model + scene) and to visualize the resulting keypoints. My current activities regards the refinement of the ISS detector, and now I’m particularly focusing on time performances. After a brief analysis about the time performances of the detector, I refined it by using low-cost data structures. Now, I’m working on the introduction of the openMP directives in order to further speed-up the computation when 2 or more cores are avalilable. The detector will be available in the trunk when it will be fully completed and optimized.

Stay tuned with my roadmap: it always shows my current activities even if I’m not posting so much.

Working in parallel on new detectors: GSS and ISS
Wednesday, July 11, 2012
../../_images/gsoc124.png

Last week I started working on GSS but I stopped this week since I need some feedback from Alex and he may be on vacation now. In order not to lose much time, I have decided to switch to the implementation of ISS. After reading the related paper and after taking some knowledge on the code already implemented, I have defined the input and output parameters of the detector and the operations that have to be done in order to compute the keypoints. Then, I have prepared a skeleton class and now I’m currently filling and extending it with the implementation of new methods.

What were my other activities during the past and the current week? Bug solving and user support.