PCL Developers blog
Changhyun Choi
| email: | cchoi@cc.gatech.edu |
|---|---|
| project: | 3D edge extraction from an organized point cloud |
| mentor: | Alex Trevor |
About me
I am a Ph.D. student in Center for Robotics and Intelligent Machines, College of Computing, Georgia Institute of Technology.
Project Description
The goal of our GSOC is to implement a 3D edge extraction algorithm from an organized point cloud. We are interested in the edges come from boundaries of either geometric structures or photometric texture. To find these edges, we will combine multiple cues from RGB-D channels. We plan to extract 2D edges from the given RGB channels and back-project these edges to the given depth channel so as to determine reliable 3D edges. We also plan to exploit edges from depth discontinuities or high curvature regions that may not be captured from the RGB channels. We want the code to provide several options to choose the types of resulting edges such as, depth discontinuity, RGB edge, high curvature, and etc.
Roadmap
Implement various types of edges
- Occluding boundary edges (edge of occluding object)
- Occluded boundary edges (edge on surface being occluded)
- High curvature edge
- 2D edge (image only edge, not curvature or occlusion boundary)
- 2D + high curvature edge
- 2D + occluding boundary edge
- 2D + occluded boundary edge
Implement a nice interface for those edges
Recent status updates
My primary goal of GSOC‘12 was to design and implement a 3D edge detection algorithm from an organized point cloud. Various edges are detected from geometric shapes (boundary, occluding, occluded, and high curvature edges) or photometric texture (rgb edges). These edges can be applicable to registration, tracking, etc. Following code shows how to use the organized edge detection:
pcl::OrganizedEdgeFromRGBNormals<pcl::PointXYZRGBA, pcl::Normal, pcl::Label> oed;
oed.setInputNormals (normal);
oed.setInputCloud (cloud);
oed.setDepthDisconThreshold (0.02); // 2cm
oed.setMaxSearchNeighbors (50);
pcl::PointCloud<pcl::Label> labels;
std::vector<pcl::PointIndices> label_indices;
oed.compute (labels, label_indices);
pcl::PointCloud<pcl::PointXYZRGBA>::Ptr occluding_edges (new pcl::PointCloud<pcl::PointXYZRGBA>),
occluded_edges (new pcl::PointCloud<pcl::PointXYZRGBA>),
boundary_edges (new pcl::PointCloud<pcl::PointXYZRGBA>),
high_curvature_edges (new pcl::PointCloud<pcl::PointXYZRGBA>),
rgb_edges (new pcl::PointCloud<pcl::PointXYZRGBA>);
pcl::copyPointCloud (*cloud, label_indices[0].indices, *boundary_edges);
pcl::copyPointCloud (*cloud, label_indices[1].indices, *occluding_edges);
pcl::copyPointCloud (*cloud, label_indices[2].indices, *occluded_edges);
pcl::copyPointCloud (*cloud, label_indices[3].indices, *high_curvature_edges);
pcl::copyPointCloud (*cloud, label_indices[4].indices, *rgb_edges);
For more information, please refer to following codes in PCL trunk:
- pcl-trunk/apps/src/pcd_organized_edge_detection.cpp.
- pcl-trunk/apps/src/openni_organized_edge_detection.cpp.
It was a great pleasure to be one of the GSOC participants. I hope that my small contribution will be useful to PCL users. Thank Google and PCL for the nice opportunity and kind support. Lastly, thank Alex Trevor for mentoring me.
An openni interface for 3D edge detection is added in PCL trunk. Once it starts, you can show and hide each edge by pressing the corresponding number:
- 1: boundary edges (blue)
- 2: occluding edges (green)
- 3: occluded edges (red)
- 4: high curvature edges (yellow)
- 5: rgb edges (cyan)
The high curvature and rgb edges are not enabled for fast frame rate, but you can easily enable these two edges if you want to test.
I have modified the previous code so that it finds possible edges from various point cloud types:
- OrganizedEdgeBase accepts PCL_XYZ_POINT_TYPES and returns EDGELABEL_NAN_BOUNDARY, EDGELABEL_OCCLUDING, and EDGELABEL_OCCLUDED.
- OrganizedEdgeFromRGB accepts PCL_RGB_POINT_TYPES and returns EDGELABEL_NAN_BOUNDARY, EDGELABEL_OCCLUDING, EDGELABEL_OCCLUDED, and EDGELABEL_RGB_CANNY.
- OrganizedEdgeFromNormals accepts PCL_XYZ_POINT_TYPES with PCL_NORMAL_POINT_TYPES and returns EDGELABEL_NAN_BOUNDARY, EDGELABEL_OCCLUDING, EDGELABEL_OCCLUDED, and EDGELABEL_HIGH_CURVATURE.
- OrganizedEdgeFromRGBNormals accepts PCL_RGB_POINT_TYPES with PCL_NORMAL_POINT_TYPES and returns EDGELABEL_NAN_BOUNDARY, EDGELABEL_OCCLUDING, EDGELABEL_OCCLUDED, EDGELABEL_HIGH_CURVATURE, and EDGELABEL_RGB_CANNY.
OrganizedEdgeFromRGB and OrganizedEdgeFromNormals are derived from OrganizedEdgeBase. OrganizedEdgeFromRGBNormals is then derived from both OrganizedEdgeFromRGB and OrganizedEdgeFromNormals.
I added RGB edge detection, which is 2D canny edge detection from the RGB channels, in pcl::OrganizedEdgeDetection. Right now, the class only takes point types having RGB channels, but it will be changed so that possible edges can be detected from a given point cloud type. For example, ‘occluding’, ‘occluded’, and ‘boundary’ edges can be detected from any XYZ point types. And ‘high curvature’ and ‘rgb’ edges can be obtained from Normal and RGB point types, respectively.
Following images show the detected occluding (green), occluded (red), boundary (blue), high curvature (yellow), and rgb (cyan) edges:
During this week, I have implemented ‘high curvature edge’ estimation. Here, the high curvature edges are defined as ridge or valley edges that are not on objects’ boundaries. These sharp edges could be useful in registration, recognition, and tracking applications. I first tried the curvature values obtained from normal estimation, but it turned out that a simple thresholding scheme using curvature values does not work very well. Especially, it was hard to get clean and thin edges on these high curvature regions. I noticed that non-maximum suppression and hysteresis thresholding are required. So I employed a canny edge implementation (pcl::pcl_2d::edge::canny()). Instead of using the RGB values available in the given organized point cloud, I used ‘normal_x’ and ‘normal_y’ images, since high gradient responses on these normal images correspond to high curvature regions.
Following images show the detected occluding (green), occluded (red), boundary (blue), and high curvature (yellow) edges:
