PCL Developers blog
Code Sprints
Q: What is a code sprint?
A code sprint is an accelerated research and development program, where we pair talented developers with senior researchers and engineers for 3-6 months of extended programming work. The model is inspired by the Google Summer Of Code initiative, and is meant to create open source solutions for interesting 2D/3D computer perception problems. Each code sprint is financially supported by a different institution (see below).
Code sprints are made possible through the involvement of our host organization, Open Perception. Please see our mission page for more information.
Q: What does this blog represent?
The Point Cloud Library (PCL) developer blog represents a great way to keep track of daily updates in PCL sponsored code sprints. Our developers are writing about their experiences and progress updates while working on exciting PCL projects.
Active Sprints
The following shows the current list of ongoing code sprints. Click on any of the logos below to go to the respective sprint blog page.
Completed Sprints
The list of code sprints which have been completed is:
We would like to thank our financial sponsors for their generous support. For details on how to start a new code sprint or contribute to PCL, please visit http://www.pointclouds.org/about and our organization’s web site at http://www.openperception.org/get-involved/.
Latest 15 blog updates
The project “Stereo-based road area detection” has been started!
This is the first post of the Spectrolab PCL Code Sprint. Spectrolab is a Boeing company that has developed a new LIDAR camera. This camera is meant for industrial and robotics uses in both outdoor and indoor environments. It uses a scanning time of flight laser to generate a 256x128 range image at 5-6 hertz. It is still in development right now, but it promises to be a valuable tool for robotics and perception work. Spectrolab has sponsored integration of the camera into PCL so that when it hits the market, it can take advantage of the wealth of tools and knowledge within the PCL community.
As apart of the code sprint, I will be developing a new PCL grabber driver for the Spectrolab camera and a 3D viewer interface. As I complete the driver and software, I will post screen shots and code explanations to this blog.
Test
Here’s a video demonstrating the RobotEye Viewer application with the Ocular Robotics RE05 lidar sensor. See my previous blog post for a description of the application.
Last two weeks I’ve finished the NPP implementation of the Haar Cascade detector, it is now fully ported to PCL. I’ve included Alex Trevor his organized multi-plane segmentation to people api as well, so that people can do background subtraction based on the planes found in the image. This week I wrote the final report summarizing the work done as well as how to use the module.
I’ve finished developing a new pcl::Grabber called the pcl::RobotEyeGrabber, and a visualization application called RobotEye Viewer. The grabber uses the boost asio library to do asynchronous I/O on a UDP socket using a separate thread. The RobotEye sensor sends lidar data in UDP data packets which are converted by the grabber into pcl::PointCloud data. The RobotEye Viewer is a Qt application that uses the QVTKWidget to embed a pcl::visualization::PCLVisualizer visualization window.
I developed the code using MacOSX and Ubuntu Linux. I ssh to a machine that is located at the Ocular Robotics lab and has a live sensor connected to it. With this machine, I was able to run the RobotEye Viewer application remotely using the RE05 lidar sensor. Ocular Robotics setup a webcam that points at the RobotEye sensor so I can see it in action as I run the application remotely. This setup worked out very well. Here’s a screenshot of the application, stay tuned for a video.
The PCL Dinast Grabber Framework
At PCL 1.7 we offer a new driver for Dinast Cameras making use of the generic grabber interface that is present since PCL 1.0. This tutorial shows, in a nutshell, how to set up the pcl grabber to obtain data from the cameras.
So far it has been currently tested with the IPA-1110, Cyclopes II and the IPA-1002 ng T-Less NG but it is meant to work properly on the rest of the Dinast devices, since manufacturer specifications has been taken into account.
Small example
As the Dinast Grabber implements the generic grabber interface you will see high usage similarities with other pcl grabbers. In applications you can find a small example that contains the code required to set up a pcl::PointCloud<XYZI> callback to a Dinast camera device.
Here you can see a screenshot of the PCL Cloud Viewer showing the data from a cup laying on a table obtained through the Dinast Grabber interface:
And this is a video of the PCL Cloud Viewer showing the point cloud data corresponding to a face:
Dinast Grabber currently offer this data type, as is the one currently available from Dinast devices:
- void (const boost::shared_ptr<const pcl::PointCloud<pcl::PointXYZI> >&)
The code
The code from apps/src/dinast_grabber_example.cpp will be used for this tutorial:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | #include <pcl/common/time.h>
#include <pcl/point_types.h>
#include <pcl/io/dinast_grabber.h>
#include <pcl/visualization/cloud_viewer.h>
template <typename PointType>
class DinastProcessor
{
public:
typedef pcl::PointCloud<PointType> Cloud;
typedef typename Cloud::ConstPtr CloudConstPtr;
DinastProcessor(pcl::Grabber& grabber) : interface(grabber), viewer("Dinast Cloud Viewer") {}
void
cloud_cb_ (CloudConstPtr cloud_cb)
{
static unsigned count = 0;
static double last = pcl::getTime ();
if (++count == 30)
{
double now = pcl::getTime ();
std::cout << "Average framerate: " << double(count)/double(now - last) << " Hz" << std::endl;
count = 0;
last = now;
}
if (!viewer.wasStopped())
viewer.showCloud(cloud_cb);
}
int
run ()
{
boost::function<void (const CloudConstPtr&)> f =
boost::bind (&DinastProcessor::cloud_cb_, this, _1);
boost::signals2::connection c = interface.registerCallback (f);
interface.start ();
while (!viewer.wasStopped())
{
boost::this_thread::sleep (boost::posix_time::seconds (1));
}
interface.stop ();
return(0);
}
pcl::Grabber& interface;
pcl::visualization::CloudViewer viewer;
};
int
main ()
{
pcl::DinastGrabber grabber;
DinastProcessor<pcl::PointXYZI> v (grabber);
v.run ();
return (0);
}
|
The explanation
At first, when the constructor of DinastProcessor gets called, the Grabber and CloudViewer Classes are also initialized:
DinastProcessor(pcl::Grabber& grabber) : interface(grabber), viewer("Dinast Cloud Viewer") {}
At the run function what we first have is actually the callback and its registration:
boost::function<void (const CloudConstPtr&)> f =
boost::bind (&DinastProcessor::cloud_cb_, this, _1);
boost::signals2::connection c = interface.registerCallback (f);
We create a boost::bind object with the address of the callback cloud_cb_, we pass a reference to our DinastProcessor and the argument place holder _1. The bind then gets casted to a boost::function object which is templated on the callback function type, in this case void (const CloudConstPtr&). The resulting function object is then registered with the DinastGrabber interface.
The registerCallback call returns a boost::signals2::connection object, which we do not use in the this example. However, if you want to interrupt or cancel one or more of the registered data streams, you can call disconnect the callback without stopping the whole grabber:
boost::signals2::connection = interface (registerCallback (f));
// ...
if (c.connected ())
c.disconnect ();
After the callback is set up we start the interface. Then we loop until the viewer is stopped. Finally interface is stopped although this is not actually needed since the destructor takes care of that.
On the callback function cloud_cb_ we just do some framerate calculations and we show the obtained point cloud through the CloudViewer.
Testing the code
We will test the grabber with the previous example. Write down the whole code to a file called dinast_grabber.cpp at your preferred location. Then add this as a CMakeLists.txt file:
1 2 3 4 5 6 7 8 9 10 11 12 | cmake_minimum_required(VERSION 2.8 FATAL_ERROR)
project(dinast_grabber)
find_package(PCL 1.7 REQUIRED)
include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})
add_executable (dinast_grabber dinast_grabber.cpp)
target_link_libraries (dinast_grabber ${PCL_LIBRARIES})
|
Then just proceed as a usual cmake compilation:
$ cd /PATH/TO/DINAST_EXAMPLE
$ mkdir build
$ cd build
$ cmake
$ make
If everything went as expected you should now have a binary to test your Dinast device. Go ahead, run it and you should be able to see the point cloud data from the camera:
$ ./dinast_grabber
Troubleshooting
Q: When I run the application I get an error similar to this one:
$ ./dinast_grabber
libusb: 0.000000 error [op_open] libusb couldn't open USB device /dev/bus/usb/002/010: Permission denied.
libusb: 0.009155 error [op_open] libusb requires write access to USB device nodes.
Where the last numbers of the /dev/bus/usb/... might vary.
A: This means you do not have permission to access the device. You can do a quick fix on the permissions of that specific device:
$ sudo chmod 666 /dev/bus/usb/002/010
Or you can make this changes permanent for all future Dinast devices writing a rule for udev. In debian-like systems it is usually done writing this:
# make dinast device mount with writing permissions (default is read only for unknown devices)
SUBSYSTEM=="usb", ATTR{idProduct}=="1402", ATTR{idVendor}=="18d1", MODE:="0666", OWNER:="root", GROUP:="video"
to a file like /etc/udev/rules.d/60-dinast-usb.rules.
If you still have problems you can always use the users mailing list: pcl-users@pointclouds.org to find some extra help.
Conclusions
With this new grabber a new kind of short-range sensors are available through the PCL Grabber interface. It is now a breeze to connect and obtain data from Dinast devices as you do with the rest of devices supported at PCL.
If you have any development suggestions on these or new devices you can contact us through pcl-developers@pointclouds.org.
It has been fast time since VLCS starting. Keven announces “pcl_hdl_simple_viewer” in Dec. 2012. Now, it supports live mode (network) and recorded file mode (PCAP file) with several visualization format (XYZ, XYZI, XYZRGB). He mainly develops under Linux-Fedora. I have tested and supported under Linux-Ubuntu and Windows 7.
For windows implementation, Hdl_simple_viewer needs WinPcap
- WinPcap user version, WinPcap development version (with environment setting PCAPDIR). It will be explained on document later.
- WinPcap Windows Installer : http://www.winpcap.org/install/default.htm
- WinPcap Developer’s Pack : http://www.winpcap.org/devel.htm
Unfortunately, PCAP is not stable under Windows 64bits. It takes a long time to know about it. So, I build up 32-bit development environment.
Ubuntu and Windows test image are shown as belows
Recording file mode(PCAP file) - road.pcap from velodyne website.
Ubuntu XYZ / XYZI / XYZRGB
Windows XYZ / XYZI / XYZRGB
Live mode (Live mode also provides three visualization format).
Ubuntu / windows
Ocular Robotics sent me some lidar scans to get started with. The data is stored in binary files with azimuth, elevation, range, and intensity fields. I wrote some code to load the binary data and convert it to the pcl::PointCloud data structure. From there, I saved the data in the pcd file format and opened it with ParaView using the PCL Plugin for ParaView. I used ParaView’s animation controls to create a video of the lidar scan:
I’m pleased to announce that PCL now supports the Velodyne High Definition Laser (HDL) -32 and -64 lasers. The interface is provided as a Grabber (HDL_Grabber) and accepts packets from the network (live) or PCAP file (recorded). Two sample programs, pcl_hdl_grabber and pcl_hdl_viewer_simple are provided to demonstrate the capabilities of the system. libpcap-devel is required to build the PCAP reading portions of code.
Sample PCAP Files are provided by Velodyne: http://velodyne.com/lidar/doc/Sample%20sets/HDL-32/
The image below came from the Building-Side.pcap.
I’m excited to start the PCL - Ocular Robotics code sprint. This week I’ll have a kickoff meeting over Skype with engineers from Ocular Robotics. In the meantime, I’m reading about the RE0x sensors and updating my PCL build directories!
The Honda Research Code Sprint for ground segmentation from stereo has been completed. PCL now includes tools for generating disparity images and point clouds from stereo data courtesy of Federico Tombari, as well as tools for segmenting a ground surface from such point clouds from myself. Attached is a report detailing the additions to PCL and the results, as well as a video overview of the project. There is a demo available in trunk apps, as pcl_stereo_ground_segmentation.
It has been quite a long time since my last post so I will give a full update on all the developments that I have taken care of in this time.
After a first implementation of the pcl::DinastGrabber for Dinast cameras (IPA-1002, IPA-1110, IPA-2001) I did some testing with multiple cameras. For that I mounted two of them on a robotics arm. After calibration, I got the combination of two pointclouds in one single view. The following picture shows the result of me standing in front of the two cameras.
Also here is the setup of the cameras on the robotics manipulator:
Then we used the RRT implementation with the robotic arm for planning purposes. The calibrated cameras where also used to build a collision shield around the robot. First RRT was run to get the path to the goal. When the robot was moving, collision checking was performed using the information obtained from the cameras. When an object was detected, the robot was stopped and RRT was run again in order to obtain a new path to the goal avoiding the detected object. The pic below shows the replanning part of the whole testing, while trying to avoid a box that was in the path to the goal.
Over the weekend, I finished the URCS final report. I would like to thank my mentors, Jacob Schloss at Urban Robotics, Radu, and Julius, who all provided me a lot of patient support and inspiration on this project. Justin Rosen on TRCS has been a great collaborator and deserves a lot of credit for the success of the outofcore integration.
I would like to draw some attention to a few items related to outofcore that I explain in this document in further detail. First of all, even though the URCS is drawing to a close, outofcore is sill very much under development. The API is not 100% set, though significant strides have been made in restructuring the internals for long term stability and integration with PCL. Most importantly, all functionality is available for PointCloud2-based insertion and query methods now! These should be the preferred methods of the interface.
Currently, for the average user, the pipeline will be:
- Construct an outofcore octree with pcl_outofcore_process from a set of PCD files
- Use one of the two available outofcore viewers for visualization.
- When building LOD, remember the number of internal root nodes can grow exponentially the deeper the tree gets. As a consequence, building LOD can take quite a long time if you are inserting tens or hundreds of millions of points into a tree that is very deep.
- pcl_outofcore_process computes the size of the bounding box of the ENTIRE set of PCD files. That said, if you want to update the tree later, it is your responsibility to make sure the points fall within the original bounding box. If you want to set an arbitrarily large bounding box, the source of pcl_outofcore_process is easy to modify for this case.
For developers, I should emphasize two things to ensure compatibility with future changes that will be introduced into outofcore. If anyone has opinions on this, we would certainly like to entertain some discussion on the pcl-developers@ mailing list.
- The classes are still templated, but this will change. Because rendering is dynamic, please use the PointCloud2 methods of the outofcore octree. It is admittedly a little confusing because the interfaces are still in a single class with overloaded methods. The easiest way to handle this while it is in a state of flux is via a typedef:
typedef OutofcoreOctreeBase<OutofcoreOctreeDiskContainer<pcl::PointXYZ>, pcl::PointXYZ> OutofcoreBase;
Then, FORGET that it is templated on pcl::PointXYZ; it is not important. You can use whatever Point Type you would like packed into a PointCloud2 data structure. You can easily convert your PointCloud to a PointCloud2 by using toRosMsg.
- I also emphasize that Outofcore, while an octree, is NOT related to pcl_octree at this time. I have added some methods to make their interfaces somewhat similar, but please beware of a false sense of consistency.
The final report should contain enough information to get started developing with outofcore. I hope it also provides a sense of where the code is coming from and where the library is heading in the context of its future place in the PCL library.
Filters points lying within the frustum of the camera. The frustum is defined by pose and field of view of the camera. The parameters to this method are horizontal FOV, vertical FOV, near plane distance and far plane distance. I have added this method in the filters module. The frustum and the filtered points are shown in the images below.
