Introduction
The last AutoAI conference in Berlin showed the importance of AI methods using Deep Neural Networks (DNN) and machine learning in the context of autonomous vehicles. One of the most important methods is using DNNs to do semantic segmentation of sensor data. Semantic Segmentation means that for every pixel in an image and every point in a point cloud an object class is assigned. This pixelwise assignment is the task of a DNN.
The DNNs used have to be trained by supervised learning using training data and the results have to be tested against test data. Both data sets have to be created by manuall pixelwise annotation. The labeling for semantic segmentation or semantic labeling is very time consuming and requires special tools. The next version of Philosys Label Editor allows to generate the ground-truth for semantic segmentation, also in combination with conventional 2D/3D labeling.
Description
Sarting point for Semantic Labeling by using Philosys Label Editor are video images. They may come from an ADTF DAT file, like with conventional labeling, or be from any source or have any format by implementing and using a Player Plugin (PNG, JPG, ...). In the second case the image have to be converted to an image format compatible to one of the ADTF image formats. The images may be raw images or processed or rectified images..
User Interface
The main user interface is similar to the one known before. Just like the Video3D Provider for 3D labeling now a Segmentation Provider is available. The Segmentation Provider can use the window manager just like the Video3D provider, so that content from multiple data streams can be displayed concurrently and/or by using tabbed windows.The following image shows the content of rear and front camera using tabbed windows.
The Segment Provider shows a tool box for the different tools. On the left side the configured segmentation clases are shown together with the defined color, value and the possibility to lock the class to prevent unintentional changes. The segmentation classes, color and values can be freely defined by definition the structure definition file.
Brush Tool
The brush tool allows to set pixels to a specific segment class. It works just like brush tools in most graphic programs.
SuperPixel Tool
The superpixel tool segments the image into larger pixel areas with similar properties by using special algorithms. Multiple different algorithms are available.
FloodFill Tool
The FloodFill tool can be used to fill areas with similar color values. Smoothing filters can be used to filter high frequency component before applying the FloodFill tool.
PolyFill Tool
The PolyFill tool is used to mark areas with a polygon and set this areas to a specific segmentation class.
AutoADD Tool
Like with conventional labeling with Philosys Label Editor there is an AutoAdd tool available which help with labeling video streams, by placing the semantic segmentation data from current frame to next frame. This is done by an intelligent way so that motion is taken into account. This reduces labeling effort because the image doesn't have to be segmented completely.
Image Frame 1:
Image Frame 10 after AutoAdd without any manual correction:
For best results all errors of the automatically created segmentation have to be corrected before the next step, as the resulte will deteriorate more and more.
Label Format
The resulting segmentation image is stored as a blob containing a compressed PNG image within the defined XML label data format.
Summary
The new function allows full semantic segmentation of images, and can be combined with conventional labeling in 2D/3D space. This gives a complete ground truth for recorded data with one tool.
Further Information
Philosys Label Editor Version 6 - Release 6.1
Philosys Label Editor Version 5 - Release 5.1
Philosys Label Editor is used during development and test of diverse Advanced Driver Assistance Systems (ADAS) for ground-truth-data collection. Various objects are manually or automatically marked and tagged with detailed traits. All kinds of vehicles, lane boundaries, traffic signs as well as pedestrians and wildlife are registered and verified by the assistance system for use during validation, and the data can be used for generating reference data for Deep Neural network (DNN) machine learning and validation.
The ADTF based Philosys Label Editor is available as of the beginning of 2011. Its many features facilitate the annotation of video scenes and reference data. This results in a significant cost reduction for the annotation process.