fahrzeugtechnik header


The last AutoAI conference in Berlin showed the importance of AI methods using Deep Neural Networks (DNN) and machine learning in the context of autonomous vehicles. One of the most important methods is using DNNs to do semantic segmentation of sensor data. Semantic Segmentation means that for every pixel in an image and every point in a point cloud an object class is assigned. This pixelwise assignment is the task of a DNN.

The DNNs used have to be trained by supervised learning using training data and the results have to be tested against test data. Both data sets have to be created by manuall pixelwise annotation. The labeling for semantic segmentation or semantic labeling is very time consuming and requires special tools. The next version of Philosys Label Editor allows to generate the ground-truth for semantic segmentation, also in combination with conventional 2D/3D labeling.


Sarting point for Semantic Labeling by using Philosys Label Editor are video images. They may come from an ADTF DAT file, like with conventional labeling, or be from any source or have any format by implementing and using a Player Plugin (PNG, JPG, ...). In the second case the image have to be converted to an image format compatible to one of the ADTF image formats. The images may be raw images or processed or rectified images.. 

Segmentation RAW FV


User Interface

The main user interface is similar to the one known before. Just like the Video3D Provider for 3D labeling now a Segmentation Provider is available. The Segmentation Provider can use the window manager just like the Video3D provider, so that content from multiple data streams can be displayed concurrently and/or by using tabbed windows.The following image shows the content of rear and front camera using tabbed windows. 

Segmentation LE FV annotated


The Segment Provider shows a tool box for the different tools. On the left side the configured segmentation clases are shown together with the defined color, value and the possibility to lock the class to prevent unintentional changes. The segmentation classes, color and values can be freely defined by definition the structure definition file.

Brush Tool

The brush tool allows to set pixels to a specific segment class. It works just like brush tools in most graphic programs. 

Segmentation Brush

Segmentation Brush Example


SuperPixel Tool

The superpixel tool segments the image into larger pixel areas with similar properties by using special algorithms. Multiple different algorithms are available.


Segmentation Superpixel

Segmentation Superpixel Example


FloodFill Tool

The FloodFill tool can be used to fill areas with similar color values. Smoothing filters can be used to filter high frequency component before applying the FloodFill tool.

Segmentation FloodFill

Segmentation Floodfill Example


PolyFill Tool

The PolyFill tool is used to mark areas with a polygon and set this areas to a specific segmentation class.

Segmentation Polyfill

Segmentation PolylineFill Example


AutoADD Tool

Like with conventional labeling with Philosys Label Editor there is an AutoAdd tool available which help with labeling video streams, by placing the semantic segmentation data from current frame to next frame. This is done by an intelligent way so that motion is taken into account. This reduces labeling effort because the image doesn't have to be segmented completely.  

Image Frame 1:


Segmentation Frame1


Image Frame 10 after AutoAdd without any manual correction:


Segmentation Frame10


For best results all errors of the automatically created segmentation have to be corrected before the next step, as the resulte will deteriorate more and more.


Label Format

The resulting segmentation image is stored as a blob containing a compressed PNG image within the defined XML label data format. 


Segmentation RAW FV image



The new function allows full semantic segmentation of images, and can be combined with conventional labeling in 2D/3D space. This gives a complete ground truth for recorded data with one tool.


Further Information


YouTube Videos


Philosys Label Editor Version 6 - Release 6.1

Philosys Label Editor Version 5 - Release 5.1

Why 3D labeling?


Philosys Label Editor is used during development and test of diverse Advanced Driver Assistance Systems (ADAS) for ground-truth-data collection. Various objects are manually or automatically marked and tagged with detailed traits. All kinds of vehicles, lane boundaries, traffic signs as well as pedestrians and wildlife are registered and verified by the assistance system for use during validation, and the data can be used for generating reference data for Deep Neural network (DNN) machine learning and validation.

The ADTF based Philosys Label Editor is available as of the beginning of 2011. Its many features facilitate the annotation of video scenes and reference data. This results in a significant cost reduction for the annotation process.