Online Panoptic 3D Reconstruction as a Linear Assignment Problem

Leevi Raivio, Esa Rahtu

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Real-time holistic scene understanding would allow machines to interpret their surrounding in a much more detailed manner than is currently possible. While panoptic image segmentation methods have brought image segmentation closer to this goal, this information has to be described relative to the 3D environment for the machine to be able to utilise it effectively. In this paper, we investigate methods for sequentially reconstructing static environments from panoptic image segmentations in 3D. We specifically target real-time operation: the algorithm must process data strictly online and be able to run at relatively fast frame rates. Additionally, the method should be scalable for environments large enough for practical applications. By applying a simple but powerful data-association algorithm, we outperform earlier similar works when operating purely online. Our method is also capable of reaching frame-rates high enough for real-time applications and is scalable to larger environments as well. Source code and further demonstrations are released to the public at:

Original languageEnglish
Title of host publicationImage Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings
EditorsStan Sclaroff, Cosimo Distante, Marco Leo, Giovanni M. Farinella, Federico Tombari
Number of pages12
ISBN (Electronic)9783031064302
ISBN (Print)9783031064296
Publication statusPublished - 2022
Publication typeA4 Article in conference proceedings
EventInternational Conference on Image Analysis and Processing - Lecce, Italy
Duration: 23 May 202227 May 2022

Publication series

NameLecture Notes in Computer Science
Volume13232 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceInternational Conference on Image Analysis and Processing


  • 3D reconstruction
  • Panoptic segmentation
  • Real time

Publication forum classification

  • Publication forum level 1

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Online Panoptic 3D Reconstruction as a Linear Assignment Problem'. Together they form a unique fingerprint.

Cite this