Online Panoptic 3D Reconstruction as a Linear Assignment Problem

Leevi Raivio, Esa Rahtu

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

Real-time holistic scene understanding would allow machines to interpret their surrounding in a much more detailed manner than is currently possible. While panoptic image segmentation methods have brought image segmentation closer to this goal, this information has to be described relative to the 3D environment for the machine to be able to utilise it effectively. In this paper, we investigate methods for sequentially reconstructing static environments from panoptic image segmentations in 3D. We specifically target real-time operation: the algorithm must process data strictly online and be able to run at relatively fast frame rates. Additionally, the method should be scalable for environments large enough for practical applications. By applying a simple but powerful data-association algorithm, we outperform earlier similar works when operating purely online. Our method is also capable of reaching frame-rates high enough for real-time applications and is scalable to larger environments as well. Source code and further demonstrations are released to the public at: https://tutvision.github.io/Online-Panoptic-3D/.

Original languageEnglish
Title of host publicationImage Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings
EditorsStan Sclaroff, Cosimo Distante, Marco Leo, Giovanni M. Farinella, Federico Tombari
PublisherSpringer
Pages39-50
Number of pages12
ISBN (Electronic)9783031064302
ISBN (Print)9783031064296
DOIs
Publication statusPublished - 2022
Publication typeA4 Article in conference proceedings
EventInternational Conference on Image Analysis and Processing - Lecce, Italy
Duration: 23 May 202227 May 2022

Publication series

NameLecture Notes in Computer Science
Volume13232 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceInternational Conference on Image Analysis and Processing
Country/TerritoryItaly
CityLecce
Period23/05/2227/05/22

Keywords

  • 3D reconstruction
  • Panoptic segmentation
  • Real time

Publication forum classification

  • Publication forum level 1

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Online Panoptic 3D Reconstruction as a Linear Assignment Problem'. Together they form a unique fingerprint.

Cite this