Datasets › SUN RGB-D

SUN RGB-D

Introduced by Shuran Song et al. in SUN RGB-D: A RGB-D Scene Understanding Benchmark Suite1 Jan 2015 archive 2025-07-28

The SUN RGBD dataset contains 10335 real RGB-D images of room scenes. Each RGB image has a corresponding depth and segmentation map. As many as 700 object categories are labeled. The training and testing sets contain 5285 and 5050 images, respectively.

Source: Mix and match networks: multi-domain alignment for unpaired image-to-image translation Image Source: https://rgbd.cs.princeton.edu/

Benchmarks archive 2025-07-28

All 11 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

Syntology 3 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): Semantic Segmentation · SUN-RGBD (2 judged, 0 placed), Scene Recognition · SUN-RGBD (1 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).

First row (archive order)PaperCode
Semantic Segmentation SUN-RGBD GeminiFusion (Swin-Large) Mean IoU 54.6 GeminiFusion: Efficient Pixel-wise Multimodal Fusion for... jiadingcn/geminifusion 44 Compare
3D Object Detection SUN-RGBD val Point-GCC+TR3D+FF mAP@0.25 69.7 Point-GCC: Universal Self-supervised 3D Scene... asterisci/point-gcc 32 Compare
3D Object Detection SUN-RGBD Uni3DETR mAP@0.25 67.0 Uni3DETR: Unified 3D Detection Transformer zhenyuw16/uni3detr 8 Compare
Monocular 3D Object Detection SUN RGB-D IM3D AP@0.15 (NYU-37) 24.10 Holistic 3D Scene Understanding from a Single Image with... chengzhag/Implicit3DUnderstanding 7 Compare
Object Detection In Indoor Scenes SUN RGB-D YONOD + CPPM (RGB + Depth) AP 0.5 58.1 Unified Object Detector for Different Modalities based... liketheflower/uoddm 7 Compare
Room Layout Estimation SUN RGB-D IM3D IoU 64.4 Holistic 3D Scene Understanding from a Single Image with... chengzhag/Implicit3DUnderstanding 7 Compare
Scene Segmentation SUN-RGBD ICM Mean IoU 50.60 Scene Parsing via Integrated Classification Model and... shihengcan/ICM-matcaffe 5 Compare
Monocular Depth Estimation SUN-RGBD RPSF Delta < 1.25 0.937 End-to-end Learning for Joint Depth and Image... — 3 Compare
Scene Recognition SUN-RGBD OMNIVORE (Swin-B) Accuracy (%) 67.2 Omnivore: A Single Model for Many Visual Modalities towhee-io/towhee +1 2 Compare
Object Detection SUN-RGBD val CDSSD MAP 7 How To Extract Fashion Trends From Social Media? A... trhgu/awesome-fashion-contents 1 Compare
Panoptic Segmentation SUN-RGBD EMSANet PQ 52.84 Efficient Multi-Task RGB-D Scene Analysis for Indoor Environments tui-nicr/emsanet +1 1 Compare

Papers archive 2025-07-28

papers with a benchmark row: 84 · with a code link: 63 · where Syntology ran a sample: 26 (22 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (26 of 84 with a benchmark row: 22 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 84 of 84 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 477. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
HDBFormer: Efficient RGB-D Semantic Segmentation with A Heterogeneous Dual-Branch Framework 1 1 18 Apr 2025 not harvested
DFormerv2: Geometry Self-Attention for RGBD Semantic Segmentation 1 3 7 Apr 2025 official: harvested, nothing ran · 0 ran · 3 unverified
State Space Model Meets Transformer: A New Paradigm for 3D Object Detection 1 2 18 Mar 2025 10 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 3 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence)
Diffusion-based RGB-D Semantic Segmentation with Deformable Attention Transformer 0 1 23 Sep 2024 not harvested
ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation 1 1 11 Jul 2024 not harvested
GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformer 1 3 3 Jun 2024 official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified
SPGroup3D: Superpoint Grouping Network for Indoor 3D Object Detection 1 1 21 Dec 2023 not harvested
Efficient Multimodal Semantic Segmentation via Dual-Prompt Learning 1 1 1 Dec 2023 not harvested
Uni3DETR: Unified 3D Detection Transformer 1 1 9 Oct 2023 official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified
AsymFormer: Asymmetrical Cross-Modal Representation Learning for Mobile Platform Real-Time RGB-D Semantic Segmentation 1 1 25 Sep 2023 not harvested
PanopticNDT: Efficient and Robust Panoptic Mapping 4 1 24 Sep 2023 community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 7 unverified
DFormer: Rethinking RGBD Representation Learning for Semantic Segmentation 1 4 18 Sep 2023 not harvested
V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection 1 1 8 Aug 2023 official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (2 pointer-only for licence)
Efficient Multi-Task Scene Analysis with RGB-D Transformers 2 1 8 Jun 2023 not harvested
Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrast 1 2 31 May 2023 not harvested
OctFormer: Octree-based Transformers for 3D Point Clouds 4 1 4 May 2023 official (archive's flag): 2 ran · 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (3 pointer-only for licence)
Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentation 1 1 21 Apr 2023 not harvested
DDP: Diffusion Model for Dense Visual Prediction 1 1 30 Mar 2023 not harvested
Pixel Difference Convolutional Network for RGB-D Semantic Segmentation 0 1 23 Feb 2023 not harvested
TR3D: Towards Real-Time Indoor 3D Object Detection 0 2 6 Feb 2023 not harvested
LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformers 1 1 23 Oct 2022 not harvested
DCANet: Differential Convolution Attention Network for RGB-D Semantic Segmentation 0 1 13 Oct 2022 not harvested
CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds 1 2 9 Oct 2022 not harvested
Boosting 3D Object Detection via Object-Focused Image Fusion 1 1 21 Jul 2022 not harvested
Efficient Multi-Task RGB-D Scene Analysis for Indoor Environments 2 2 10 Jul 2022 not harvested
Unified Object Detector for Different Modalities based on Vision Transformers 1 1 3 Jul 2022 not harvested
Surface Representation for Point Clouds 1 1 11 May 2022 official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (2 pointer-only for licence)
Multimodal Token Fusion for Vision Transformers 11 3 19 Apr 2022 official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (1 pointer-only for licence)
End-to-end Learning for Joint Depth and Image Reconstruction from Diffracted Rotation 0 1 14 Apr 2022 not harvested
RBGNet: Ray-based Grouping for 3D Object Detection 1 1 5 Apr 2022 not harvested
simCrossTrans: A Simple Cross-Modality Transfer Learning for Object Detection with ConvNets or Vision Transformers 3 1 20 Mar 2022 not harvested
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers 1 3 9 Mar 2022 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified
Omnivore: A Single Model for Many Visual Modalities 2 1 20 Jan 2022 official (archive's flag): 1 ran · 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 2 samples that ran constructed an object rather than computing a result (1 pointer-only for licence)
Attention-based Dual Supervised Decoder for RGBD Semantic Segmentation 0 1 5 Jan 2022 not harvested
Point Cloud Pre-Training With Natural 3D Structures 1 1 1 Jan 2022 not harvested
FCAF3D: Fully Convolutional Anchor-Free 3D Object Detection 2 1 1 Dec 2021 not harvested
An End-to-End Transformer Model for 3D Object Detection 1 1 16 Sep 2021 6 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (1 pointer-only for licence)
Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds 1 2 1 Sep 2021 official (archive's flag): 3 ran · 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 3 samples that ran constructed an object rather than computing a result (3 pointer-only for licence)
ShapeConv: Shape-aware Convolutional Layer for Indoor RGB-D Semantic Segmentation 1 1 24 Aug 2021 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified
CI-Net: Contextual Information for Joint Semantic Segmentation and Depth Estimation 0 1 29 Jul 2021 not harvested
ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection 2 2 2 Jun 2021 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified
Deep feature selection-and-fusion for RGB-D semantic segmentation 0 1 10 May 2021 not harvested
Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds 1 1 13 Apr 2021 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
Group-Free 3D Object Detection via Transformers 4 2 1 Apr 2021 official: harvested, nothing ran · 9 ran (of which 7 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 1 where Syntology's instrument failed) · 8 unverified
Holistic 3D Scene Understanding from a Single Image with Implicit Representation 1 2 11 Mar 2021 not harvested
3D Object Detection and Instance Segmentation from 3D Range and 2D Color Images 0 2 9 Feb 2021 not harvested
Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis 4 1 13 Nov 2020 not harvested
Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation 2 1 17 Jul 2020 not harvested
H3DNet: 3D Object Detection Using Hybrid Geometric Primitives 2 1 10 Jun 2020 not harvested
A Hierarchical Graph Network for 3D Object Detection on Point Clouds 0 1 1 Jun 2020 not harvested
Pattern-Structure Diffusion for Multi-Task Learning 0 1 1 Jun 2020 not harvested
When CNNs Meet Random RNNs: Towards Multi-Level Analysis for RGB-D Object and Scene Recognition 1 1 26 Apr 2020 not harvested
Spatial Information Guided Convolution for Real-Time RGBD Semantic Segmentation 1 1 9 Apr 2020 not harvested
Attention-guided Chained Context Aggregation for Semantic Segmentation 3 1 27 Feb 2020 not harvested
Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image 1 4 27 Feb 2020 official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified
ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votes 1 1 29 Jan 2020 official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified
Multi-Modal Attention-based Fusion Model for Semantic Segmentation of RGB-Depth Images 0 1 25 Dec 2019 not harvested
PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points 0 1 16 Dec 2019 not harvested
Frustum VoxNet for 3D object detection from RGB-D or Depth images 0 4 12 Oct 2019 not harvested
Index Network 6 1 11 Aug 2019 not harvested
Clouds of Oriented Gradients for 3D Detection of Objects, Surfaces, and Indoor Scene Layouts 0 1 11 Jun 2019 not harvested
Scene Parsing via Integrated Classification Model and Variance-Based Regularization 1 1 1 Jun 2019 not harvested
ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation 1 1 24 May 2019 not harvested
Deep Hough Voting for 3D Object Detection in Point Clouds 13 1 21 Apr 2019 community repositories only · 8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 3 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (10 pointer-only for licence)
Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimation 1 2 31 Oct 2018 official (archive's flag): 16 ran · 16 ran (of which 0 constructed an object rather than computing a result; 15 with no instrument failure: 0 honoured, 0 violated, 15 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (1 pointer-only for licence)
Self-Supervised Model Adaptation for Multimodal Semantic Segmentation 1 2 11 Aug 2018 not harvested
Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image 1 2 7 Aug 2018 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified
How To Extract Fashion Trends From Social Media? A Robust Object Detector With Support For Unsupervised Learning 1 1 28 Jun 2018 not harvested
RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation 8 1 4 Jun 2018 community repositories only · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified
3D Object Detection With Latent Support Surfaces 0 1 1 Jun 2018 not harvested
Context Contrasted Feature and Gated Multi-Scale Aggregation for Scene Segmentation 1 1 1 Jun 2018 not harvested
Depth-aware CNN for RGB-D Segmentation 4 1 19 Mar 2018 official (archive's flag): 4 ran · 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified
Frustum PointNets for 3D Object Detection from RGB-D Data 68 3 22 Nov 2017 community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (2 pointer-only for licence)
2D-Driven 3D Object Detection in RGB-D Images 0 2 1 Oct 2017 not harvested
3D Graph Neural Networks for RGBD Semantic Segmentation 2 1 1 Oct 2017 not harvested
RDFNet: RGB-D Multi-Level Residual Feature Fusion for Indoor Semantic Segmentation 0 1 1 Oct 2017 not harvested
Recurrent Scene Parsing with Perspective Understanding in the Loop 1 1 20 May 2017 not harvested
Three-Dimensional Object Detection and Layout Prediction Using Clouds of Oriented Gradients 0 1 1 Jun 2016 not harvested
Fully Convolutional Networks for Semantic Segmentation 37 1 20 May 2016 not harvested
Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images 0 1 7 Nov 2015 not harvested
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation 74 1 2 Nov 2015 22 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 1 honoured, 0 violated, 13 with no contract checked; 8 where Syntology's instrument failed) · 22 unverified (11 pointer-only for licence)
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs 18 1 22 Dec 2014 official: not harvested · 0 ran · 1 unverified (1 pointer-only for licence)
Learning Rich Features from RGB-D Images for Object Detection and Segmentation 1 1 22 Jul 2014 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified
Understanding Indoor Scenes Using 3D Geometric Phrases 0 1 1 Jun 2013 not harvested

Dataset loaders archive 2025-07-28

1 loader as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

License archive 2025-07-28

Unknown

Modalities archive 2025-07-28

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

  • SUN-RGBD
  • SUN-RGBD val
  • SUN RGB-D

3 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections