{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/ponderv2-pave-the-way-for-3d-foundataion","title":"PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm","arxiv_id":"2310.08586","date":"2023-10-12","proceeding":null,"authors":["Haoyi Zhu","Honghui Yang","Xiaoyang Wu","Di Huang","Sha Zhang","Xianglong He","Hengshuang Zhao","Chunhua Shen","Yu Qiao","Tong He","Wanli Ouyang"],"abstract":"In contrast to numerous NLP and 2D vision foundational models, learning a 3D foundational model poses considerably greater challenges. This is primarily due to the inherent data variability and diversity of downstream tasks. In this paper, we introduce a novel universal 3D pre-training framework designed to facilitate the acquisition of efficient 3D representation, thereby establishing a pathway to 3D foundational models. Considering that informative 3D features should encode rich geometry and appearance cues that can be utilized to render realistic images, we propose to learn 3D representations by differentiable neural rendering. We train a 3D backbone with a devised volumetric neural renderer by comparing the rendered with the real images. Notably, our approach seamlessly integrates the learned 3D encoder into various downstream tasks. These tasks encompass not only high-level challenges such as 3D detection and segmentation but also low-level objectives like 3D reconstruction and image synthesis, spanning both indoor and outdoor scenarios. Besides, we also illustrate the capability of pre-training a 2D backbone using the proposed methodology, surpassing conventional pre-training methods by a large margin. For the first time, PonderV2 achieves state-of-the-art performance on 11 indoor and outdoor benchmarks, implying its effectiveness. Code and models are available at https://github.com/OpenGVLab/PonderV2.","url_abs":"https://arxiv.org/abs/2310.08586v3","url_pdf":"https://arxiv.org/pdf/2310.08586v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ponderv2-pave-the-way-for-3d-foundataion","repo_url":"https://github.com/OpenGVLab/PonderV2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"3d-semantic-segmentation","task_name":"3D Semantic Segmentation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"lidar-semantic-segmentation","task_name":"LIDAR Semantic Segmentation"},{"task_slug":"neural-rendering","task_name":"Neural Rendering"},{"task_slug":"point-cloud-pre-training","task_name":"Point Cloud Pre-training"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-semantic-segmentation-on-scannet200","task":"3D Semantic Segmentation","dataset":"ScanNet200","model":"PonderV2 + SparseUNet","rank_in_archive_order":11,"of":16,"metrics":{"test mIoU":"34.6","val mIoU":"32.3"},"uses_additional_data":true},{"leaderboard":"/sota/semantic-segmentation-on-s3dis","task":"Semantic Segmentation","dataset":"S3DIS","model":"PonderV2 + SparseUNet","rank_in_archive_order":3,"of":54,"metrics":{"Mean IoU":"79.9","mAcc":"86.5","oAcc":"92.5"},"uses_additional_data":true},{"leaderboard":"/sota/semantic-segmentation-on-s3dis-area5","task":"Semantic Segmentation","dataset":"S3DIS Area5","model":"PonderV2 + SparseUNet","rank_in_archive_order":14,"of":61,"metrics":{"mAcc":"79.0","mIoU":"73.2","oAcc":"92.2"},"uses_additional_data":true},{"leaderboard":"/sota/semantic-segmentation-on-scannet","task":"Semantic Segmentation","dataset":"ScanNet","model":"PonderV2 + SparseUNet","rank_in_archive_order":9,"of":45,"metrics":{"test mIoU":"78.5","val mIoU":"77.0"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.08586","atlas_url":"https://app.syntology.ai/?focus=2310.08586","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2310.08586"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2310.08586\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2310.08586"},"call":"get_concepts_for_paper(arxiv_id=\"2310.08586\")"}],"by_repo":[{"repo":"OpenGVLab/PonderV2","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"9ea66947e0f9c253c7b974333a3ab78f5cd6380b2c7e32962fe09899ed3c000b","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}