{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/vote2cap-detr-decoupling-localization-and","title":"Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning","arxiv_id":"2309.02999","date":"2023-09-06","proceeding":null,"authors":["Sijin Chen","Hongyuan Zhu","Mingsheng Li","Xin Chen","Peng Guo","Yinjie Lei","Gang Yu","Taihao Li","Tao Chen"],"abstract":"3D dense captioning requires a model to translate its understanding of an input 3D scene into several captions associated with different object regions. Existing methods adopt a sophisticated \"detect-then-describe\" pipeline, which builds explicit relation modules upon a 3D detector with numerous hand-crafted components. While these methods have achieved initial success, the cascade pipeline tends to accumulate errors because of duplicated and inaccurate box estimations and messy 3D scenes. In this paper, we first propose Vote2Cap-DETR, a simple-yet-effective transformer framework that decouples the decoding process of caption generation and object localization through parallel decoding. Moreover, we argue that object localization and description generation require different levels of scene understanding, which could be challenging for a shared set of queries to capture. To this end, we propose an advanced version, Vote2Cap-DETR++, which decouples the queries into localization and caption queries to capture task-specific features. Additionally, we introduce the iterative spatial refinement strategy to vote queries for faster convergence and better localization performance. We also insert additional spatial information to the caption head for more accurate descriptions. Without bells and whistles, extensive experiments on two commonly used datasets, ScanRefer and Nr3D, demonstrate Vote2Cap-DETR and Vote2Cap-DETR++ surpass conventional \"detect-then-describe\" methods by a large margin. Codes will be made available at https://github.com/ch3cook-fdu/Vote2Cap-DETR.","url_abs":"https://arxiv.org/abs/2309.02999v1","url_pdf":"https://arxiv.org/pdf/2309.02999v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vote2cap-detr-decoupling-localization-and","repo_url":"https://github.com/ch3cook-fdu/vote2cap-detr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-dense-captioning","task_name":"3D dense captioning"},{"task_slug":"caption-generation","task_name":"Caption Generation"},{"task_slug":"dense-captioning","task_name":"Dense Captioning"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-localization","task_name":"Object Localization"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-dense-captioning-on-nr3d","task":"3D dense captioning","dataset":"Nr3D","model":"Vote2Cap-DETR++","rank_in_archive_order":3,"of":10,"metrics":{"BLEU-4":"27.70","CIDEr":"47.08","METEOR":"25.44","ROUGE-L":"55.22"},"uses_additional_data":false},{"leaderboard":"/sota/3d-dense-captioning-on-scanrefer-dataset","task":"3D dense captioning","dataset":"ScanRefer Dataset","model":"Vote2Cap-DETR++","rank_in_archive_order":4,"of":12,"metrics":{"BLEU-4":"41.37","CIDEr":"76.36","METEOR":"28.70","ROUGE-L":"60.00"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2309.02999","atlas_url":"https://app.syntology.ai/?focus=2309.02999","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2309.02999"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2309.02999\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2309.02999"},"call":"get_concepts_for_paper(arxiv_id=\"2309.02999\")"}],"by_repo":[{"repo":"ch3cook-fdu/vote2cap-detr","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2301.02508"],"harvested_for_calls":[{"arxiv_id":"2301.02508","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2301.02508"},"call":"get_harvested_code_for_paper(arxiv_id=\"2301.02508\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"8a9f25ae8e68f2ebc192d5b3cb1cf44a8506a105edbdd0e94dfa82622849d0bf","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}