{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/sofar-language-grounded-orientation-bridges","title":"SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation","arxiv_id":"2502.13143","date":"2025-02-18","proceeding":null,"authors":["Zekun Qi","Wenyao Zhang","Yufei Ding","Runpei Dong","Xinqiang Yu","Jingwen Li","Lingyun Xu","Baoyu Li","Xialin He","Guofan Fan","Jiazhao Zhang","JiaWei He","Jiayuan Gu","Xin Jin","Kaisheng Ma","Zhizheng Zhang","He Wang","Li Yi"],"abstract":"Spatial intelligence is a critical component of embodied AI, promoting robots to understand and interact with their environments. While recent advances have enhanced the ability of VLMs to perceive object locations and positional relationships, they still lack the capability to precisely understand object orientations-a key requirement for tasks involving fine-grained manipulations. Addressing this limitation not only requires geometric reasoning but also an expressive and intuitive way to represent orientation. In this context, we propose that natural language offers a more flexible representation space than canonical frames, making it particularly suitable for instruction-following robotic systems. In this paper, we introduce the concept of semantic orientation, which defines object orientations using natural language in a reference-frame-free manner (e.g., the ''plug-in'' direction of a USB or the ''handle'' direction of a knife). To support this, we construct OrienText300K, a large-scale dataset of 3D models annotated with semantic orientations that link geometric understanding to functional semantics. By integrating semantic orientation into a VLM system, we enable robots to generate manipulation actions with both positional and orientational constraints. Extensive experiments in simulation and real world demonstrate that our approach significantly enhances robotic manipulation capabilities, e.g., 48.7% accuracy on Open6DOR and 74.9% accuracy on SIMPLER.","url_abs":"https://arxiv.org/abs/2502.13143v1","url_pdf":"https://arxiv.org/pdf/2502.13143v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sofar-language-grounded-orientation-bridges","repo_url":"https://github.com/qizekun/SoFar","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"sofar-language-grounded-orientation-bridges","repo_url":"https://github.com/zhangwenyao1/open6dor_v2_execution","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"object-rearrangement","task_name":"Object Rearrangement"},{"task_slug":"robot-manipulation","task_name":"Robot Manipulation"},{"task_slug":"robot-navigation","task_name":"Robot Navigation"},{"task_slug":"spatial-reasoning","task_name":"Spatial Reasoning"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/object-rearrangement-on-open6dor-v2","task":"Object Rearrangement","dataset":"Open6DOR V2","model":"SoFar","rank_in_archive_order":1,"of":5,"metrics":{"6-DoF":"48.7","pos-level0":"96.0","pos-level1":"81.5","rot-level0":"68.6","rot-level1":"42.2","rot-level2":"70.1"},"uses_additional_data":false},{"leaderboard":"/sota/robot-manipulation-on-simpler-env","task":"Robot Manipulation","dataset":"SimplerEnv-Google Robot","model":"SoFar","rank_in_archive_order":1,"of":9,"metrics":{"Variant Aggregation":"0.676","Variant Aggregation-Move Near":"0.740","Variant Aggregation-Open/Close Drawer":"0.297","Variant Aggregation-Pick Coke Can":"0.907","Visual Matching":"0.749","Visual Matching-Move Near":"0.917","Visual Matching-Open/Close Drawer":"0.403","Visual Matching-Pick Coke Can":"0.923"},"uses_additional_data":false},{"leaderboard":"/sota/robot-manipulation-on-simplerenv-widow-x","task":"Robot Manipulation","dataset":"SimplerEnv-Widow X","model":"SoFar","rank_in_archive_order":1,"of":7,"metrics":{"Average":"0.583","Put Carrot on Plate":"0.667","Put Eggplant in Yellow Basket":"0.375","Put Spoon on Towel":"0.583","Stack Green Block on Yellow Block":"0.708"},"uses_additional_data":false},{"leaderboard":"/sota/spatial-reasoning-on-6-dof-spatialbench","task":"Spatial Reasoning","dataset":"6-DoF SpatialBench","model":"SoFar","rank_in_archive_order":1,"of":7,"metrics":{"Orientation-abs":"31.3","Orientation-rel":"54.6","Position-abs":"33.8","Position-rel":"59.6","Total":"43.9"},"uses_additional_data":false},{"leaderboard":"/sota/spatial-reasoning-on-embspatial-bench","task":"Spatial Reasoning","dataset":"EmbSpatial-Bench","model":"SoFar","rank_in_archive_order":1,"of":5,"metrics":{"Generation":"70.88"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2502.13143","atlas_url":"https://app.syntology.ai/?focus=2502.13143","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2502.13143"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2502.13143\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2502.13143"},"call":"get_concepts_for_paper(arxiv_id=\"2502.13143\")"}],"by_repo":[{"repo":"qizekun/SoFar","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"zhangwenyao1/open6dor_v2_execution","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"306cd29158e86acde2b57aa7f903e6c6f53673b47d5a5e396df6eec3a35062e7","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}