{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/vlocnet-deep-multitask-learning-for-semantic","title":"VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry","arxiv_id":"1804.08366","date":"2018-04-23","proceeding":null,"authors":["Noha Radwan","Abhinav Valada","Wolfram Burgard"],"abstract":"Semantic understanding and localization are fundamental enablers of robot\nautonomy that have for the most part been tackled as disjoint problems. While\ndeep learning has enabled recent breakthroughs across a wide spectrum of scene\nunderstanding tasks, its applicability to state estimation tasks has been\nlimited due to the direct formulation that renders it incapable of encoding\nscene-specific constrains. In this work, we propose the VLocNet++ architecture\nthat employs a multitask learning approach to exploit the inter-task\nrelationship between learning semantics, regressing 6-DoF global pose and\nodometry, for the mutual benefit of each of these tasks. Our network overcomes\nthe aforementioned limitation by simultaneously embedding geometric and\nsemantic knowledge of the world into the pose regression network. We propose a\nnovel adaptive weighted fusion layer to aggregate motion-specific temporal\ninformation and to fuse semantic features into the localization stream based on\nregion activations. Furthermore, we propose a self-supervised warping technique\nthat uses the relative motion to warp intermediate network representations in\nthe segmentation stream for learning consistent semantics. Finally, we\nintroduce a first-of-a-kind urban outdoor localization dataset with pixel-level\nsemantic labels and multiple loops for training deep networks. Extensive\nexperiments on the challenging Microsoft 7-Scenes benchmark and our DeepLoc\ndataset demonstrate that our approach exceeds the state-of-the-art\noutperforming local feature-based methods while simultaneously performing\nmultiple tasks and exhibiting substantial robustness in challenging scenarios.","url_abs":"http://arxiv.org/abs/1804.08366v6","url_pdf":"http://arxiv.org/pdf/1804.08366v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"outdoor-localization","task_name":"Outdoor Localization"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"state-estimation","task_name":"State Estimation"},{"task_slug":"visual-localization","task_name":"Visual Localization"}],"methods":[],"datasets_introduced":[{"slug":"deeploc","name":"DeepLoc","full_name":"DeepLoc"},{"slug":"deeploccross","name":"DeepLocCross","full_name":"DeepLocCross"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.08366","atlas_url":"https://app.syntology.ai/?focus=1804.08366","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1804.08366"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1804.08366\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1804.08366"},"call":"get_concepts_for_paper(arxiv_id=\"1804.08366\")"}],"by_repo":[],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":"the archive holds no code link for this paper","arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"d9434838244f1c19f44a926e7b9f464f90d4ff5a8a41b42ce9c16005ed1b4bd9","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}