{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/using-syntax-to-ground-referring-expressions","title":"Using Syntax to Ground Referring Expressions in Natural Images","arxiv_id":"1805.10547","date":"2018-05-26","proceeding":null,"authors":["Volkan Cirik","Taylor Berg-Kirkpatrick","Louis-Philippe Morency"],"abstract":"We introduce GroundNet, a neural network for referring expression recognition\n-- the task of localizing (or grounding) in an image the object referred to by\na natural language expression. Our approach to this task is the first to rely\non a syntactic analysis of the input referring expression in order to inform\nthe structure of the computation graph. Given a parse tree for an input\nexpression, we explicitly map the syntactic constituents and relationships\npresent in the tree to a composed graph of neural modules that defines our\narchitecture for performing localization. This syntax-based approach aids\nlocalization of \\textit{both} the target object and auxiliary supporting\nobjects mentioned in the expression. As a result, GroundNet is more\ninterpretable than previous methods: we can (1) determine which phrase of the\nreferring expression points to which object in the image and (2) track how the\nlocalization of the target object is determined by the network. We study this\nproperty empirically by introducing a new set of annotations on the GoogleRef\ndataset to evaluate localization of supporting objects. Our experiments show\nthat GroundNet achieves state-of-the-art accuracy in identifying supporting\nobjects, while maintaining comparable performance in the localization of target\nobjects.","url_abs":"http://arxiv.org/abs/1805.10547v1","url_pdf":"http://arxiv.org/pdf/1805.10547v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"using-syntax-to-ground-referring-expressions","repo_url":"https://github.com/volkancirik/groundnet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"object","task_name":"Object"},{"task_slug":"referring-expression","task_name":"Referring Expression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1805.10547","atlas_url":"https://app.syntology.ai/?focus=1805.10547","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1805.10547"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1805.10547\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1805.10547"},"call":"get_concepts_for_paper(arxiv_id=\"1805.10547\")"}],"by_repo":[{"repo":"volkancirik/groundnet","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"72630940a5a288f56a52ab157a3494bf3d3a5f39e89a3a3f5f8c84f0e40d3b9d","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}