{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/visual-semantics-allow-for-textual-reasoning-1","title":"Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition","arxiv_id":"2112.12916","date":"2021-12-24","proceeding":"AAAI 2022 2021 12","authors":["Yue He","Chen Chen","Jing Zhang","Juhua Liu","Fengxiang He","Chaoyue Wang","Bo Du"],"abstract":"Existing Scene Text Recognition (STR) methods typically use a language model to optimize the joint probability of the 1D character sequence predicted by a visual recognition (VR) model, which ignore the 2D spatial context of visual semantics within and between character instances, making them not generalize well to arbitrary shape scene text. To address this issue, we make the first attempt to perform textual reasoning based on visual semantics in this paper. Technically, given the character segmentation maps predicted by a VR model, we construct a subgraph for each instance, where nodes represent the pixels in it and edges are added between nodes based on their spatial similarity. Then, these subgraphs are sequentially connected by their root nodes and merged into a complete graph. Based on this graph, we devise a graph convolutional network for textual reasoning (GTR) by supervising it with a cross-entropy loss. GTR can be easily plugged in representative STR models to improve their performance owing to better textual reasoning. Specifically, we construct our model, namely S-GTR, by paralleling GTR to the language model in a segmentation-based STR baseline, which can effectively exploit the visual-linguistic complementarity via mutual learning. S-GTR sets new state-of-the-art on six challenging STR benchmarks and generalizes well to multi-linguistic datasets. Code is available at https://github.com/adeline-cs/GTR.","url_abs":"https://arxiv.org/abs/2112.12916v1","url_pdf":"https://arxiv.org/pdf/2112.12916v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"visual-semantics-allow-for-textual-reasoning-1","repo_url":"https://github.com/adeline-cs/GTR","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"scene-text-recognition","task_name":"Scene Text Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/scene-text-recognition-on-cute80","task":"Scene Text Recognition","dataset":"CUTE80","model":"S-GTR","rank_in_archive_order":12,"of":18,"metrics":{"Accuracy":"94.7"},"uses_additional_data":true},{"leaderboard":"/sota/scene-text-recognition-on-icdar2013","task":"Scene Text Recognition","dataset":"ICDAR2013","model":"S-GTR","rank_in_archive_order":11,"of":38,"metrics":{"Accuracy":"97.8"},"uses_additional_data":true},{"leaderboard":"/sota/scene-text-recognition-on-icdar2015","task":"Scene Text Recognition","dataset":"ICDAR2015","model":"S-GTR","rank_in_archive_order":10,"of":27,"metrics":{"Accuracy":"87.3"},"uses_additional_data":true},{"leaderboard":"/sota/scene-text-recognition-on-iiit5k","task":"Scene Text Recognition","dataset":"IIIT5k","model":"S-GTR","rank_in_archive_order":11,"of":17,"metrics":{"Accuracy":"97.5"},"uses_additional_data":true},{"leaderboard":"/sota/scene-text-recognition-on-svt","task":"Scene Text Recognition","dataset":"SVT","model":"S-GTR","rank_in_archive_order":13,"of":37,"metrics":{"Accuracy":"95.8"},"uses_additional_data":true},{"leaderboard":"/sota/scene-text-recognition-on-svtp","task":"Scene Text Recognition","dataset":"SVTP","model":"S-GTR","rank_in_archive_order":12,"of":17,"metrics":{"Accuracy":"90.6"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2112.12916","atlas_url":"https://app.syntology.ai/?focus=2112.12916","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2112.12916"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2112.12916\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2112.12916"},"call":"get_concepts_for_paper(arxiv_id=\"2112.12916\")"}],"by_repo":[{"repo":"adeline-cs/GTR","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["aaai_19971"],"harvested_for_calls":[]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"444318fef3639f448ecb3f2be77c78ed35ee5a6cc4226586fb1b3bb97938331c","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}