{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/napreg-nouns-as-proxies-regularization-for","title":"NAPReg: Nouns As Proxies Regularization for Semantically Aware Cross-Modal Embeddings","arxiv_id":null,"date":"2023-01-07","proceeding":"IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023 1","authors":["Bhavin Jawade","Deen Dayal Mohan","Naji Mohamed Ali","Srirangaraj Setlur","Venu Govindaraju"],"abstract":"Cross-modal retrieval is a fundamental vision-language task with a broad range of practical applications. Text-to-image matching is the most common form of cross-modal retrieval where, given a large database of images and a textual query, the task is to retrieve the most relevant set of images. Existing methods utilize dual encoders with an attention mechanism and a ranking loss for learning embeddings that can be used for retrieval based on cosine similarity. Despite the fact that these methods attempt to perform semantic alignment across visual regions and textual words using tailored attention mechanisms, there is no explicit supervision from the training objective to enforce such alignment. To address this, we propose NAPReg, a novel regularization formulation that projects high-level semantic entities i.e Nouns into the embedding space as shared learnable proxies. We show that using such a formulation allows the attention mechanism to learn better word-region alignment while also utilizing region information from other samples to build a more generalized latent representation for semantic concepts. Experiments on three benchmark datasets i.e. MS-COCO, Flickr30k and Flickr8k demonstrate that our method achieves state-of-the-art results in cross-modal metric learning for text-image and image-text retrieval tasks. Code: https://github.com/bhavinjawade/NAPReq","url_abs":"https://openaccess.thecvf.com/content/WACV2023/html/Jawade_NAPReg_Nouns_As_Proxies_Regularization_for_Semantically_Aware_Cross-Modal_Embeddings_WACV_2023_paper.html","url_pdf":"https://openaccess.thecvf.com/content/WACV2023/papers/Jawade_NAPReg_Nouns_As_Proxies_Regularization_for_Semantically_Aware_Cross-Modal_Embeddings_WACV_2023_paper.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"napreg-nouns-as-proxies-regularization-for","repo_url":"https://github.com/bhavinjawade/NAPReq","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"cross-modal-retrieval","task_name":"Cross-Modal Retrieval"},{"task_slug":"image-text-retrieval","task_name":"Image-text Retrieval"},{"task_slug":"metric-learning","task_name":"Metric Learning"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"text-retrieval","task_name":"Text Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/cross-modal-retrieval-on-coco-2014","task":"Cross-Modal Retrieval","dataset":"COCO 2014","model":"NAPReg","rank_in_archive_order":26,"of":36,"metrics":{"Image-to-text R@1":"59.8","Text-to-image R@1":"43.0"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-on-flickr-8k","task":"Cross-Modal Retrieval","dataset":"Flickr-8k","model":"NAPReg","rank_in_archive_order":1,"of":1,"metrics":{"Image-to-text R@1":"56.2","Text-to-image R@1":"39.2"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-on-flickr30k","task":"Cross-Modal Retrieval","dataset":"Flickr30k","model":"NAPReg","rank_in_archive_order":16,"of":27,"metrics":{"Image-to-text R@1":"79.6","Text-to-image R@1":"60.0"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-on-ms-coco-2014-1","task":"Cross-Modal Retrieval","dataset":"MS-COCO-2014","model":"NAPReg","rank_in_archive_order":1,"of":1,"metrics":{"Text-to-image R@1":"43.0"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-on-mscoco-1k","task":"Cross-Modal Retrieval","dataset":"MSCOCO-1k","model":"NAPReg","rank_in_archive_order":1,"of":2,"metrics":{"Image-to-text R@1":"81.9","Text-to-image R@1":"66.9"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"no_arxiv_id","checked_against":null,"by_repo":[{"repo":"bhavinjawade/NAPReq","kind":"named_in_paper","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"188550cbc8c6c8e06c325e137262732cc56c3435e777fb0f7a42e0e56276314b","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}