{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/cross-modal-retrieval-with-noisy","title":"Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and Mining","arxiv_id":null,"date":"2024-03-25","proceeding":"IEEE Transactions on Image Processing 2024 3","authors":["Xinran Ma","Mouxing Yang","Yunfan Li","Peng Hu","Jiancheng Lv","Xi Peng"],"abstract":"The success of existing cross-modal retrieval (CMR) methods heavily rely on the assumption that the annotated cross-modal correspondence is faultless. In practice, however, the correspondence of some pairs would be inevitably contaminated during data collection or annotation, thus leading to the so-called Noisy Correspondence (NC) problem. To alleviate the influence of NC, we propose a novel method termed Consistency REfining And Mining (CREAM) by revealing and exploiting the difference between correspondence and consistency. Specifically, the correspondence and the consistency only be coincident for true positive and true negative pairs, while being distinct for false positive and false negative pairs. Based on the observation, CREAM employs a collaborative learning paradigm to detect and rectify the correspondence of positives, and a negative mining approach to explore and utilize the consistency. Thanks to the consistency refining and mining strategy of CREAM, the overfitting on the false positives could be prevented and the consistency rooted in the false negatives could be exploited, thus leading to a robust CMR method. Extensive experiments verify the effectiveness of our method on three image-text benchmarks including Flickr30K, MS-COCO, and Conceptual Captions. Furthermore, we adopt our method into the graph matching task and the results demonstrate the robustness of our method against fine-grained NC problem. The code is available on https://github.com/XLearning-SCU/2024-TIP-CREAM .","url_abs":"https://ieeexplore.ieee.org/document/10477322","url_pdf":"https://ieeexplore.ieee.org/document/10477322","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cross-modal-retrieval-with-noisy","repo_url":"https://github.com/XLearning-SCU/2024-TIP-CREAM","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"cross-modal-retrieval","task_name":"Cross-Modal Retrieval"},{"task_slug":"cross-modal-retrieval-with-noisy","task_name":"Cross-modal retrieval with noisy correspondence"},{"task_slug":"graph-matching","task_name":"Graph Matching"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[{"slug":"cc152k","name":"CC152K","full_name":"Conceptual Captions 152K"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/cross-modal-retrieval-with-noisy-1","task":"Cross-modal retrieval with noisy correspondence","dataset":"CC152K","model":"CREAM","rank_in_archive_order":7,"of":15,"metrics":{"Image-to-text R@1":"40.3","Image-to-text R@10":"77.1","Image-to-text R@5":"68.5","R-Sum":"372.6","Text-to-image R@1":"40.2","Text-to-image R@10":"78.3","Text-to-image R@5":"68.2"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-with-noisy-3","task":"Cross-modal retrieval with noisy correspondence","dataset":"COCO-Noisy","model":"CREAM","rank_in_archive_order":12,"of":17,"metrics":{"Image-to-text R@1":"78.9","Image-to-text R@10":"98.6","Image-to-text R@5":"96.3","R-Sum":"523","Text-to-image R@1":"63.3","Text-to-image R@10":"95.8","Text-to-image R@5":"90.1"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-with-noisy-2","task":"Cross-modal retrieval with noisy correspondence","dataset":"Flickr30K-Noisy","model":"CREAM","rank_in_archive_order":11,"of":16,"metrics":{"Image-to-text R@1":"77.4","Image-to-text R@10":"97.3","Image-to-text R@5":"95.0","R-Sum":"502.3","Text-to-image R@1":"58.7","Text-to-image R@10":"89.8","Text-to-image R@5":"84.1"},"uses_additional_data":false},{"leaderboard":"/sota/graph-matching-on-pascal-voc","task":"Graph Matching","dataset":"PASCAL VOC","model":"CREAM","rank_in_archive_order":17,"of":31,"metrics":{"matching accuracy":"0.814"},"uses_additional_data":false},{"leaderboard":"/sota/graph-matching-on-spair-71k","task":"Graph Matching","dataset":"SPair-71k","model":"CREAM","rank_in_archive_order":1,"of":8,"metrics":{"matching accuracy":"0.851"},"uses_additional_data":false},{"leaderboard":"/sota/graph-matching-on-willow-object-class","task":"Graph Matching","dataset":"Willow Object Class","model":"CREAM","rank_in_archive_order":4,"of":23,"metrics":{"matching accuracy":"0.988"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"no_arxiv_id","checked_against":null,"by_repo":[{"repo":"XLearning-SCU/2024-TIP-CREAM","kind":"named_in_paper","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"f4e572fa987e1d462983e909cb41d208230ef3d8a0d31e4e2e70535f80880638","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}