{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/task/image-to-text","name":"Image to text","slug":"image-to-text","description_markdown":null,"categories":[],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":246,"papers_with_code":103,"benchmarks":0,"benchmark_tables_in_archive":0,"benchmark_tables_shown":0,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":3,"subtasks":0,"parent_tasks":0},"benchmarks":[],"datasets":[{"url":"/dataset/cii-bench","name":"CII-Bench","full_name":"Chinese Image Implication understanding Benchmark","num_papers_in_archive":2},{"url":"/dataset/psocr","name":"PsOCR","full_name":"Pashto OCR Dataset","num_papers_in_archive":1},{"url":"/dataset/urdu-text-scene-images","name":"Urdu Text Scene Images","full_name":"","num_papers_in_archive":0}],"subtasks":[],"parent_tasks":[],"papers":{"order":"repositories listed in the archive (most first), then date (newest first), then slug; the archive holds no stars","population":"every paper the archive tags with this task (distinct papers; spam titles not listed). 'of' counts the papers listed; until 2026-09-25 it counted only the papers with a code link, and that number is of_with_code","page":1,"pages_in_order":3,"rows_per_page":100,"shown":100,"of":246,"of_with_code":103,"tagged_in_all":246,"counts":{"archive_papers_tagged":246,"with_a_code_link":103,"where_syntology_ran_a_sample":33,"not_listed_spam_title":0,"listed":246,"listed_where_code_ran":33,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":29,"every_run_a_failure_of_syntologys_instrument":4,"listed_with_a_run_with_no_instrument_failure":29,"listed_every_run_a_failure_of_syntologys_instrument":4,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"items":[{"url":"/paper/blip-2-bootstrapping-language-image-pre","slug":"blip-2-bootstrapping-language-image-pre","title":"BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models","date":"2023-01-30","arxiv_id":"2301.12597","repositories_listed":17,"syntology":{"n":8,"n_ran":4,"n_constructed":4,"n_ran_checked":4,"n_instrument":0,"n_unverified":4,"n_honours":0,"n_violates":0,"n_no_contract":4,"n_pointer_only":1,"phrase":"4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified; every one of the 4 samples that ran constructed an object rather than computing a result","sample_list":"/paper/blip-2-bootstrapping-language-image-pre#ran","syntology_url":"https://syntology.ai/paper/2301.12597","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.12597"}},"official":{"repos":["salesforce/lavis"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"url":"/paper/pix2struct-screenshot-parsing-as-pretraining","slug":"pix2struct-screenshot-parsing-as-pretraining","title":"Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding","date":"2022-10-07","arxiv_id":"2210.03347","repositories_listed":4,"syntology":{"n":5,"n_ran":5,"n_constructed":0,"n_ran_checked":5,"n_instrument":0,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":0,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/pix2struct-screenshot-parsing-as-pretraining#ran","syntology_url":"https://syntology.ai/paper/2210.03347","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.03347"}},"official":{"repos":["google-research/pix2struct"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/distilled-dual-encoder-model-for-vision","slug":"distilled-dual-encoder-model-for-vision","title":"Distilled Dual-Encoder Model for Vision-Language Understanding","date":"2021-12-16","arxiv_id":"2112.08723","repositories_listed":4,"syntology":null},{"url":"/paper/effective-use-of-word-order-for-text-1","slug":"effective-use-of-word-order-for-text-1","title":"Effective Use of Word Order for Text Categorization with Convolutional Neural Networks","date":"2014-12-01","arxiv_id":"1412.1058","repositories_listed":4,"syntology":null},{"url":"/paper/cephalo-multi-modal-vision-language-models","slug":"cephalo-multi-modal-vision-language-models","title":"Cephalo: Multi-Modal Vision-Language Models for Bio-Inspired Materials Analysis and Design","date":"2024-05-29","arxiv_id":"2405.19076","repositories_listed":3,"syntology":null},{"url":"/paper/evaluating-text-to-visual-generation-with","slug":"evaluating-text-to-visual-generation-with","title":"Evaluating Text-to-Visual Generation with Image-to-Text Generation","date":"2024-04-01","arxiv_id":"2404.01291","repositories_listed":3,"syntology":null},{"url":"/paper/one-transformer-fits-all-distributions-in","slug":"one-transformer-fits-all-distributions-in","title":"One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale","date":"2023-03-12","arxiv_id":"2303.06555","repositories_listed":3,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":1,"n_instrument":1,"n_unverified":0,"n_honours":1,"n_violates":0,"n_no_contract":0,"n_pointer_only":2,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/one-transformer-fits-all-distributions-in#ran","syntology_url":"https://syntology.ai/paper/2303.06555","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.06555"}},"official":{"repos":["thu-ml/unidiffuser"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official","unlocated"]}}},{"url":"/paper/versatile-diffusion-text-images-and","slug":"versatile-diffusion-text-images-and","title":"Versatile Diffusion: Text, Images and Variations All in One Diffusion Model","date":"2022-11-15","arxiv_id":"2211.08332","repositories_listed":3,"syntology":null},{"url":"/paper/improving-factual-completeness-and","slug":"improving-factual-completeness-and","title":"Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation","date":"2020-10-20","arxiv_id":"2010.10042","repositories_listed":3,"syntology":{"n":3,"n_ran":3,"n_constructed":0,"n_ran_checked":0,"n_instrument":3,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/improving-factual-completeness-and#ran","syntology_url":"https://syntology.ai/paper/2010.10042","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.10042"}},"official":{"repos":["ysmiura/ifcc"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed","official"]}}},{"url":"/paper/lmm4lmm-benchmarking-and-evaluating-large","slug":"lmm4lmm-benchmarking-and-evaluating-large","title":"LMM4LMM: Benchmarking and Evaluating Large-multimodal Image Generation with LMMs","date":"2025-04-11","arxiv_id":"2504.08358","repositories_listed":2,"syntology":null},{"url":"/paper/magma-a-foundation-model-for-multimodal-ai","slug":"magma-a-foundation-model-for-multimodal-ai","title":"Magma: A Foundation Model for Multimodal AI Agents","date":"2025-02-18","arxiv_id":"2502.13130","repositories_listed":2,"syntology":{"n":16,"n_ran":12,"n_constructed":0,"n_ran_checked":10,"n_instrument":2,"n_unverified":4,"n_honours":0,"n_violates":0,"n_no_contract":10,"n_pointer_only":0,"phrase":"12 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified","sample_list":"/paper/magma-a-foundation-model-for-multimodal-ai#ran","syntology_url":"https://syntology.ai/paper/2502.13130","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.13130"}},"official":{"repos":["microsoft/Magma"],"state":"official (archive's flag): 12 ran","n_ran":12,"n_constructed":0,"n_ran_no_instrument_failure":10,"n_unverified":4,"ran_from_kinds":["official"]}}},{"url":"/paper/survey-on-abstractive-text-summarization","slug":"survey-on-abstractive-text-summarization","title":"Survey on Abstractive Text Summarization: Dataset, Models, and Metrics","date":"2024-12-22","arxiv_id":"2412.17165","repositories_listed":2,"syntology":null},{"url":"/paper/semantic-editing-increment-benefits-zero-shot","slug":"semantic-editing-increment-benefits-zero-shot","title":"Semantic Editing Increment Benefits Zero-Shot Composed Image Retrieval","date":"2024-10-28","arxiv_id":null,"repositories_listed":2,"syntology":null},{"url":"/paper/ldre-llm-based-divergent-reasoning-and","slug":"ldre-llm-based-divergent-reasoning-and","title":"LDRE: LLM-based Divergent Reasoning and Ensemble for Zero-Shot Composed Image Retrieval","date":"2024-07-11","arxiv_id":null,"repositories_listed":2,"syntology":null},{"url":"/paper/comat-aligning-text-to-image-diffusion-model","slug":"comat-aligning-text-to-image-diffusion-model","title":"CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching","date":"2024-04-04","arxiv_id":"2404.03653","repositories_listed":2,"syntology":{"n":9,"n_ran":5,"n_constructed":0,"n_ran_checked":5,"n_instrument":0,"n_unverified":4,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":9,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified","sample_list":"/paper/comat-aligning-text-to-image-diffusion-model#ran","syntology_url":"https://syntology.ai/paper/2404.03653","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.03653"}},"official":{"repos":["caraj7/comat"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":4,"ran_from_kinds":["official"]}}},{"url":"/paper/large-multilingual-models-pivot-zero-shot","slug":"large-multilingual-models-pivot-zero-shot","title":"Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages","date":"2023-08-23","arxiv_id":"2308.12038","repositories_listed":2,"syntology":{"n":7,"n_ran":6,"n_constructed":0,"n_ran_checked":6,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":1,"n_no_contract":5,"n_pointer_only":7,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 1 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/large-multilingual-models-pivot-zero-shot#ran","syntology_url":"https://syntology.ai/paper/2308.12038","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.12038"}},"official":{"repos":["openbmb/viscpm"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":6,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/multimodal-dataset-distillation-for-image","slug":"multimodal-dataset-distillation-for-image","title":"Vision-Language Dataset Distillation","date":"2023-08-15","arxiv_id":"2308.07545","repositories_listed":2,"syntology":{"n":2,"n_ran":2,"n_constructed":0,"n_ran_checked":2,"n_instrument":0,"n_unverified":0,"n_honours":1,"n_violates":1,"n_no_contract":0,"n_pointer_only":2,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/multimodal-dataset-distillation-for-image#ran","syntology_url":"https://syntology.ai/paper/2308.07545","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.07545"}},"official":{"repos":["princetonvisualai/multimodal_dataset_distillation"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/generative-pretraining-in-multimodality","slug":"generative-pretraining-in-multimodality","title":"Emu: Generative Pretraining in Multimodality","date":"2023-07-11","arxiv_id":"2307.05222","repositories_listed":2,"syntology":{"n":2,"n_ran":0,"n_constructed":0,"n_ran_checked":0,"n_instrument":0,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"0 ran · 2 unverified","sample_list":"/paper/generative-pretraining-in-multimodality#ran","syntology_url":"https://syntology.ai/paper/2307.05222","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.05222"}},"official":{"repos":["baaivision/emu"],"state":"official: harvested for another paper","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":[]}}},{"url":"/paper/zeronlg-aligning-and-autoencoding-domains-for","slug":"zeronlg-aligning-and-autoencoding-domains-for","title":"ZeroNLG: Aligning and Autoencoding Domains for Zero-Shot Multimodal and Multilingual Natural Language Generation","date":"2023-03-11","arxiv_id":"2303.06458","repositories_listed":2,"syntology":null},{"url":"/paper/safe-latent-diffusion-mitigating","slug":"safe-latent-diffusion-mitigating","title":"Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models","date":"2022-11-09","arxiv_id":"2211.05105","repositories_listed":2,"syntology":null},{"url":"/paper/linearly-mapping-from-image-to-text-space","slug":"linearly-mapping-from-image-to-text-space","title":"Linearly Mapping from Image to Text Space","date":"2022-09-30","arxiv_id":"2209.15162","repositories_listed":2,"syntology":{"n":14,"n_ran":13,"n_constructed":0,"n_ran_checked":8,"n_instrument":5,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":8,"n_pointer_only":0,"phrase":"13 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/linearly-mapping-from-image-to-text-space#ran","syntology_url":"https://syntology.ai/paper/2209.15162","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.15162"}},"official":{"repos":["jmerullo/limber"],"state":"official (archive's flag): 13 ran","n_ran":13,"n_constructed":0,"n_ran_no_instrument_failure":8,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/ernie-vilg-unified-generative-pre-training","slug":"ernie-vilg-unified-generative-pre-training","title":"ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation","date":"2021-12-31","arxiv_id":"2112.15283","repositories_listed":2,"syntology":null},{"url":"/paper/translation-equivariant-image-quantizer-for","slug":"translation-equivariant-image-quantizer-for","title":"Exploration into Translation-Equivariant Image Quantization","date":"2021-12-01","arxiv_id":"2112.00384","repositories_listed":2,"syntology":{"n":14,"n_ran":13,"n_constructed":0,"n_ran_checked":3,"n_instrument":10,"n_unverified":1,"n_honours":2,"n_violates":0,"n_no_contract":1,"n_pointer_only":10,"phrase":"13 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 10 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/translation-equivariant-image-quantizer-for#ran","syntology_url":"https://syntology.ai/paper/2112.00384","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.00384"}},"official":{"repos":["wcshin-git/te-vqgan"],"state":"official (archive's flag): 13 ran","n_ran":13,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/mirrorgan-learning-text-to-image-generation","slug":"mirrorgan-learning-text-to-image-generation","title":"MirrorGAN: Learning Text-to-image Generation by Redescription","date":"2019-03-14","arxiv_id":"1903.05854","repositories_listed":2,"syntology":null},{"url":"/paper/text-to-image-to-text-translation-using-cycle","slug":"text-to-image-to-text-translation-using-cycle","title":"Text-to-Image-to-Text Translation using Cycle Consistent Adversarial Networks","date":"2018-08-14","arxiv_id":"1808.04538","repositories_listed":2,"syntology":null},{"url":"/paper/efficient-medical-vision-language-alignment","slug":"efficient-medical-vision-language-alignment","title":"Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Models","date":"2025-06-10","arxiv_id":"2506.08990","repositories_listed":1,"syntology":null},{"url":"/paper/unimoco-unified-modality-completion-for","slug":"unimoco-unified-modality-completion-for","title":"UniMoCo: Unified Modality Completion for Robust Multi-Modal Embeddings","date":"2025-05-17","arxiv_id":"2505.11815","repositories_listed":1,"syntology":null},{"url":"/paper/lrsclip-a-vision-language-foundation-model","slug":"lrsclip-a-vision-language-foundation-model","title":"LRSCLIP: A Vision-Language Foundation Model for Aligning Remote Sensing Image with Longer Text","date":"2025-03-25","arxiv_id":"2503.19311","repositories_listed":1,"syntology":null},{"url":"/paper/real-world-validation-of-a-multimodal-llm","slug":"real-world-validation-of-a-multimodal-llm","title":"Real-world validation of a multimodal LLM-powered pipeline for High-Accuracy Clinical Trial Patient Matching leveraging EHR data","date":"2025-03-19","arxiv_id":"2503.15374","repositories_listed":1,"syntology":null},{"url":"/paper/flowtok-flowing-seamlessly-across-text-and","slug":"flowtok-flowing-seamlessly-across-text-and","title":"FlowTok: Flowing Seamlessly Across Text and Image Tokens","date":"2025-03-13","arxiv_id":"2503.10772","repositories_listed":1,"syntology":{"n":9,"n_ran":6,"n_constructed":6,"n_ran_checked":6,"n_instrument":0,"n_unverified":3,"n_honours":0,"n_violates":0,"n_no_contract":6,"n_pointer_only":0,"phrase":"6 ran (of which 6 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified; every one of the 6 samples that ran constructed an object rather than computing a result","sample_list":"/paper/flowtok-flowing-seamlessly-across-text-and#ran","syntology_url":"https://syntology.ai/paper/2503.10772","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.10772"}},"official":{"repos":["bytedance/1d-tokenizer"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":6,"n_ran_no_instrument_failure":6,"n_unverified":3,"ran_from_kinds":["official"]}}},{"url":"/paper/doracycle-domain-oriented-adaptation-of","slug":"doracycle-domain-oriented-adaptation-of","title":"DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cycles","date":"2025-03-05","arxiv_id":"2503.03651","repositories_listed":1,"syntology":{"n":13,"n_ran":12,"n_constructed":0,"n_ran_checked":5,"n_instrument":7,"n_unverified":1,"n_honours":1,"n_violates":0,"n_no_contract":4,"n_pointer_only":8,"phrase":"12 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 7 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/doracycle-domain-oriented-adaptation-of#ran","syntology_url":"https://syntology.ai/paper/2503.03651","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.03651"}},"official":{"repos":["showlab/doracycle"],"state":"official (archive's flag): 12 ran","n_ran":12,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/reading-the-unreadable-creating-a-dataset-of","slug":"reading-the-unreadable-creating-a-dataset-of","title":"Reading the unreadable: Creating a dataset of 19th century English newspapers using image-to-text language models","date":"2025-02-18","arxiv_id":"2502.14901","repositories_listed":1,"syntology":null},{"url":"/paper/unicms-a-unified-consistency-model-for","slug":"unicms-a-unified-consistency-model-for","title":"UniCMs: A Unified Consistency Model For Efficient Multimodal Generation and Understanding","date":"2025-02-08","arxiv_id":"2502.05415","repositories_listed":1,"syntology":null},{"url":"/paper/generalizing-from-simple-to-hard-visual","slug":"generalizing-from-simple-to-hard-visual","title":"Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs?","date":"2025-01-05","arxiv_id":"2501.02669","repositories_listed":1,"syntology":{"n":3,"n_ran":3,"n_constructed":0,"n_ran_checked":1,"n_instrument":2,"n_unverified":0,"n_honours":1,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/generalizing-from-simple-to-hard-visual#ran","syntology_url":"https://syntology.ai/paper/2501.02669","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.02669"}},"official":{"repos":["princeton-pli/vlm_s2h"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/multi-llm-collaborative-caption-generation-in","slug":"multi-llm-collaborative-caption-generation-in","title":"Multi-LLM Collaborative Caption Generation in Scientific Documents","date":"2025-01-05","arxiv_id":"2501.02552","repositories_listed":1,"syntology":null},{"url":"/paper/flame-frozen-large-language-models-enable","slug":"flame-frozen-large-language-models-enable","title":"FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training","date":"2024-11-18","arxiv_id":"2411.11927","repositories_listed":1,"syntology":null},{"url":"/paper/revealing-and-reducing-gender-biases-in","slug":"revealing-and-reducing-gender-biases-in","title":"Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs)","date":"2024-10-25","arxiv_id":"2410.19314","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/revealing-and-reducing-gender-biases-in#ran","syntology_url":"https://syntology.ai/paper/2410.19314","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.19314"}},"official":{"repos":["explainableml/vla-gender-bias"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/see-or-guess-counterfactually-regularized","slug":"see-or-guess-counterfactually-regularized","title":"See or Guess: Counterfactually Regularized Image Captioning","date":"2024-08-29","arxiv_id":"2408.16809","repositories_listed":1,"syntology":null},{"url":"/paper/unifashion-a-unified-vision-language-model","slug":"unifashion-a-unified-vision-language-model","title":"UniFashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generation","date":"2024-08-21","arxiv_id":"2408.11305","repositories_listed":1,"syntology":{"n":11,"n_ran":8,"n_constructed":0,"n_ran_checked":4,"n_instrument":4,"n_unverified":3,"n_honours":1,"n_violates":1,"n_no_contract":2,"n_pointer_only":11,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 1 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/unifashion-a-unified-vision-language-model#ran","syntology_url":"https://syntology.ai/paper/2408.11305","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.11305"}},"official":{"repos":["xiangyu-mm/unifashion"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":3,"ran_from_kinds":["official"]}}},{"url":"/paper/in-defense-of-lazy-visual-grounding-for-open","slug":"in-defense-of-lazy-visual-grounding-for-open","title":"In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentation","date":"2024-08-09","arxiv_id":"2408.04961","repositories_listed":1,"syntology":null},{"url":"/paper/gabinsight-exploring-gender-activity-binding","slug":"gabinsight-exploring-gender-activity-binding","title":"GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models","date":"2024-07-30","arxiv_id":"2407.21001","repositories_listed":1,"syntology":null},{"url":"/paper/contrasting-deepfakes-diffusion-via","slug":"contrasting-deepfakes-diffusion-via","title":"Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities","date":"2024-07-29","arxiv_id":"2407.20337","repositories_listed":1,"syntology":{"n":8,"n_ran":7,"n_constructed":0,"n_ran_checked":3,"n_instrument":4,"n_unverified":1,"n_honours":0,"n_violates":1,"n_no_contract":2,"n_pointer_only":0,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/contrasting-deepfakes-diffusion-via#ran","syntology_url":"https://syntology.ai/paper/2407.20337","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.20337"}},"official":{"repos":["aimagelab/code"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/towards-a-text-based-quantitative-and","slug":"towards-a-text-based-quantitative-and","title":"Towards a text-based quantitative and explainable histopathology image analysis","date":"2024-07-10","arxiv_id":"2407.07360","repositories_listed":1,"syntology":null},{"url":"/paper/a-data-driven-guided-decoding-mechanism-for","slug":"a-data-driven-guided-decoding-mechanism-for","title":"A Data-Driven Guided Decoding Mechanism for Diagnostic Captioning","date":"2024-06-20","arxiv_id":"2406.14164","repositories_listed":1,"syntology":null},{"url":"/paper/bivlc-extending-vision-language","slug":"bivlc-extending-vision-language","title":"BiVLC: Extending Vision-Language Compositionality Evaluation with Text-to-Image Retrieval","date":"2024-06-14","arxiv_id":"2406.09952","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/bivlc-extending-vision-language#ran","syntology_url":"https://syntology.ai/paper/2406.09952","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.09952"}},"official":{"repos":["imirandam/bivlc"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/cmc-bench-towards-a-new-paradigm-of-visual","slug":"cmc-bench-towards-a-new-paradigm-of-visual","title":"CMC-Bench: Towards a New Paradigm of Visual Signal Compression","date":"2024-06-13","arxiv_id":"2406.09356","repositories_listed":1,"syntology":null},{"url":"/paper/benchmarking-vision-language-contrastive","slug":"benchmarking-vision-language-contrastive","title":"Benchmarking Vision-Language Contrastive Methods for Medical Representation Learning","date":"2024-06-11","arxiv_id":"2406.07450","repositories_listed":1,"syntology":null},{"url":"/paper/multi-modality-regional-alignment-network-for","slug":"multi-modality-regional-alignment-network-for","title":"Multi-modality Regional Alignment Network for Covid X-Ray Survival Prediction and Report Generation","date":"2024-05-23","arxiv_id":"2405.14113","repositories_listed":1,"syntology":null},{"url":"/paper/language-oriented-semantic-latent","slug":"language-oriented-semantic-latent","title":"Language-Oriented Semantic Latent Representation for Image Transmission","date":"2024-05-16","arxiv_id":"2405.09976","repositories_listed":1,"syntology":null},{"url":"/paper/libra-building-decoupled-vision-system-on","slug":"libra-building-decoupled-vision-system-on","title":"Libra: Building Decoupled Vision System on Large Language Models","date":"2024-05-16","arxiv_id":"2405.10140","repositories_listed":1,"syntology":{"n":10,"n_ran":8,"n_constructed":0,"n_ran_checked":2,"n_instrument":6,"n_unverified":2,"n_honours":0,"n_violates":1,"n_no_contract":1,"n_pointer_only":2,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 1 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/libra-building-decoupled-vision-system-on#ran","syntology_url":"https://syntology.ai/paper/2405.10140","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.10140"}},"official":{"repos":["yifanxu74/libra"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/ladic-are-diffusion-models-really-inferior-to","slug":"ladic-are-diffusion-models-really-inferior-to","title":"LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation?","date":"2024-04-16","arxiv_id":"2404.10763","repositories_listed":1,"syntology":null},{"url":"/paper/from-pixels-to-graphs-open-vocabulary-scene","slug":"from-pixels-to-graphs-open-vocabulary-scene","title":"From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models","date":"2024-04-01","arxiv_id":"2404.00906","repositories_listed":1,"syntology":{"n":9,"n_ran":7,"n_constructed":0,"n_ran_checked":4,"n_instrument":3,"n_unverified":2,"n_honours":1,"n_violates":0,"n_no_contract":3,"n_pointer_only":1,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/from-pixels-to-graphs-open-vocabulary-scene#ran","syntology_url":"https://syntology.ai/paper/2404.00906","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.00906"}},"official":{"repos":["shtuplus/pix2grp_cvpr2024"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/objectcompose-evaluating-resilience-of-vision","slug":"objectcompose-evaluating-resilience-of-vision","title":"ObjectCompose: Evaluating Resilience of Vision-Based Models on Object-to-Background Compositional Changes","date":"2024-03-07","arxiv_id":"2403.04701","repositories_listed":1,"syntology":null},{"url":"/paper/probing-multimodal-large-language-models-for","slug":"probing-multimodal-large-language-models-for","title":"Probing Multimodal Large Language Models for Global and Local Semantic Representations","date":"2024-02-27","arxiv_id":"2402.17304","repositories_listed":1,"syntology":null},{"url":"/paper/can-mllms-perform-text-to-image-in-context","slug":"can-mllms-perform-text-to-image-in-context","title":"Can MLLMs Perform Text-to-Image In-Context Learning?","date":"2024-02-02","arxiv_id":"2402.01293","repositories_listed":1,"syntology":null},{"url":"/paper/benchmarking-large-multimodal-models-against","slug":"benchmarking-large-multimodal-models-against","title":"Benchmarking Large Multimodal Models against Common Corruptions","date":"2024-01-22","arxiv_id":"2401.11943","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":0,"n_instrument":1,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":0,"n_pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/benchmarking-large-multimodal-models-against#ran","syntology_url":"https://syntology.ai/paper/2401.11943","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.11943"}},"official":{"repos":["sail-sg/mmcbench"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/cocot-contrastive-chain-of-thought-prompting","slug":"cocot-contrastive-chain-of-thought-prompting","title":"CoCoT: Contrastive Chain-of-Thought Prompting for Large Multimodal Models with Multiple Image Inputs","date":"2024-01-05","arxiv_id":"2401.02582","repositories_listed":1,"syntology":null},{"url":"/paper/improving-image-restoration-through-removing","slug":"improving-image-restoration-through-removing","title":"Improving Image Restoration through Removing Degradations in Textual Representations","date":"2023-12-28","arxiv_id":"2312.17334","repositories_listed":1,"syntology":null},{"url":"/paper/pragmatic-radiology-report-generation","slug":"pragmatic-radiology-report-generation","title":"Pragmatic Radiology Report Generation","date":"2023-11-28","arxiv_id":"2311.17154","repositories_listed":1,"syntology":{"n":10,"n_ran":7,"n_constructed":0,"n_ran_checked":5,"n_instrument":2,"n_unverified":3,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":10,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/pragmatic-radiology-report-generation#ran","syntology_url":"https://syntology.ai/paper/2311.17154","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.17154"}},"official":{"repos":["chicagohai/llm_radiology"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":3,"ran_from_kinds":["official"]}}},{"url":"/paper/removing-nsfw-concepts-from-vision-and","slug":"removing-nsfw-concepts-from-vision-and","title":"Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models","date":"2023-11-27","arxiv_id":"2311.16254","repositories_listed":1,"syntology":{"n":7,"n_ran":4,"n_constructed":0,"n_ran_checked":3,"n_instrument":1,"n_unverified":3,"n_honours":2,"n_violates":0,"n_no_contract":1,"n_pointer_only":7,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/removing-nsfw-concepts-from-vision-and#ran","syntology_url":"https://syntology.ai/paper/2311.16254","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.16254"}},"official":{"repos":["aimagelab/safe-clip"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":3,"ran_from_kinds":["official"]}}},{"url":"/paper/when-urban-region-profiling-meets-large","slug":"when-urban-region-profiling-meets-large","title":"UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web","date":"2023-10-22","arxiv_id":"2310.18340","repositories_listed":1,"syntology":null},{"url":"/paper/symmetrical-linguistic-feature-distillation","slug":"symmetrical-linguistic-feature-distillation","title":"Symmetrical Linguistic Feature Distillation with CLIP for Scene Text Recognition","date":"2023-10-08","arxiv_id":"2310.04999","repositories_listed":1,"syntology":null},{"url":"/paper/fine-grained-late-interaction-multi-modal-1","slug":"fine-grained-late-interaction-multi-modal-1","title":"Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answering","date":"2023-09-29","arxiv_id":"2309.17133","repositories_listed":1,"syntology":{"n":8,"n_ran":8,"n_constructed":0,"n_ran_checked":2,"n_instrument":6,"n_unverified":0,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":8,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/fine-grained-late-interaction-multi-modal-1#ran","syntology_url":"https://syntology.ai/paper/2309.17133","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.17133"}},"official":{"repos":["linweizhedragon/retrieval-augmented-visual-question-answering"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/align-before-search-aligning-ads-image-to","slug":"align-before-search-aligning-ads-image-to","title":"Align before Search: Aligning Ads Image to Text for Accurate Cross-Modal Sponsored Search","date":"2023-09-28","arxiv_id":"2309.16141","repositories_listed":1,"syntology":null},{"url":"/paper/clip-based-synergistic-knowledge-transfer-for","slug":"clip-based-synergistic-knowledge-transfer-for","title":"CLIP-based Synergistic Knowledge Transfer for Text-based Person Retrieval","date":"2023-09-18","arxiv_id":"2309.09496","repositories_listed":1,"syntology":null},{"url":"/paper/scaling-autoregressive-multi-modal-models","slug":"scaling-autoregressive-multi-modal-models","title":"Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning","date":"2023-09-05","arxiv_id":"2309.02591","repositories_listed":1,"syntology":null},{"url":"/paper/multimodal-foundation-models-for","slug":"multimodal-foundation-models-for","title":"Multimodal Foundation Models For Echocardiogram Interpretation","date":"2023-08-29","arxiv_id":"2308.15670","repositories_listed":1,"syntology":null},{"url":"/paper/beyond-one-to-one-rethinking-the-referring","slug":"beyond-one-to-one-rethinking-the-referring","title":"Beyond One-to-One: Rethinking the Referring Image Segmentation","date":"2023-08-26","arxiv_id":"2308.13853","repositories_listed":1,"syntology":{"n":13,"n_ran":9,"n_constructed":0,"n_ran_checked":3,"n_instrument":6,"n_unverified":4,"n_honours":1,"n_violates":0,"n_no_contract":2,"n_pointer_only":13,"phrase":"9 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 6 where Syntology's instrument failed) · 4 unverified","sample_list":"/paper/beyond-one-to-one-rethinking-the-referring#ran","syntology_url":"https://syntology.ai/paper/2308.13853","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.13853"}},"official":{"repos":["toggle1995/ris-dmmi"],"state":"official (archive's flag): 9 ran","n_ran":9,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":4,"ran_from_kinds":["official"]}}},{"url":"/paper/unifying-two-stream-encoders-with","slug":"unifying-two-stream-encoders-with","title":"Unifying Two-Stream Encoders with Transformers for Cross-Modal Retrieval","date":"2023-08-08","arxiv_id":"2308.04343","repositories_listed":1,"syntology":null},{"url":"/paper/transferable-decoding-with-visual-entities","slug":"transferable-decoding-with-visual-entities","title":"Transferable Decoding with Visual Entities for Zero-Shot Image Captioning","date":"2023-07-31","arxiv_id":"2307.16525","repositories_listed":1,"syntology":{"n":7,"n_ran":6,"n_constructed":3,"n_ran_checked":5,"n_instrument":1,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":5,"n_pointer_only":7,"phrase":"6 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/transferable-decoding-with-visual-entities#ran","syntology_url":"https://syntology.ai/paper/2307.16525","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.16525"}},"official":{"repos":["feielysia/viecap"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":3,"n_ran_no_instrument_failure":5,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/prior-prototype-representation-joint-learning","slug":"prior-prototype-representation-joint-learning","title":"PRIOR: Prototype Representation Joint Learning from Medical Images and Reports","date":"2023-07-24","arxiv_id":"2307.12577","repositories_listed":1,"syntology":{"n":3,"n_ran":2,"n_constructed":0,"n_ran_checked":2,"n_instrument":0,"n_unverified":1,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/prior-prototype-representation-joint-learning#ran","syntology_url":"https://syntology.ai/paper/2307.12577","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.12577"}},"official":{"repos":["qtacierp/prior"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/planting-a-seed-of-vision-in-large-language","slug":"planting-a-seed-of-vision-in-large-language","title":"Planting a SEED of Vision in Large Language Model","date":"2023-07-16","arxiv_id":"2307.08041","repositories_listed":1,"syntology":null},{"url":"/paper/bootstrapping-vision-language-learning-with-1","slug":"bootstrapping-vision-language-learning-with-1","title":"Bootstrapping Vision-Language Learning with Decoupled Language Pre-training","date":"2023-07-13","arxiv_id":"2307.07063","repositories_listed":1,"syntology":{"n":9,"n_ran":7,"n_constructed":0,"n_ran_checked":4,"n_instrument":3,"n_unverified":2,"n_honours":1,"n_violates":0,"n_no_contract":3,"n_pointer_only":1,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/bootstrapping-vision-language-learning-with-1#ran","syntology_url":"https://syntology.ai/paper/2307.07063","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.07063"}},"official":{"repos":["yiren-jian/blitext"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/multiqg-ti-towards-question-generation-from","slug":"multiqg-ti-towards-question-generation-from","title":"MultiQG-TI: Towards Question Generation from Multi-modal Sources","date":"2023-07-07","arxiv_id":"2307.04643","repositories_listed":1,"syntology":null},{"url":"/paper/zero-shot-nuclei-detection-via-visual","slug":"zero-shot-nuclei-detection-via-visual","title":"Zero-shot Nuclei Detection via Visual-Language Pre-trained Models","date":"2023-06-30","arxiv_id":"2306.17659","repositories_listed":1,"syntology":null},{"url":"/paper/brain-captioning-decoding-human-brain","slug":"brain-captioning-decoding-human-brain","title":"Brain Captioning: Decoding human brain activity into images and text","date":"2023-05-19","arxiv_id":"2305.11560","repositories_listed":1,"syntology":null},{"url":"/paper/what-you-see-is-what-you-read-improving-text-1","slug":"what-you-see-is-what-you-read-improving-text-1","title":"What You See is What You Read? Improving Text-Image Alignment Evaluation","date":"2023-05-17","arxiv_id":"2305.10400","repositories_listed":1,"syntology":null},{"url":"/paper/multimodal-procedural-planning-via-dual-text","slug":"multimodal-procedural-planning-via-dual-text","title":"Multimodal Procedural Planning via Dual Text-Image Prompting","date":"2023-05-02","arxiv_id":"2305.01795","repositories_listed":1,"syntology":null},{"url":"/paper/rococo-robust-benchmark-ms-coco-to-stress","slug":"rococo-robust-benchmark-ms-coco-to-stress","title":"RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching Models","date":"2023-04-21","arxiv_id":"2304.10727","repositories_listed":1,"syntology":null},{"url":"/paper/magvlt-masked-generative-vision-and-language","slug":"magvlt-masked-generative-vision-and-language","title":"MAGVLT: Masked Generative Vision-and-Language Transformer","date":"2023-03-21","arxiv_id":"2303.12208","repositories_listed":1,"syntology":{"n":10,"n_ran":7,"n_constructed":5,"n_ran_checked":7,"n_instrument":0,"n_unverified":3,"n_honours":0,"n_violates":0,"n_no_contract":7,"n_pointer_only":0,"phrase":"7 ran (of which 5 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified","sample_list":"/paper/magvlt-masked-generative-vision-and-language#ran","syntology_url":"https://syntology.ai/paper/2303.12208","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.12208"}},"official":{"repos":["kakaobrain/magvlt"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":5,"n_ran_no_instrument_failure":7,"n_unverified":3,"ran_from_kinds":["official"]}}},{"url":"/paper/towards-unifying-medical-vision-and-language","slug":"towards-unifying-medical-vision-and-language","title":"Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts","date":"2023-02-17","arxiv_id":"2302.08958","repositories_listed":1,"syntology":null},{"url":"/paper/generative-diffusion-models-on-graphs-methods","slug":"generative-diffusion-models-on-graphs-methods","title":"Generative Diffusion Models on Graphs: Methods and Applications","date":"2023-02-06","arxiv_id":"2302.02591","repositories_listed":1,"syntology":null},{"url":"/paper/language-quantized-autoencoders-towards-1","slug":"language-quantized-autoencoders-towards-1","title":"Language Quantized AutoEncoders: Towards Unsupervised Text-Image Alignment","date":"2023-02-02","arxiv_id":"2302.00902","repositories_listed":1,"syntology":null},{"url":"/paper/adaptively-clustering-neighbor-elements-for","slug":"adaptively-clustering-neighbor-elements-for","title":"Adaptively Clustering Neighbor Elements for Image-Text Generation","date":"2023-01-05","arxiv_id":"2301.01955","repositories_listed":1,"syntology":null},{"url":"/paper/matcha-enhancing-visual-language-pretraining","slug":"matcha-enhancing-visual-language-pretraining","title":"MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering","date":"2022-12-19","arxiv_id":"2212.09662","repositories_listed":1,"syntology":null},{"url":"/paper/improving-the-factual-correctness-of","slug":"improving-the-factual-correctness-of","title":"Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards","date":"2022-10-21","arxiv_id":"2210.12186","repositories_listed":1,"syntology":null},{"url":"/paper/visual-spatial-description-controlled-spatial","slug":"visual-spatial-description-controlled-spatial","title":"Visual Spatial Description: Controlled Spatial-Oriented Image-to-Text Generation","date":"2022-10-20","arxiv_id":"2210.11109","repositories_listed":1,"syntology":null},{"url":"/paper/feta-towards-specializing-foundation-models","slug":"feta-towards-specializing-foundation-models","title":"FETA: Towards Specializing Foundation Models for Expert Task Applications","date":"2022-09-08","arxiv_id":"2209.03648","repositories_listed":1,"syntology":null},{"url":"/paper/what-is-where-by-looking-weakly-supervised","slug":"what-is-where-by-looking-weakly-supervised","title":"What is Where by Looking: Weakly-Supervised Open-World Phrase-Grounding without Text Inputs","date":"2022-06-19","arxiv_id":"2206.09358","repositories_listed":1,"syntology":null},{"url":"/paper/prefix-language-models-are-unified-modal","slug":"prefix-language-models-are-unified-modal","title":"Write and Paint: Generative Vision-Language Models are Unified Modal Learners","date":"2022-06-15","arxiv_id":"2206.07699","repositories_listed":1,"syntology":{"n":9,"n_ran":8,"n_constructed":0,"n_ran_checked":6,"n_instrument":2,"n_unverified":1,"n_honours":1,"n_violates":0,"n_no_contract":5,"n_pointer_only":9,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified","sample_list":"/paper/prefix-language-models-are-unified-modal#ran","syntology_url":"https://syntology.ai/paper/2206.07699","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.07699"}},"official":{"repos":["shizhediao/davinci"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":6,"n_unverified":1,"ran_from_kinds":["official"]}}},{"url":"/paper/rethinking-the-openness-of-clip","slug":"rethinking-the-openness-of-clip","title":"Delving into the Openness of CLIP","date":"2022-06-04","arxiv_id":"2206.01986","repositories_listed":1,"syntology":{"n":10,"n_ran":5,"n_constructed":0,"n_ran_checked":2,"n_instrument":3,"n_unverified":5,"n_honours":0,"n_violates":0,"n_no_contract":2,"n_pointer_only":3,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified","sample_list":"/paper/rethinking-the-openness-of-clip#ran","syntology_url":"https://syntology.ai/paper/2206.01986","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.01986"}},"official":{"repos":["lancopku/clip-openness"],"state":"official (archive's flag): 5 ran","n_ran":5,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":5,"ran_from_kinds":["official"]}}},{"url":"/paper/git-a-generative-image-to-text-transformer","slug":"git-a-generative-image-to-text-transformer","title":"GIT: A Generative Image-to-text Transformer for Vision and Language","date":"2022-05-27","arxiv_id":"2205.14100","repositories_listed":1,"syntology":{"n":21,"n_ran":14,"n_constructed":0,"n_ran_checked":14,"n_instrument":0,"n_unverified":7,"n_honours":0,"n_violates":0,"n_no_contract":14,"n_pointer_only":0,"phrase":"14 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 0 where Syntology's instrument failed) · 7 unverified","sample_list":"/paper/git-a-generative-image-to-text-transformer#ran","syntology_url":"https://syntology.ai/paper/2205.14100","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.14100"}},"official":{"repos":["microsoft/GenerativeImage2Text"],"state":"official (archive's flag): 14 ran","n_ran":14,"n_constructed":0,"n_ran_no_instrument_failure":14,"n_unverified":7,"ran_from_kinds":["official"]}}},{"url":"/paper/characterizing-and-understanding-the-behavior","slug":"characterizing-and-understanding-the-behavior","title":"Characterizing and Understanding the Behavior of Quantized Models for Reliable Deployment","date":"2022-04-08","arxiv_id":"2204.04220","repositories_listed":1,"syntology":null},{"url":"/paper/self-supervised-image-to-text-and-text-to","slug":"self-supervised-image-to-text-and-text-to","title":"Self-Supervised Image-to-Text and Text-to-Image Synthesis","date":"2021-12-09","arxiv_id":"2112.04928","repositories_listed":1,"syntology":null},{"url":"/paper/zero-shot-image-to-text-generation-for-visual","slug":"zero-shot-image-to-text-generation-for-visual","title":"ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic","date":"2021-11-29","arxiv_id":"2111.14447","repositories_listed":1,"syntology":{"n":1,"n_ran":1,"n_constructed":0,"n_ran_checked":1,"n_instrument":0,"n_unverified":0,"n_honours":1,"n_violates":0,"n_no_contract":0,"n_pointer_only":1,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","sample_list":"/paper/zero-shot-image-to-text-generation-for-visual#ran","syntology_url":"https://syntology.ai/paper/2111.14447","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.14447"}},"official":{"repos":["yoadtew/zero-shot-image-to-text"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":0,"ran_from_kinds":["official"]}}},{"url":"/paper/l-verse-bidirectional-generation-between","slug":"l-verse-bidirectional-generation-between","title":"L-Verse: Bidirectional Generation Between Image and Text","date":"2021-11-22","arxiv_id":"2111.11133","repositories_listed":1,"syntology":null},{"url":"/paper/unifying-multimodal-transformer-for-bi","slug":"unifying-multimodal-transformer-for-bi","title":"Unifying Multimodal Transformer for Bi-directional Image and Text Generation","date":"2021-10-19","arxiv_id":"2110.09753","repositories_listed":1,"syntology":{"n":8,"n_ran":6,"n_constructed":0,"n_ran_checked":6,"n_instrument":0,"n_unverified":2,"n_honours":0,"n_violates":0,"n_no_contract":6,"n_pointer_only":0,"phrase":"6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified","sample_list":"/paper/unifying-multimodal-transformer-for-bi#ran","syntology_url":"https://syntology.ai/paper/2110.09753","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.09753"}},"official":{"repos":["researchmm/generate-it"],"state":"official (archive's flag): 6 ran","n_ran":6,"n_constructed":0,"n_ran_no_instrument_failure":6,"n_unverified":2,"ran_from_kinds":["official"]}}},{"url":"/paper/concadia-tackling-image-accessibility-with","slug":"concadia-tackling-image-accessibility-with","title":"Concadia: Towards Image-Based Text Generation with a Purpose","date":"2021-04-16","arxiv_id":"2104.08376","repositories_listed":1,"syntology":null},{"url":"/paper/progressive-transformer-based-generation-of","slug":"progressive-transformer-based-generation-of","title":"Progressive Transformer-Based Generation of Radiology Reports","date":"2021-02-19","arxiv_id":"2102.09777","repositories_listed":1,"syntology":null},{"url":"/paper/aligning-multilingual-word-embeddings-for","slug":"aligning-multilingual-word-embeddings-for","title":"Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task","date":"2019-10-08","arxiv_id":"1910.03291","repositories_listed":1,"syntology":null}],"syntology_records":34,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"key_notes":{"n_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"since_archive_judged":{"papers":0,"papers_placed":0,"judged_by":{"model":"a model","models_as_recorded":[],"from":"lb_rows_run_verified.jsonl: each record's `model` field (the extractor's call that wrote decisions[])"},"note":"a model's judgement from the paper's own tables that they report on this leaderboard, not a result: no value from a paper that was not placed is printed or carried here; papers counts each paper once over these leaderboards","backfill":"8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read)","backfill_complete":false,"leaderboards":[]},"papers_where_code_ran":33,"papers_where_code_ran_first_page":"/task/image-to-text/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not run it on this task or check it against the task's benchmarks.","pages":[{"order":"archive","page":2,"of":3,"url":"/task/image-to-text/papers/2","json":"/task/image-to-text/papers/2.json","rows":100},{"order":"archive","page":3,"of":3,"url":"/task/image-to-text/papers/3","json":"/task/image-to-text/papers/3.json","rows":46},{"order":"ran","page":1,"of":1,"url":"/task/image-to-text/papers/ran/1","json":"/task/image-to-text/papers/ran/1.json","rows":33}],"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-10-05T00:36:11+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"row_route":"each syntology object on a benchmark's first row and on a paper carries sample_list (this site's paper page, #ran), syntology_url (syntology.ai's page for the paper, only when that route probed live) and mcp (the agent call that lists the paper's harvested samples; not a verdict on the paper)","developers":"https://syntology.ai/developers"},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"},"record_sha256":"8268dc3eab206b8175bc21c0b59893c595755b7c9fd1879002882a96306d9ccc","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}