{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/blip","slug":"blip","name":"BLIP","full_name":"BLIP: Bootstrapping Language-Image Pre-training","full_name_withheld":false,"description_markdown":"Vision-Language Pre-training (VLP) has advanced the performance for many vision-language tasks. However, most existing pre-trained models only excel in either understanding-based tasks or generation-based tasks. Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision. In this paper, we propose BLIP, a new VLP framework which transfers flexibly to both vision-language understanding and generation tasks. BLIP effectively utilizes the noisy web data by bootstrapping the captions, where a captioner generates synthetic captions and a filter removes the noisy ones. We achieve state-of-the-art results on a wide range of vision-language tasks, such as image-text retrieval (+2.7% in average recall@1), image captioning (+2.8% in CIDEr), and VQA (+1.6% in VQA score). BLIP also demonstrates strong generalization ability when directly transferred to video-language tasks in a zero-shot manner. Code, models, and datasets are released at https://github.com/salesforce/BLIP.","description_state":"present","introduced_year":null,"introduced_by":{"title":"BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation","paper":"/paper/blip-bootstrapping-language-image-pre","first_author":"Junnan Li","n_authors":4,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/blip-bootstrapping-language-image-pre"},"source":{"url":"https://arxiv.org/abs/2201.12086v2","title":"BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Vision and Language Pre-Trained Models","url":"/methods/category/vision-and-language-pre-trained-models","pwc_aliases":[]}],"n_papers_tagged":93,"archive_num_papers":93,"papers_newest_first":[{"paper":"/paper/text-visual-semantic-constrained-ai-generated","slug":"text-visual-semantic-constrained-ai-generated","title":"Text-Visual Semantic Constrained AI-Generated Image Quality Assessment","date":"2025-07-14","arxiv_id":"2507.10432","n_code_links":1,"syntology":null},{"paper":"/paper/vistext-mosquito-a-multimodal-dataset-and","slug":"vistext-mosquito-a-multimodal-dataset-and","title":"VisText-Mosquito: A Multimodal Dataset and Benchmark for AI-Based Mosquito Breeding Site Detection and Reasoning","date":"2025-06-17","arxiv_id":"2506.14629","n_code_links":1,"syntology":null},{"paper":null,"slug":"fusing-cross-modal-and-uni-modal","title":"Fusing Cross-modal and Uni-modal Representations: A Kronecker Product Approach","date":"2025-06-10","arxiv_id":"2506.08645","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-narrative-review-on-large-ai-models-in-lung","title":"A Narrative Review on Large AI Models in Lung Cancer Screening, Diagnosis, and Treatment Planning","date":"2025-06-08","arxiv_id":"2506.07236","n_code_links":0,"syntology":null},{"paper":null,"slug":"when-vlms-meet-image-classification-test-sets","title":"When VLMs Meet Image Classification: Test Sets Renovation via Missing Label Identification","date":"2025-05-22","arxiv_id":"2505.16149","n_code_links":0,"syntology":null},{"paper":null,"slug":"medblip-fine-tuning-blip-for-medical-image","title":"MedBLIP: Fine-tuning BLIP for Medical Image Captioning","date":"2025-05-20","arxiv_id":"2505.14726","n_code_links":0,"syntology":null},{"paper":"/paper/from-complexity-to-clarity-transforming-chest","slug":"from-complexity-to-clarity-transforming-chest","title":"From Complexity to Clarity: Transforming Chest X-ray Reports with Chained Prompting (Student Abstract)","date":"2025-04-11","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/from-complexity-to-clarity-transforming-chest-1","slug":"from-complexity-to-clarity-transforming-chest-1","title":"From Complexity to Clarity: Transforming Chest X-ray Reports with Chained Prompting (Student Abstract) Authors","date":"2025-04-11","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"slug":"learning-sparse-disentangled-representations","title":"Learning Sparse Disentangled Representations for Multimodal Exclusion Retrieval","date":"2025-04-04","arxiv_id":"2504.03184","n_code_links":0,"syntology":null},{"paper":null,"slug":"omr-diffusion-optimizing-multi-round-enhanced","title":"OMR-Diffusion:Optimizing Multi-Round Enhanced Training in Diffusion Models for Improved Intent Understanding","date":"2025-03-22","arxiv_id":"2503.17660","n_code_links":0,"syntology":null},{"paper":null,"slug":"tdri-two-phase-dialogue-refinement-and-co","title":"TDRI: Two-Phase Dialogue Refinement and Co-Adaptation for Interactive Image Generation","date":"2025-03-22","arxiv_id":"2503.17669","n_code_links":0,"syntology":null},{"paper":null,"slug":"are-large-language-models-good-data","title":"Are Large Language Models Good Data Preprocessors?","date":"2025-02-24","arxiv_id":"2502.16790","n_code_links":0,"syntology":null},{"paper":null,"slug":"nanovlms-how-small-can-we-go-and-still-make","title":"NanoVLMs: How small can we go and still make coherent Vision Language Models?","date":"2025-02-11","arxiv_id":"2502.07838","n_code_links":0,"syntology":null},{"paper":null,"slug":"an-evaluation-framework-for-product-images","title":"An Evaluation Framework for Product Images Background Inpainting based on Human Feedback and Product Consistency","date":"2024-12-23","arxiv_id":"2412.17504","n_code_links":0,"syntology":null},{"paper":null,"slug":"barking-up-the-syntactic-tree-enhancing-vlm","title":"Barking Up The Syntactic Tree: Enhancing VLM Training with Syntactic Losses","date":"2024-12-11","arxiv_id":"2412.08110","n_code_links":0,"syntology":null},{"paper":"/paper/attacks-on-multimodal-models","slug":"attacks-on-multimodal-models","title":"Attacks on multimodal models","date":"2024-12-02","arxiv_id":"2412.01725","n_code_links":1,"syntology":null},{"paper":null,"slug":"understanding-the-world-s-museums-through","title":"Understanding the World's Museums through Vision-Language Reasoning","date":"2024-12-02","arxiv_id":"2412.01370","n_code_links":0,"syntology":null},{"paper":"/paper/nearest-neighbor-normalization-improves","slug":"nearest-neighbor-normalization-improves","title":"Nearest Neighbor Normalization Improves Multimodal Retrieval","date":"2024-10-31","arxiv_id":"2410.24114","n_code_links":1,"syntology":null},{"paper":null,"slug":"technical-report-for-soccernet-2023-dense","title":"Technical Report for Soccernet 2023 -- Dense Video Captioning","date":"2024-10-31","arxiv_id":"2411.00882","n_code_links":0,"syntology":null},{"paper":null,"slug":"efficienteqa-an-efficient-approach-for-open","title":"EfficientEQA: An Efficient Approach for Open Vocabulary Embodied Question Answering","date":"2024-10-26","arxiv_id":"2410.20263","n_code_links":0,"syntology":null},{"paper":null,"slug":"backdoor-in-seconds-unlocking-vulnerabilities","title":"Backdoor in Seconds: Unlocking Vulnerabilities in Large Pre-trained Models via Model Editing","date":"2024-10-23","arxiv_id":"2410.18267","n_code_links":0,"syntology":null},{"paper":null,"slug":"towards-zero-shot-camera-trap-image","title":"Towards Zero-Shot Camera Trap Image Categorization","date":"2024-10-16","arxiv_id":"2410.12769","n_code_links":0,"syntology":null},{"paper":null,"slug":"anyattack-towards-large-scale-self-supervised","title":"AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models","date":"2024-10-07","arxiv_id":"2410.05346","n_code_links":0,"syntology":null},{"paper":null,"slug":"vlm2vec-training-vision-language-models-for","title":"VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks","date":"2024-10-07","arxiv_id":"2410.05160","n_code_links":0,"syntology":null},{"paper":"/paper/multiclimate-multimodal-stance-detection-on","slug":"multiclimate-multimodal-stance-detection-on","title":"MultiClimate: Multimodal Stance Detection on Climate Change Videos","date":"2024-09-26","arxiv_id":"2409.18346","n_code_links":1,"syntology":{"ran":1,"of":1,"n_ran_checked":0,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["werywjw/multiclimate"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"nein-telling-what-you-don-t-want","title":"NeIn: Telling What You Don't Want","date":"2024-09-09","arxiv_id":"2409.06481","n_code_links":0,"syntology":null},{"paper":"/paper/evaluation-and-comparison-of-visual-language","slug":"evaluation-and-comparison-of-visual-language","title":"Evaluation and Comparison of Visual Language Models for Transportation Engineering Problems","date":"2024-09-03","arxiv_id":"2409.02278","n_code_links":1,"syntology":null},{"paper":null,"slug":"medical-report-generation-is-a-multi-label","title":"Medical Report Generation Is A Multi-label Classification Problem","date":"2024-08-30","arxiv_id":"2409.00250","n_code_links":0,"syntology":null},{"paper":null,"slug":"response-wide-shut-surprising-observations-in","title":"Response Wide Shut: Surprising Observations in Basic Vision Language Model Capabilities","date":"2024-08-13","arxiv_id":"2408.06721","n_code_links":0,"syntology":null},{"paper":null,"slug":"2407-21758","title":"MOSAIC: Multimodal Multistakeholder-aware Visual Art Recommendation","date":"2024-07-31","arxiv_id":"2407.21758","n_code_links":0,"syntology":null},{"paper":"/paper/prior-knowledge-integration-via-llm-encoding","slug":"prior-knowledge-integration-via-llm-encoding","title":"Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrieval","date":"2024-07-21","arxiv_id":"2407.15051","n_code_links":1,"syntology":{"ran":8,"of":11,"n_ran_checked":4,"n_instrument":4,"unverified":3,"pointer_only":9,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 2 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified","official":{"repos":["fletcherjiang/llmepet"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":4,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"cosmoclip-generalizing-large-vision-language","title":"CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical Imaging","date":"2024-07-10","arxiv_id":"2407.07315","n_code_links":0,"syntology":null},{"paper":"/paper/what-to-do-if-language-models-disagree-black","slug":"what-to-do-if-language-models-disagree-black","title":"Black-box Model Ensembling for Textual and Visual Question Answering via Information Fusion","date":"2024-07-04","arxiv_id":"2407.12841","n_code_links":1,"syntology":null},{"paper":"/paper/visual-robustness-benchmark-for-visual","slug":"visual-robustness-benchmark-for-visual","title":"Visual Robustness Benchmark for Visual Question Answering (VQA)","date":"2024-07-03","arxiv_id":"2407.03386","n_code_links":1,"syntology":null},{"paper":"/paper/the-surprising-effectiveness-of-multimodal","slug":"the-surprising-effectiveness-of-multimodal","title":"The Surprising Effectiveness of Multimodal Large Language Models for Video Moment Retrieval","date":"2024-06-26","arxiv_id":"2406.18113","n_code_links":1,"syntology":{"ran":0,"of":1,"n_ran_checked":0,"n_instrument":0,"unverified":1,"pointer_only":1,"phrase":"0 ran · 1 unverified","official":{"repos":["sudo-Boris/mr-Blip"],"state":"official: harvested, nothing ran","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":1,"ran_from_kinds":[]}}},{"paper":"/paper/what-do-vlms-notice-a-mechanistic","slug":"what-do-vlms-notice-a-mechanistic","title":"What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation","date":"2024-06-24","arxiv_id":"2406.16320","n_code_links":1,"syntology":null},{"paper":null,"slug":"ospc-detecting-harmful-memes-with-large","title":"OSPC: Detecting Harmful Memes with Large Language Model as a Catalyst","date":"2024-06-14","arxiv_id":"2406.09779","n_code_links":0,"syntology":null},{"paper":null,"slug":"how-structured-are-the-representations-in","title":"Interpreting the structure of multi-object representations in vision encoders","date":"2024-06-13","arxiv_id":"2406.09067","n_code_links":0,"syntology":null},{"paper":null,"slug":"blip-facilitating-the-exploration-of","title":"BLIP: Facilitating the Exploration of Undesirable Consequences of Digital Technologies","date":"2024-05-10","arxiv_id":"2405.06783","n_code_links":0,"syntology":null},{"paper":null,"slug":"q-groundcam-quantifying-grounding-in-vision","title":"Q-GroundCAM: Quantifying Grounding in Vision Language Models via GradCAM","date":"2024-04-29","arxiv_id":"2404.19128","n_code_links":0,"syntology":null},{"paper":null,"slug":"accidentblip2-accident-detection-with-multi","title":"AccidentBlip: Agent of Accident Warning based on MA-former","date":"2024-04-18","arxiv_id":"2404.12149","n_code_links":0,"syntology":null},{"paper":"/paper/aapmt-agi-assessment-through-prompt-and","slug":"aapmt-agi-assessment-through-prompt-and","title":"AAPMT: AGI Assessment Through Prompt and Metric Transformer","date":"2024-03-28","arxiv_id":"2403.19101","n_code_links":1,"syntology":null},{"paper":null,"slug":"the-solution-for-the-cvpr-2023-1st-foundation","title":"The Solution for the CVPR 2023 1st foundation model challenge-Track2","date":"2024-03-26","arxiv_id":"2403.17702","n_code_links":0,"syntology":null},{"paper":null,"slug":"learning-from-models-and-data-for-visual","title":"Learning from Synthetic Data for Visual Grounding","date":"2024-03-20","arxiv_id":"2403.13804","n_code_links":0,"syntology":null},{"paper":null,"slug":"tt-blip-enhancing-fake-news-detection-using","title":"TT-BLIP: Enhancing Fake News Detection Using BLIP and Tri-Transformer","date":"2024-03-19","arxiv_id":"2403.12481","n_code_links":0,"syntology":null},{"paper":"/paper/does-the-performance-of-text-to-image","slug":"does-the-performance-of-text-to-image","title":"Does the Performance of Text-to-Image Retrieval Models Generalize Beyond Captions-as-a-Query?","date":"2024-03-15","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/madtp-multimodal-alignment-guided-dynamic","slug":"madtp-multimodal-alignment-guided-dynamic","title":"MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer","date":"2024-03-05","arxiv_id":"2403.02991","n_code_links":1,"syntology":{"ran":3,"of":7,"n_ran_checked":2,"n_instrument":1,"unverified":4,"pointer_only":1,"phrase":"3 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified","official":{"repos":["double125/madtp"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":2,"n_ran_no_instrument_failure":2,"n_unverified":4,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"exploring-failure-cases-in-multimodal","title":"Exploring Failure Cases in Multimodal Reasoning About Physical Dynamics","date":"2024-02-24","arxiv_id":"2402.15654","n_code_links":0,"syntology":null},{"paper":null,"slug":"ci-w-o-tn-context-injection-without-task-name","title":"CI w/o TN: Context Injection without Task Name for Procedure Planning","date":"2024-02-23","arxiv_id":"2402.15579","n_code_links":0,"syntology":null},{"paper":null,"slug":"cic-a-framework-for-culturally-aware-image","title":"CIC: A Framework for Culturally-Aware Image Captioning","date":"2024-02-08","arxiv_id":"2402.05374","n_code_links":0,"syntology":null},{"paper":"/paper/colorswap-a-color-and-word-order-dataset-for","slug":"colorswap-a-color-and-word-order-dataset-for","title":"ColorSwap: A Color and Word Order Dataset for Multimodal Evaluation","date":"2024-02-07","arxiv_id":"2402.04492","n_code_links":1,"syntology":{"ran":3,"of":3,"n_ran_checked":2,"n_instrument":1,"unverified":0,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 2 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified","official":{"repos":["top34051/colorswap"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"cdvgan-one-flexible-model-for-multi-class","title":"cDVGAN: One Flexible Model for Multi-class Gravitational Wave Signal and Glitch Generation","date":"2024-01-29","arxiv_id":"2401.16356","n_code_links":0,"syntology":null},{"paper":"/paper/grounded-sam-assembling-open-world-models-for","slug":"grounded-sam-assembling-open-world-models-for","title":"Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks","date":"2024-01-25","arxiv_id":"2401.14159","n_code_links":5,"syntology":{"ran":15,"of":15,"n_ran_checked":7,"n_instrument":8,"unverified":0,"pointer_only":5,"phrase":"15 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 0 violated, 6 with no contract checked; 8 where Syntology's instrument failed) · 0 unverified","official":{"repos":["IDEA-Research/Grounded-Segment-Anything"],"state":"community repositories only","n_ran":0,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["listed"]}}},{"paper":"/paper/scimmir-benchmarking-scientific-multi-modal","slug":"scimmir-benchmarking-scientific-multi-modal","title":"SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval","date":"2024-01-24","arxiv_id":"2401.13478","n_code_links":1,"syntology":{"ran":9,"of":10,"n_ran_checked":3,"n_instrument":6,"unverified":1,"pointer_only":10,"phrase":"9 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 1 unverified","official":{"repos":["wusiwei0410/scimmir"],"state":"official (archive's flag): 9 ran","n_ran":9,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"bootstrapping-ots-funcimg-pre-training-model","title":"Bootstrapping OTS-Funcimg Pre-training Model (Botfip) -- A Comprehensive Symbolic Regression Framework","date":"2024-01-18","arxiv_id":"2401.09748","n_code_links":0,"syntology":null},{"paper":null,"slug":"compress-align-curating-image-text-data-with","title":"Filter & Align: Leveraging Human Knowledge to Curate Image-Text Data","date":"2023-12-11","arxiv_id":"2312.06726","n_code_links":0,"syntology":null},{"paper":"/paper/self-guided-open-vocabulary-semantic","slug":"self-guided-open-vocabulary-semantic","title":"Auto-Vocabulary Semantic Segmentation","date":"2023-12-07","arxiv_id":"2312.04539","n_code_links":1,"syntology":null},{"paper":null,"slug":"sinkhorn-transformations-for-single-query","title":"Sinkhorn Transformations for Single-Query Postprocessing in Text-Video Retrieval","date":"2023-11-14","arxiv_id":"2311.08143","n_code_links":0,"syntology":null},{"paper":"/paper/language-guided-visual-question-answering","slug":"language-guided-visual-question-answering","title":"Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts","date":"2023-10-31","arxiv_id":"2310.20159","n_code_links":1,"syntology":{"ran":3,"of":4,"n_ran_checked":2,"n_instrument":1,"unverified":1,"pointer_only":4,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified","official":{"repos":["declare-lab/lg-vqa"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/are-natural-domain-foundation-models-useful","slug":"are-natural-domain-foundation-models-useful","title":"Are Natural Domain Foundation Models Useful for Medical Image Classification?","date":"2023-10-30","arxiv_id":"2310.19522","n_code_links":1,"syntology":null},{"paper":"/paper/what-s-up-with-vision-language-models","slug":"what-s-up-with-vision-language-models","title":"What's \"up\" with vision-language models? Investigating their struggle with spatial reasoning","date":"2023-10-30","arxiv_id":"2310.19785","n_code_links":2,"syntology":{"ran":5,"of":9,"n_ran_checked":0,"n_instrument":5,"unverified":4,"pointer_only":8,"phrase":"5 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 4 unverified","official":{"repos":["amitakamath/whatsup_vlms"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":4,"ran_from_kinds":["listed","official","unlocated"]}}},{"paper":"/paper/vision-by-language-for-training-free","slug":"vision-by-language-for-training-free","title":"Vision-by-Language for Training-Free Compositional Image Retrieval","date":"2023-10-13","arxiv_id":"2310.09291","n_code_links":1,"syntology":{"ran":4,"of":8,"n_ran_checked":3,"n_instrument":1,"unverified":4,"pointer_only":2,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified","official":{"repos":["explainableml/vision_by_language"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":3,"n_unverified":4,"ran_from_kinds":["official"]}}},{"paper":"/paper/implicit-differentiable-outlier-detection","slug":"implicit-differentiable-outlier-detection","title":"Implicit Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis","date":"2023-09-21","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/diversemotion-towards-diverse-human-motion","slug":"diversemotion-towards-diverse-human-motion","title":"DiverseMotion: Towards Diverse Human Motion Generation via Discrete Diffusion","date":"2023-09-04","arxiv_id":"2309.01372","n_code_links":0,"syntology":null},{"paper":"/paper/fire-food-image-to-recipe-generation","slug":"fire-food-image-to-recipe-generation","title":"FIRE: Food Image to REcipe generation","date":"2023-08-28","arxiv_id":"2308.14391","n_code_links":1,"syntology":{"ran":12,"of":16,"n_ran_checked":10,"n_instrument":2,"unverified":4,"pointer_only":16,"phrase":"12 ran (of which 2 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified","official":{"repos":["prateekchhikara/fire"],"state":"official (archive's flag): 12 ran","n_ran":12,"n_constructed":2,"n_ran_no_instrument_failure":10,"n_unverified":4,"ran_from_kinds":["official"]}}},{"paper":"/paper/towards-vision-language-mechanistic","slug":"towards-vision-language-mechanistic","title":"Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP","date":"2023-08-27","arxiv_id":"2308.14179","n_code_links":1,"syntology":null},{"paper":null,"slug":"dlip-distilling-language-image-pre-training","title":"DLIP: Distilling Language-Image Pre-training","date":"2023-08-24","arxiv_id":"2308.12956","n_code_links":0,"syntology":null},{"paper":null,"slug":"sculpt-shape-conditioned-unpaired-learning-of","title":"SCULPT: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes","date":"2023-08-21","arxiv_id":"2308.10638","n_code_links":0,"syntology":null},{"paper":"/paper/figcaps-hf-a-figure-to-caption-generative","slug":"figcaps-hf-a-figure-to-caption-generative","title":"FigCaps-HF: A Figure-to-Caption Generative Framework and Benchmark with Human Feedback","date":"2023-07-20","arxiv_id":"2307.10867","n_code_links":1,"syntology":{"ran":2,"of":4,"n_ran_checked":0,"n_instrument":2,"unverified":2,"pointer_only":4,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified","official":{"repos":["figcapshf/figcapshf"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"divide-evaluate-and-refine-evaluating-and","title":"Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback","date":"2023-07-10","arxiv_id":"2307.04749","n_code_links":0,"syntology":null},{"paper":"/paper/probvlm-probabilistic-adapter-for-frozen","slug":"probvlm-probabilistic-adapter-for-frozen","title":"ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models","date":"2023-07-01","arxiv_id":"2307.00398","n_code_links":1,"syntology":{"ran":8,"of":15,"n_ran_checked":5,"n_instrument":3,"unverified":7,"pointer_only":2,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 7 unverified","official":{"repos":["explainableml/probvlm"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":7,"ran_from_kinds":["official"]}}},{"paper":"/paper/zero-shot-nuclei-detection-via-visual","slug":"zero-shot-nuclei-detection-via-visual","title":"Zero-shot Nuclei Detection via Visual-Language Pre-trained Models","date":"2023-06-30","arxiv_id":"2306.17659","n_code_links":1,"syntology":null},{"paper":null,"slug":"training-multimedia-event-extraction-with","title":"Training Multimedia Event Extraction With Generated Images and Captions","date":"2023-06-15","arxiv_id":"2306.08966","n_code_links":0,"syntology":null},{"paper":"/paper/recognize-anything-a-strong-image-tagging","slug":"recognize-anything-a-strong-image-tagging","title":"Recognize Anything: A Strong Image Tagging Model","date":"2023-06-06","arxiv_id":"2306.03514","n_code_links":2,"syntology":null},{"paper":null,"slug":"using-visual-cropping-to-enhance-fine-detail","title":"Using Visual Cropping to Enhance Fine-Detail Question Answering of BLIP-Family Models","date":"2023-05-31","arxiv_id":"2306.00228","n_code_links":0,"syntology":null},{"paper":"/paper/fusecap-leveraging-large-language-models-to","slug":"fusecap-leveraging-large-language-models-to","title":"FuseCap: Leveraging Large Language Models for Enriched Fused Image Captions","date":"2023-05-28","arxiv_id":"2305.17718","n_code_links":1,"syntology":null},{"paper":"/paper/on-evaluating-adversarial-robustness-of-large","slug":"on-evaluating-adversarial-robustness-of-large","title":"On Evaluating Adversarial Robustness of Large Vision-Language Models","date":"2023-05-26","arxiv_id":"2305.16934","n_code_links":1,"syntology":{"ran":4,"of":4,"n_ran_checked":2,"n_instrument":2,"unverified":0,"pointer_only":2,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified","official":{"repos":["yunqing-me/attackvlm"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/can-language-models-understand-physical","slug":"can-language-models-understand-physical","title":"Can Language Models Understand Physical Concepts?","date":"2023-05-23","arxiv_id":"2305.14057","n_code_links":1,"syntology":null},{"paper":null,"slug":"instructvid2vid-controllable-video-editing","title":"InstructVid2Vid: Controllable Video Editing with Natural Language Instructions","date":"2023-05-21","arxiv_id":"2305.12328","n_code_links":0,"syntology":null},{"paper":"/paper/llmscore-unveiling-the-power-of-large-1","slug":"llmscore-unveiling-the-power-of-large-1","title":"LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation","date":"2023-05-18","arxiv_id":"2305.11116","n_code_links":1,"syntology":null},{"paper":"/paper/iiitd-20k-dense-captioning-for-text-image","slug":"iiitd-20k-dense-captioning-for-text-image","title":"IIITD-20K: Dense captioning for Text-Image ReID","date":"2023-05-08","arxiv_id":"2305.04497","n_code_links":1,"syntology":null},{"paper":"/paper/data-curation-for-image-captioning-with-text","slug":"data-curation-for-image-captioning-with-text","title":"The Role of Data Curation in Image Captioning","date":"2023-05-05","arxiv_id":"2305.03610","n_code_links":1,"syntology":null},{"paper":"/paper/rococo-robust-benchmark-ms-coco-to-stress","slug":"rococo-robust-benchmark-ms-coco-to-stress","title":"RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching Models","date":"2023-04-21","arxiv_id":"2304.10727","n_code_links":1,"syntology":null},{"paper":"/paper/anything-3d-towards-single-view-anything","slug":"anything-3d-towards-single-view-anything","title":"Anything-3D: Towards Single-view Anything Reconstruction in the Wild","date":"2023-04-19","arxiv_id":"2304.10261","n_code_links":1,"syntology":{"ran":11,"of":13,"n_ran_checked":8,"n_instrument":3,"unverified":2,"pointer_only":3,"phrase":"11 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 2 honoured, 1 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified","official":{"repos":["anything-of-anything/anything-3d"],"state":"official (archive's flag): 11 ran","n_ran":11,"n_constructed":0,"n_ran_no_instrument_failure":8,"n_unverified":2,"ran_from_kinds":["official"]}}},{"paper":"/paper/cross-domain-image-captioning-with","slug":"cross-domain-image-captioning-with","title":"Cross-Domain Image Captioning with Discriminative Finetuning","date":"2023-04-04","arxiv_id":"2304.01662","n_code_links":1,"syntology":{"ran":2,"of":2,"n_ran_checked":2,"n_instrument":0,"unverified":0,"pointer_only":0,"phrase":"2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified","official":{"repos":["facebookresearch/EGG"],"state":"official (archive's flag): 2 ran","n_ran":2,"n_constructed":0,"n_ran_no_instrument_failure":2,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":"/paper/is-multi-modal-vision-supervision-beneficial","slug":"is-multi-modal-vision-supervision-beneficial","title":"Is Multimodal Vision Supervision Beneficial to Language?","date":"2023-02-10","arxiv_id":"2302.05016","n_code_links":1,"syntology":null},{"paper":null,"slug":"do-dall-e-and-flamingo-understand-each-other","title":"Do DALL-E and Flamingo Understand Each Other?","date":"2022-12-23","arxiv_id":"2212.12249","n_code_links":0,"syntology":null},{"paper":"/paper/position-guided-text-prompt-for-vision","slug":"position-guided-text-prompt-for-vision","title":"Position-guided Text Prompt for Vision-Language Pre-training","date":"2022-12-19","arxiv_id":"2212.09737","n_code_links":1,"syntology":{"ran":4,"of":5,"n_ran_checked":1,"n_instrument":3,"unverified":1,"pointer_only":0,"phrase":"4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified","official":{"repos":["sail-sg/ptp"],"state":"official (archive's flag): 4 ran","n_ran":4,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"renmin-university-of-china-at-trecvid-2022","title":"Renmin University of China at TRECVID 2022: Improving Video Search by Feature Fusion and Negation Understanding","date":"2022-11-28","arxiv_id":"2211.15039","n_code_links":0,"syntology":null},{"paper":"/paper/abductive-action-inference","slug":"abductive-action-inference","title":"Inferring Past Human Actions in Homes with Abductive Reasoning","date":"2022-10-24","arxiv_id":"2210.13984","n_code_links":1,"syntology":null},{"paper":null,"slug":"litevl-efficient-video-language-learning-with","title":"LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling","date":"2022-10-21","arxiv_id":"2210.11929","n_code_links":0,"syntology":null},{"paper":null,"slug":"synthetic-blip-effects-generalizing-synthetic","title":"Synthetic Blip Effects: Generalizing Synthetic Controls for the Dynamic Treatment Regime","date":"2022-10-20","arxiv_id":"2210.11003","n_code_links":0,"syntology":null},{"paper":"/paper/blip-bootstrapping-language-image-pre","slug":"blip-bootstrapping-language-image-pre","title":"BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation","date":"2022-01-28","arxiv_id":"2201.12086","n_code_links":9,"syntology":null}],"papers_shown":93,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":93,"with_a_code_link":44,"where_syntology_ran_a_sample":16,"not_listed_spam_title":0,"listed":93,"listed_where_code_ran":16,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":13,"every_run_a_failure_of_syntologys_instrument":3,"listed_with_a_run_with_no_instrument_failure":13,"listed_every_run_a_failure_of_syntologys_instrument":3,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":16,"papers_where_code_ran_first_page":"/method/blip/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/blip/papers/ran/1","json":"/method/blip/papers/ran/1.json","rows":16}],"tasks":[{"task":"/task/retrieval","name":"Retrieval","papers":22},{"task":"/task/image-captioning","name":"Image Captioning","papers":19},{"task":"/task/question-answering","name":"Question Answering","papers":14},{"task":"/task/visual-question-answering-1","name":"Visual Question Answering","papers":13},{"task":"/task/language-modelling","name":"Language Modelling","papers":12},{"task":"/task/image-generation","name":"Image Generation","papers":9},{"task":"/task/text-retrieval","name":"Text Retrieval","papers":9},{"task":"/task/visual-question-answering","name":"Visual Question Answering (VQA)","papers":9},{"task":"/task/decoder","name":"Decoder","papers":7},{"task":"/task/image-text-retrieval","name":"Image-text Retrieval","papers":7},{"task":"/task/language-modeling","name":"Language Modeling","papers":7},{"task":"/task/large-language-model","name":"Large Language Model","papers":6},{"task":"/task/attribute","name":"Attribute","papers":5},{"task":"/task/cross-modal-retrieval","name":"Cross-Modal Retrieval","papers":5},{"task":"/task/image-retrieval","name":"Image Retrieval","papers":5},{"task":"/task/object-detection","name":"Object Detection","papers":5},{"task":"/task/object-detection-1","name":"object-detection","papers":5},{"task":"/task/diversity","name":"Diversity","papers":4},{"task":"/task/image-classification","name":"Image Classification","papers":4},{"task":"/task/image-text-matching","name":"Image-text matching","papers":4}],"tasks_shown":20,"n_tasks":123,"usage_by_year":[{"year":"2022","papers":7},{"year":"2023","papers":31},{"year":"2024","papers":42},{"year":"2025","papers":13}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/blip"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=BLIP","read_at":"2026-10-05T00:53:12+00:00","query":"BLIP","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"BLIP","origin_arxiv_id":"2201.12086"},"resolution":"exact","resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"same_paper","archive_paper_arxiv_id":"2201.12086","repositories":{"listed":5,"in_graph":9,"list_truncated":true,"strongest_link":"official"},"retrieve_with":null,"name_competition":{"verdict":"contested","may_be_wrong_sense":false,"bearers_found":2,"served":{"arxiv_id":"2201.12086","citations":7251},"leading_other_bearer":{"arxiv_id":"2105.04444","title":"Continual Learning via Bit-Level Information Preserving","citations":54,"named_by":["acronym_expansion"],"bearer_name":"Bit-Level Information Preserving (BLIP)","ask_with":"have('2105.04444')"},"n_other_bearers_listed":1,"note":"1 other thing(s) here are also called 'BLIP', and none of them is cited more than the paper this answer is about. They are listed in `also_named` -- check `served.arxiv_id` is the paper you meant."},"name_competition_text":null,"near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"2201.12086","page":"/paper/blip-bootstrapping-language-image-pre","syntology_url":"https://syntology.ai/paper/2201.12086","in_graph_layer":false,"samples":0},"line":"repository known, no verified implementation (declared_code; repositories: 5 listed of 9 in the graph, strongest link: official). This answer is about arXiv:2201.12086, the paper the archive credits with this method.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"df21c20d1ea868cdf285d8ce45bc2a5b03761ee9b7eff211471e39bb2a1d0671","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}