Papers › BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language...

BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

28 Jan 2022arXiv:2201.12086archive 2025-07-28

Junnan Li, Dongxu Li, Caiming Xiong, Steven Hoi

Vision-Language Pre-training (VLP) has advanced the performance for many vision-language tasks. However, most existing pre-trained models only excel in either understanding-based tasks or generation-based tasks. Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision. In this paper, we propose BLIP, a new VLP framework which transfers flexibly to both vision-language understanding and generation tasks. BLIP effectively utilizes the noisy web data by bootstrapping the captions, where a captioner generates synthetic captions and a filter removes the noisy ones. We achieve state-of-the-art results on a wide range of vision-language tasks, such as image-text retrieval (+2.7% in average recall@1), image captioning (+2.8% in CIDEr), and VQA (+1.6% in VQA score). BLIP also demonstrates strong generalization ability when directly transferred to video-language tasks in a zero-shot manner. Code, models, and datasets are released at https://github.com/salesforce/BLIP.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

salesforce/lavis officialmentioned on GitHubpytorchBSD-3-ClauseSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions (arXiv:2401.01827; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified) and When and How Does CLIP Enable Domain and Compositional Generalization? (arXiv:2502.09507; that paper's own run record: 24 ran (of which 0 constructed an object rather than computing a result; 16 with no instrument failure: 0 honoured, 0 violated, 16 with no contract checked; 8 where Syntology's instrument failed) · 8 unverified); nothing checks that they implement this paper's method. report
salesforce/blip mentioned in papermentioned on GitHubpytorchBSD-3-ClauseSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to AAAI id aaai_35495 (that paper's own run record: 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified); nothing checks that they implement this paper's method. report
huggingface/transformers mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Attention Is All You Need (arXiv:1706.03762; that paper's own run record: 610 ran (of which 293 constructed an object rather than computing a result; 529 with no instrument failure: 45 honoured, 15 violated, 469 with no contract checked; 81 where Syntology's instrument failed) · 336 unverified), HuggingFace's Transformers: State-of-the-art Natural Language Processing (arXiv:1910.03771; that paper's own run record: 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified), DeBERTa: Decoding-enhanced BERT with Disentangled Attention (arXiv:2006.03654; that paper's own run record: 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 9 unverified) and 1 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report
ninatu/howtocaption mentioned on GitHubpytorchNOASSERTIONSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to HowToCaption: Prompting LLMs to Transform Video Annotations at Scale (arXiv:2310.04900; that paper's own run record: 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
2023-MindSpore-4/Code12 mindsporeSyntology: not harvested report
2024-MindSpore-1/Code2 mindsporeSyntology: not harvested report
alibaba/EasyNLP jaxSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Learning Knowledge-Enhanced Contextual Language Representations for Domain Natural... (arXiv:2311.06761; that paper's own run record: 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
pwc-1/Paper-8 mindsporeSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models... (arXiv:2406.18321; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
towhee-io/towhee pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Deep Residual Learning for Image Recognition (arXiv:1512.03385; that paper's own run record: 254 ran (of which 108 constructed an object rather than computing a result; 163 with no instrument failure: 3 honoured, 1 violated, 159 with no contract checked; 91 where Syntology's instrument failed) · 123 unverified), Perceiver: General Perception with Iterative Attention (arXiv:2103.03206; that paper's own run record: 41 ran (of which 20 constructed an object rather than computing a result; 30 with no instrument failure: 0 honoured, 2 violated, 28 with no contract checked; 11 where Syntology's instrument failed) · 14 unverified), LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval (arXiv:2103.08784; that paper's own run record: 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified) and 9 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Image CaptioningImage-text RetrievalImage-text matchingOpen Vocabulary Attribute DetectionRetrievalText RetrievalVisual Question Answering (VQA)Visual Reasoning

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Image Captioning nocaps-val-in-domain BLIP_ViT-L CIDEr 114.9 #5 of 11 Archive leaderboard report
Image Captioning nocaps-val-in-domain BLIP_ViT-L Pre-train (#images) 129M #5 of 11 Archive leaderboard report
Image Captioning nocaps-val-in-domain BLIP_ViT-L SPICE 15.2 #5 of 11 Archive leaderboard report
Image Captioning nocaps-val-in-domain BLIP_CapFilt-L CIDEr 111.8 #7 of 11 Archive leaderboard report
Image Captioning nocaps-val-in-domain BLIP_CapFilt-L Pre-train (#images) 129M #7 of 11 Archive leaderboard report
Image Captioning nocaps-val-in-domain BLIP_CapFilt-L SPICE 14.9 #7 of 11 Archive leaderboard report
Image Captioning nocaps-val-near-domain BLIP_ViT-L CIDEr 112.1 #5 of 10 Archive leaderboard report
Image Captioning nocaps-val-near-domain BLIP_ViT-L Pre-train (#images) 129M #5 of 10 Archive leaderboard report
Image Captioning nocaps-val-near-domain BLIP_ViT-L SPICE 14.9 #5 of 10 Archive leaderboard report
Image Captioning nocaps-val-near-domain BLIP_CapFilt-L CIDEr 108.6 #7 of 10 Archive leaderboard report
Image Captioning nocaps-val-near-domain BLIP_CapFilt-L Pre-train (#images) 129M #7 of 10 Archive leaderboard report
Image Captioning nocaps-val-near-domain BLIP_CapFilt-L SPICE 14.8 #7 of 10 Archive leaderboard report
Image Captioning nocaps-val-out-domain BLIP_ViT-L CIDEr 115.3 #4 of 10 Archive leaderboard report
Image Captioning nocaps-val-out-domain BLIP_ViT-L Pretrain (#images) 129M #4 of 10 Archive leaderboard report
Image Captioning nocaps-val-out-domain BLIP_ViT-L SPICE 14.4 #4 of 10 Archive leaderboard report
Image Captioning nocaps-val-out-domain BLIP_CapFilt-L CIDEr 111.5 #6 of 10 Archive leaderboard report
Image Captioning nocaps-val-out-domain BLIP_CapFilt-L Pretrain (#images) 129M #6 of 10 Archive leaderboard report
Image Captioning nocaps-val-out-domain BLIP_CapFilt-L SPICE 14.2 #6 of 10 Archive leaderboard report
Image Captioning nocaps-val-overall BLIP_ViT-L CIDEr 113.2 #5 of 11 Archive leaderboard report
Image Captioning nocaps-val-overall BLIP_ViT-L Pretrain (#images) 129M #5 of 11 Archive leaderboard report
Image Captioning nocaps-val-overall BLIP_ViT-L SPICE 14.8 #5 of 11 Archive leaderboard report
Image Captioning nocaps-val-overall BLIP_CapFilt-L CIDEr 109.6 #7 of 11 Archive leaderboard report
Image Captioning nocaps-val-overall BLIP_CapFilt-L Pretrain (#images) 129M #7 of 11 Archive leaderboard report
Image Captioning nocaps-val-overall BLIP_CapFilt-L SPICE 14.7 #7 of 11 Archive leaderboard report
Image-text matching CommercialAdsDataset BLIP ADD(S) AUC 83.51 #6 of 8 Archive leaderboard report
Open Vocabulary Attribute Detection OVAD-Box benchmark BLIP mean average precision 24.3 #3 of 7 Archive leaderboard report
Visual Reasoning NLVR2 Test BLIP-129M Accuracy 83.09 #9 of 14 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Introduced by this paper: BLIP

BLIP

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections