{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/visualbert","slug":"visualbert","name":"VisualBERT","full_name":"VisualBERT","full_name_withheld":false,"description_markdown":"VisualBERT aims to reuse self-attention to implicitly align elements of the input text and regions in the input image. Visual embeddings are used to model images where the representations are represented by a bounding region in an image obtained from an object detector. These visual embeddings are constructed by summing three embeddings: 1) visual feature representation, 2) a segment embedding indicate whether it is an image embedding, and 3) position embedding. Essentially, image regions and language are combined with a Transformer to allow self-attention to discover implicit alignments between language and vision. VisualBERT is trained using COCO, which consists of images paired with captions. It is pre-trained using two objectives: masked language modeling objective and sentence-image prediction task. It can then be fine-tuned on different downstream tasks.","description_state":"present","introduced_year":null,"introduced_by":{"title":"VisualBERT: A Simple and Performant Baseline for Vision and Language","paper":"/paper/visualbert-a-simple-and-performant-baseline","first_author":"Liunian Harold Li","n_authors":5,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/visualbert-a-simple-and-performant-baseline"},"source":{"url":"https://arxiv.org/abs/1908.03557v1","title":"VisualBERT: A Simple and Performant Baseline for Vision and Language","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Vision and Language Pre-Trained Models","url":"/methods/category/vision-and-language-pre-trained-models","pwc_aliases":[]}],"n_papers_tagged":25,"archive_num_papers":25,"papers_newest_first":[{"paper":null,"slug":"visual-question-answering-on-multiple-remote","title":"Visual Question Answering on Multiple Remote Sensing Image Modalities","date":"2025-05-21","arxiv_id":"2505.15401","n_code_links":0,"syntology":null},{"paper":null,"slug":"seeing-through-visualbert-a-causal-adventure","title":"Seeing Through VisualBERT: A Causal Adventure on Memetic Landscapes","date":"2024-10-17","arxiv_id":"2410.13488","n_code_links":0,"syntology":null},{"paper":null,"slug":"ospc-detecting-harmful-memes-with-large","title":"OSPC: Detecting Harmful Memes with Large Language Model as a Catalyst","date":"2024-06-14","arxiv_id":"2406.09779","n_code_links":0,"syntology":null},{"paper":"/paper/beyond-image-text-matching-verb-understanding","slug":"beyond-image-text-matching-verb-understanding","title":"Beyond Image-Text Matching: Verb Understanding in Multimodal Transformers Using Guided Masking","date":"2024-01-29","arxiv_id":"2401.16575","n_code_links":1,"syntology":null},{"paper":"/paper/a-review-of-vision-language-models-and-their","slug":"a-review-of-vision-language-models-and-their","title":"A Review of Vision-Language Models and their Performance on the Hateful Memes Challenge","date":"2023-05-09","arxiv_id":"2305.06159","n_code_links":1,"syntology":null},{"paper":null,"slug":"controlling-for-stereotypes-in-multimodal","title":"Controlling for Stereotypes in Multimodal Language Model Evaluation","date":"2023-02-03","arxiv_id":"2302.01582","n_code_links":0,"syntology":null},{"paper":null,"slug":"a-survey-on-knowledge-enhanced-multimodal","title":"A survey on knowledge-enhanced multimodal learning","date":"2022-11-19","arxiv_id":"2211.12328","n_code_links":0,"syntology":null},{"paper":"/paper/transfer-learning-with-joint-fine-tuning-for","slug":"transfer-learning-with-joint-fine-tuning-for","title":"Transfer Learning with Joint Fine-Tuning for Multimodal Sentiment Analysis","date":"2022-10-11","arxiv_id":"2210.05790","n_code_links":1,"syntology":null},{"paper":null,"slug":"multi-modal-fusion-transformer-for-visual","title":"Multi-Modal Fusion Transformer for Visual Question Answering in Remote Sensing","date":"2022-10-10","arxiv_id":"2210.04510","n_code_links":0,"syntology":null},{"paper":"/paper/surgical-vqa-visual-question-answering-in","slug":"surgical-vqa-visual-question-answering-in","title":"Surgical-VQA: Visual Question Answering in Surgical Scenes using Transformer","date":"2022-06-22","arxiv_id":"2206.11053","n_code_links":3,"syntology":{"ran":1,"of":2,"n_ran_checked":1,"n_instrument":0,"unverified":1,"pointer_only":2,"phrase":"1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["lalithjets/surgical_vqa"],"state":"official (archive's flag): 1 ran","n_ran":1,"n_constructed":0,"n_ran_no_instrument_failure":1,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/visual-spatial-reasoning","slug":"visual-spatial-reasoning","title":"Visual Spatial Reasoning","date":"2022-04-30","arxiv_id":"2205.00363","n_code_links":4,"syntology":{"ran":7,"of":10,"n_ran_checked":5,"n_instrument":2,"unverified":3,"pointer_only":6,"phrase":"7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified","official":{"repos":["cambridgeltl/visual-spatial-reasoning","sohojoe/clip_visual-spatial-reasoning"],"state":"official (archive's flag): 7 ran","n_ran":7,"n_constructed":0,"n_ran_no_instrument_failure":5,"n_unverified":3,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"visio-linguistic-brain-encoding-1","title":"Visio-Linguistic Brain Encoding","date":"2022-04-18","arxiv_id":"2204.08261","n_code_links":0,"syntology":null},{"paper":"/paper/hateful-memes-challenge-an-enhanced","slug":"hateful-memes-challenge-an-enhanced","title":"Hateful Memes Challenge: An Enhanced Multimodal Framework","date":"2021-12-20","arxiv_id":"2112.11244","n_code_links":1,"syntology":null},{"paper":null,"slug":"multimodal-learning-are-captions-all-you-need","title":"Multimodal Learning: Are Captions All You Need?","date":"2021-11-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"seeing-things-or-seeing-scenes-investigating","title":"Seeing things or seeing scenes: Investigating the capabilities of V&L models to align scene descriptions to images","date":"2021-10-16","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"understanding-of-emotion-perception-from-art","title":"Understanding of Emotion Perception from Art","date":"2021-10-13","arxiv_id":"2110.06486","n_code_links":0,"syntology":null},{"paper":"/paper/marmot-a-deep-learning-framework-for","slug":"marmot-a-deep-learning-framework-for","title":"MARMOT: A Deep Learning Framework for Constructing Multimodal Representations for Vision-and-Language Tasks","date":"2021-09-23","arxiv_id":"2109.11526","n_code_links":1,"syntology":null},{"paper":null,"slug":"what-vision-language-models-see-when-they-see","title":"What Vision-Language Models `See' when they See Scenes","date":"2021-09-15","arxiv_id":"2109.07301","n_code_links":0,"syntology":null},{"paper":"/paper/broaden-the-vision-geo-diverse-visual","slug":"broaden-the-vision-geo-diverse-visual","title":"Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning","date":"2021-09-14","arxiv_id":"2109.06860","n_code_links":1,"syntology":{"ran":8,"of":9,"n_ran_checked":8,"n_instrument":0,"unverified":1,"pointer_only":0,"phrase":"8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified","official":{"repos":["wadeyin9712/gd-vcr"],"state":"official (archive's flag): 8 ran","n_ran":8,"n_constructed":0,"n_ran_no_instrument_failure":8,"n_unverified":1,"ran_from_kinds":["official"]}}},{"paper":"/paper/berthop-an-effective-vision-and-language","slug":"berthop-an-effective-vision-and-language","title":"BERTHop: An Effective Vision-and-Language Model for Chest X-ray Disease Diagnosis","date":"2021-08-10","arxiv_id":"2108.04938","n_code_links":1,"syntology":null},{"paper":null,"slug":"chop-chop-bert-visual-question-answering-by","title":"Chop Chop BERT: Visual Question Answering by Chopping VisualBERT's Heads","date":"2021-04-30","arxiv_id":"2104.14741","n_code_links":0,"syntology":null},{"paper":null,"slug":"analysis-on-image-set-visual-question","title":"Analysis on Image Set Visual Question Answering","date":"2021-03-31","arxiv_id":"2104.00107","n_code_links":0,"syntology":null},{"paper":"/paper/detecting-hate-speech-in-memes-using","slug":"detecting-hate-speech-in-memes-using","title":"Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challenge","date":"2020-12-23","arxiv_id":"2012.12975","n_code_links":1,"syntology":null},{"paper":"/paper/a-comparison-of-pre-trained-vision-and","slug":"a-comparison-of-pre-trained-vision-and","title":"A Comparison of Pre-trained Vision-and-Language Models for Multimodal Representation Learning across Medical Images and Reports","date":"2020-09-03","arxiv_id":"2009.01523","n_code_links":1,"syntology":null},{"paper":"/paper/visualbert-a-simple-and-performant-baseline","slug":"visualbert-a-simple-and-performant-baseline","title":"VisualBERT: A Simple and Performant Baseline for Vision and Language","date":"2019-08-09","arxiv_id":"1908.03557","n_code_links":10,"syntology":{"ran":4,"of":9,"n_ran_checked":2,"n_instrument":2,"unverified":5,"pointer_only":6,"phrase":"4 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified","official":null}}],"papers_shown":25,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":25,"with_a_code_link":12,"where_syntology_ran_a_sample":4,"not_listed_spam_title":0,"listed":25,"listed_where_code_ran":4,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":4,"every_run_a_failure_of_syntologys_instrument":0,"listed_with_a_run_with_no_instrument_failure":4,"listed_every_run_a_failure_of_syntologys_instrument":0,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":4,"papers_where_code_ran_first_page":"/method/visualbert/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/visualbert/papers/ran/1","json":"/method/visualbert/papers/ran/1.json","rows":4}],"tasks":[{"task":"/task/visual-question-answering","name":"Visual Question Answering (VQA)","papers":9},{"task":"/task/question-answering","name":"Question Answering","papers":7},{"task":"/task/visual-question-answering-1","name":"Visual Question Answering","papers":7},{"task":"/task/language-modeling","name":"Language Modeling","papers":4},{"task":"/task/language-modelling","name":"Language Modelling","papers":4},{"task":"/task/visual-reasoning","name":"Visual Reasoning","papers":4},{"task":"/task/image-captioning","name":"Image Captioning","papers":3},{"task":"/task/representation-learning","name":"Representation Learning","papers":3},{"task":"/task/multimodal-deep-learning","name":"Multimodal Deep Learning","papers":2},{"task":"/task/object","name":"Object","papers":2},{"task":"/task/visual-commonsense-reasoning","name":"Visual Commonsense Reasoning","papers":2},{"task":"/task/visual-entailment","name":"Visual Entailment","papers":2},{"task":"/task/all","name":"All","papers":1},{"task":"/task/conditional-image-generation","name":"Conditional Image Generation","papers":1},{"task":"/task/culture","name":"Cultural Vocal Bursts Intensity Prediction","papers":1},{"task":"/task/ensemble-learning","name":"Ensemble Learning","papers":1},{"task":"/task/factual-visual-question-answering","name":"Factual Visual Question Answering","papers":1},{"task":"/task/fairness","name":"Fairness","papers":1},{"task":"/task/image-text-retrieval","name":"Image-text Retrieval","papers":1},{"task":"/task/image-text-matching","name":"Image-text matching","papers":1}],"tasks_shown":20,"n_tasks":45,"usage_by_year":[{"year":"2019","papers":1},{"year":"2020","papers":2},{"year":"2021","papers":10},{"year":"2022","papers":6},{"year":"2023","papers":2},{"year":"2024","papers":3},{"year":"2025","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/visualbert"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=VisualBERT","read_at":"2026-10-05T01:03:44+00:00","query":"VisualBERT","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"VisualBERT","origin_arxiv_id":"1908.03557"},"resolution":"exact","resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"same_paper","archive_paper_arxiv_id":"1908.03557","repositories":{"listed":5,"in_graph":10,"list_truncated":true,"strongest_link":"mentioned_in_github"},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"1908.03557","citations":2380},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'VisualBERT'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"1908.03557","page":"/paper/visualbert-a-simple-and-performant-baseline","syntology_url":"https://syntology.ai/paper/1908.03557","in_graph_layer":true,"samples":9,"ran":4,"n_ran_checked":2,"n_instrument":2,"unverified":5,"phrase":"4 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified","instrument_means":"failures of Syntology's instrument, not of the code"},"line":"repository known, no verified implementation (declared_code; repositories: 5 listed of 10 in the graph, strongest link: mentioned on GitHub). This answer is about arXiv:1908.03557, the paper the archive credits with this method.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"7d0a641fdb7dc1d71596a9e10aaf2cc757f01a8c5aff4026e86451ab1b13ec14","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}