Papers › Improving Text-to-Image Synthesis Using Contrastive Learning

Improving Text-to-Image Synthesis Using Contrastive Learning

6 Jul 2021arXiv:2107.02423archive 2025-07-28

Hui Ye, Xiulong Yang, Martin Takac, Rajshekhar Sunderraman, Shihao Ji

The goal of text-to-image synthesis is to generate a visually realistic image that matches a given text description. In practice, the captions annotated by humans for the same image have large variance in terms of contents and the choice of words. The linguistic discrepancy between the captions of the identical image leads to the synthetic images deviating from the ground truth. To address this issue, we propose a contrastive learning approach to improve the quality and enhance the semantic consistency of synthetic images. In the pretraining stage, we utilize the contrastive learning approach to learn the consistent textual representations for the captions corresponding to the same image. Furthermore, in the following stage of GAN training, we employ the contrastive learning method to enhance the consistency between the generated images from the captions related to the same image. We evaluate our approach over two popular text-to-image synthesis models, AttnGAN and DM-GAN, on datasets CUB and COCO, respectively. Experimental results have shown that our approach can effectively improve the quality of synthetic images in terms of three metrics: IS, FID and R-precision. Especially, on the challenging COCO dataset, our approach boosts the FID signifcantly by 29.60% over AttnGAN and by 21.96% over DM-GAN.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

huiyegit/T2I_CL officialmentioned in papermentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis (arXiv:1904.01310; that paper's own run record: 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Contrastive LearningImage GenerationText-to-Image Generation

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Text-to-Image Generation COCO (Common Objects in Context) DM-GAN+CL FID 20.79 #44 of 69 Archive leaderboard report
Text-to-Image Generation COCO (Common Objects in Context) DM-GAN+CL Inception score 33.34 #44 of 69 Archive leaderboard report
Text-to-Image Generation COCO (Common Objects in Context) AttnGAN+CL FID 23.93 #48 of 69 Archive leaderboard report
Text-to-Image Generation COCO (Common Objects in Context) AttnGAN+CL Inception score 25.70 #48 of 69 Archive leaderboard report
Text-to-Image Generation CUB DM-GAN+CL FID 14.38 #10 of 20 Archive leaderboard report
Text-to-Image Generation CUB DM-GAN+CL Inception score 4.77 #10 of 20 Archive leaderboard report
Text-to-Image Generation CUB AttnGAN+CL FID 16.34 #12 of 20 Archive leaderboard report
Text-to-Image Generation CUB AttnGAN+CL Inception score 4.42 #12 of 20 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Contrastive Learning

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections