Papers › SynCo: Synthetic Hard Negatives in Contrastive Learning for Better Unsupervised Visual...

SynCo: Synthetic Hard Negatives in Contrastive Learning for Better Unsupervised Visual Representations

3 Oct 2024arXiv:2410.02401archive 2025-07-28

Nikolaos Giakoumoglou, Tania Stathaki

Contrastive learning has become a dominant approach in self-supervised visual representation learning. Hard negatives - samples closely resembling the anchor - are key to enhancing learned representations' discriminative power. However, efficiently leveraging hard negatives remains challenging. We introduce SynCo (Synthetic Negatives in Contrastive learning), a novel approach that improves model performance by generating synthetic hard negatives on the representation space. Building on the MoCo framework, SynCo introduces six strategies for creating diverse synthetic hard negatives on-the-fly with minimal computational overhead. SynCo achieves faster training and better representation learning, reaching 67.9% top-1 accuracy on ImageNet ILSVRC-2012 linear evaluation after 200 pretraining epochs, surpassing MoCo's 67.5% using the same ResNet-50 encoder. It also transfers more effectively to detection tasks: on PASCAL VOC, it outperforms both the supervised baseline and MoCo with 82.5% AP; on COCO, it sets new benchmarks with 40.9% AP for bounding box detection and 35.5% AP for instance segmentation. Our synthetic hard negative generation approach significantly enhances visual representations learned through self-supervised contrastive learning. Code is available at https://github.com/giakoumoglou/synco.

PaperPDFCode

Syntology This paper's arXiv id is not in Syntology's Atlas id list (549,449 ids, as this build read them; the read time is in the build record), so this page links to no Syntology page for it. The list is not the whole graph; Syntology's feed shows papers as they enter the graph.

Code

giakoumoglou/synco officialmentioned in papermentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Momentum Contrast for Unsupervised Visual Representation Learning (arXiv:1911.05722; that paper's own run record: 28 ran (of which 16 constructed an object rather than computing a result; 23 with no instrument failure: 0 honoured, 0 violated, 23 with no contract checked; 5 where Syntology's instrument failed) · 14 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Contrastive LearningImage ClassificationImage SegmentationInstance SegmentationLinear evaluationObject DetectionRepresentation LearningSelf-Supervised Image ClassificationSelf-Supervised LearningSemantic SegmentationSemi-Supervised Image ClassificationUnsupervised Pre-training

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Image Segmentation COCO val2017 SynCo (ResNet-50) 200ep mask AP 35.4 #1 of 1 Archive leaderboard report
Object Detection COCO val2017 SynCo (ResNet-50) 200ep Bounding Box AP 40.4 #1 of 1 Archive leaderboard report
Object Detection PASCAL VOC 2012 test SynCo (ResNet-50) 200ep Bounding Box AP 57.2 #1 of 1 Archive leaderboard report
Self-Supervised Image Classification ImageNet SynCo (ResNet-50) 800ep Number of Params 24M #101 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet SynCo (ResNet-50) 800ep Top 1 Accuracy 70.6% #101 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet SynCo (ResNet-50) 800ep Top 5 Accuracy 89.8% #101 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet SynCo (ResNet-50) 200ep Number of Params 24M #108 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet SynCo (ResNet-50) 200ep Top 1 Accuracy 67.9% #108 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet SynCo (ResNet-50) 200ep Top 5 Accuracy 88 #108 of 144 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data SynCo (ResNet-50) 800ep Number of params 24M #51 of 65 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data SynCo (ResNet-50) 800ep Top 1 Accuracy 50.8% #51 of 65 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data SynCo (ResNet-50) 800ep Top 5 Accuracy 77.5% #51 of 65 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 10% labeled data SynCo (ResNet-50) 800ep Number of params 24M #44 of 75 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 10% labeled data SynCo (ResNet-50) 800ep Top 1 Accuracy 66.6% #44 of 75 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 10% labeled data SynCo (ResNet-50) 800ep Top 5 Accuracy 88.0% #44 of 75 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Batch NormalizationContrastive LearningFaster R-CNNInfoNCEMask R-CNNMoCo

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections