Methods › General › Semi-Supervised Learning Methods › Contrastive Predictive Coding
Contrastive Predictive Coding
Introduced by Aaron van den Oord et al. in Representation Learning with Contrastive Predictive Coding
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Contrastive Predictive Coding (CPC) learns self-supervised representations by predicting the future in latent space by using powerful autoregressive models. The model uses a probabilistic contrastive loss which induces the latent space to capture information that is maximally useful to predict future samples.
First, a non-linear encoder g_(enc) maps the input sequence of observations xₜ to a sequence of latent representations zₜ = g_(enc)(xₜ), potentially with a lower temporal resolution. Next, an autoregressive model gₐᵣ summarizes all z≤t in the latent space and produces a context latent representation cₜ = gₐᵣ(z≤t).
A density ratio is modelled which preserves the mutual information between xₜ₊ₖ and cₜ as follows:
fₖ(xₜ₊ₖ, cₜ) ∝(p(xₜ₊ₖ|cₜ))/(p(xₜ₊ₖ))
where ∝ stands for ’proportional to’ (i.e. up to a multiplicative constant). Note that the density ratio f can be unnormalized (does not have to integrate to 1). The authors use a simple log-bilinear model:
fₖ(xₜ₊ₖ, cₜ) = exp(zᵀₜ₊ₖWₖcₜ)
Any type of autoencoder and autoregressive can be used. An example the authors opt for is strided convolutional layers with residual blocks and GRUs.
The autoencoder and autoregressive models are trained to minimize an InfoNCE loss (see components).
In Syntology repository known, no verified implementation (declared_code; repositories: 5 listed of 28 in the graph, strongest link: mentioned on GitHub). This answer is about arXiv:1807.03748, the paper the archive credits with this method.
Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.
- Spectral Contrastive Loss · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Spectral Contrastive Loss') - Maximum Entropy Coding (MEC) · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Maximum Entropy Coding (MEC)') - noise-contrastive estimation · shares_name_tokens (shares name tokens with this name; a different method) · level V2 · retrieve with
get_reference_implementation(name='noise-contrastive estimation') - Behavior Contrastive Learning · shares_name_tokens (shares name tokens with this name; a different method) · level V2 · retrieve with
get_reference_implementation(name='Behavior Contrastive Learning') - Relative Entropy Coding (REC) · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Relative Entropy Coding (REC)') - contrastive learning framework · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='contrastive learning framework')
Origin paper of this answer, arXiv:1807.03748 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 35 ran (of which 21 constructed an object rather than computing a result; 30 with no instrument failure: 1 honoured, 0 violated, 29 with no contract checked; 5 where Syntology's instrument failed) · 10 unverified (of 45 harvested samples) · 22 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
What each level means, quoted from Syntology's server card: V0 “imports”; V1 “executes against a synthesized fixture and returns a value of the recorded type”; V2 “two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs” “neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)”; V3 “paper-derived property tests pass on both of those same two generations.” The card adds: “No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results.” A level quoted in two parts is one definition the card writes as two clauses joined by a double hyphen, each clause quoted on its own (how these levels are quoted).
Papers archive 2025-07-28
archive papers tagged: 113 · with a code link: 39 · where Syntology ran a sample: 14 (12 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (14 of 113 tagged: 12 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 113, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Two-Player Zero-Sum Games with Bandit Feedback 17 Jun 2025 · 0 repositories · arXiv:2506.14518
-
Integration of Contrastive Predictive Coding and Spiking Neural Networks 10 Jun 2025 · 1 repository · arXiv:2506.09194
-
Koopman-Based Event-Triggered Control from Data 19 Apr 2025 · 0 repositories · arXiv:2504.14334
-
Learning Transformer-based World Models with Contrastive Predictive Coding 6 Mar 2025 · 0 repositories · arXiv:2503.04416
-
Contrastive Representation Learning Helps Cross-institutional Knowledge Transfer: A Study in Pediatric Ventilation Management 23 Jan 2025 · 0 repositories · arXiv:2501.13587
-
Performance-Barrier Event-Triggered PDE Control of Traffic Flow 1 Jan 2025 · 0 repositories · arXiv:2501.00722
-
Automated Toll Management System Using RFID and Image Processing 2 Dec 2024 · 0 repositories · arXiv:2412.01728
-
A Contrastive Self-Supervised Learning scheme for beat tracking amenable to few-shot learning 6 Nov 2024 · 0 repositories · arXiv:2411.04152
-
Trading through Earnings Seasons using Self-Supervised Contrastive Representation Learning 25 Sep 2024 · 0 repositories · arXiv:2409.17392
-
Context-Aware Predictive Coding: A Representation Learning Framework for WiFi Sensing 20 Sep 2024 · 1 repository
-
Context-Aware Predictive Coding: A Representation Learning Framework for WiFi Sensing 16 Sep 2024 · 1 repository · arXiv:2410.01825
-
Hierarchical Event-Triggered Systems: Safe Learning of Quasi-Optimal Deadline Policies 15 Sep 2024 · 0 repositories · arXiv:2409.09812
-
Speaker and Style Disentanglement of Speech Based on Contrastive Predictive Coding Supported Factorized Variational Autoencoder 5 Sep 2024 · 0 repositories · arXiv:2409.03520
-
Contrastive Representation Learning for Dynamic Link Prediction in Temporal Networks 22 Aug 2024 · 1 repository · arXiv:2408.12753
-
Performance-Barrier Event-Triggered Control of a Class of Reaction-Diffusion PDEs 11 Jul 2024 · 0 repositories · arXiv:2407.08178
-
Contextual Dynamic Pricing: Algorithms, Optimality, and Local Differential Privacy Constraints 4 Jun 2024 · 0 repositories · arXiv:2406.02424
-
Causal Contrastive Learning for Counterfactual Regression Over Time 1 Jun 2024 · 0 repositories · arXiv:2406.00535
-
Offline Reinforcement Learning from Datasets with Structured Non-Stationarity 23 May 2024 · 1 repository · arXiv:2405.14114Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 12 harvested samples)
-
Multilingual Turn-taking Prediction Using Voice Activity Projection 11 Mar 2024 · 0 repositories · arXiv:2403.06487
-
Event-Triggered Robust Cooperative Output Regulation for a Class of Linear Multi-Agent Systems with an Unknown Exosystem 1 Mar 2024 · 0 repositories · arXiv:2403.00645
-
Revisiting speech segmentation and lexicon learning with better features 31 Jan 2024 · 0 repositories · arXiv:2401.17902
-
Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection 10 Jan 2024 · 2 repositories · arXiv:2401.04868
-
Replication-proof Bandit Mechanism Design with Bayesian Agents 28 Dec 2023 · 0 repositories · arXiv:2312.16896
-
Bigger is not Always Better: The Effect of Context Size on Speech Pre-Training 3 Dec 2023 · 1 repository · arXiv:2312.01515
-
Representation Learning in a Decomposed Encoder Design for Bio-inspired Hebbian Learning 22 Nov 2023 · 1 repository · arXiv:2401.08603
-
Self-MI: Efficient Multimodal Fusion via Self-Supervised Multi-Task Learning with Auxiliary Mutual Information Maximization 7 Nov 2023 · 0 repositories · arXiv:2311.03785
-
Contrastive Difference Predictive Coding 31 Oct 2023 · 1 repository · arXiv:2310.20141Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
Learning-based Scheduling for Information Accuracy and Freshness in Wireless Networks 24 Oct 2023 · 0 repositories · arXiv:2310.15705
-
Listen to Minority: Encrypted Traffic Classification for Class Imbalance with Contrastive Pre-Training 31 Aug 2023 · 0 repositories · arXiv:2308.16453
-
Towards quantitative precision for ECG analysis: Leveraging state space models, self-supervision and patient metadata 29 Aug 2023 · 1 repository · arXiv:2308.15291Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples)
-
Federated Representation Learning for Automatic Speech Recognition 3 Aug 2023 · 0 repositories · arXiv:2308.02013
-
High-dimensional Contextual Bandit Problem without Sparsity 19 Jun 2023 · 0 repositories · arXiv:2306.11017
-
Permutation Decision Trees 5 Jun 2023 · 0 repositories · arXiv:2306.02617
-
ContraBAR: Contrastive Bayes-Adaptive Deep RL 4 Jun 2023 · 1 repository · arXiv:2306.02418
-
An Improved Heart Disease Prediction Using Stacked Ensemble Method 12 Apr 2023 · 0 repositories · arXiv:2304.06015
-
Regularizing Contrastive Predictive Coding for Speech Applications 12 Apr 2023 · 0 repositories · arXiv:2304.05974
-
Asynchronous Event-Triggered Control for Non-Linear Systems 25 Nov 2022 · 0 repositories · arXiv:2211.13846
-
Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition 11 Nov 2022 · 1 repository · arXiv:2211.06173
-
Guided contrastive self-supervised pre-training for automatic speech recognition 22 Oct 2022 · 0 repositories · arXiv:2210.12335
-
LittleBird: Efficient Faster & Longer Transformer for Question Answering 21 Oct 2022 · 0 repositories · arXiv:2210.11870
-
Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding 8 Oct 2022 · 0 repositories · arXiv:2210.03915
-
Analysis of Self-Supervised Learning and Dimensionality Reduction Methods in Clustering-Based Active Learning for Speech Emotion Recognition 21 Jun 2022 · 1 repository · arXiv:2206.10188
-
Variable-rate hierarchical CPC leads to acoustic unit discovery in speech 5 Jun 2022 · 1 repository · arXiv:2206.02211Syntology official (archive's flag): 3 ran · 3 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)
-
Do self-supervised speech models develop human-like perception biases? 31 May 2022 · 0 repositories · arXiv:2205.15819
-
SOM-CPC: Unsupervised Contrastive Learning with Self-Organizing Maps for Structured Representations of High-Rate Time Series 31 May 2022 · 1 repository · arXiv:2205.15875
-
Embed to Control Partially Observed Systems: Representation Learning with Provable Sample Efficiency 26 May 2022 · 0 repositories · arXiv:2205.13476
-
Naive Few-Shot Learning: Uncovering the fluid intelligence of machines 24 May 2022 · 0 repositories · arXiv:2205.12013
-
Zero-Episode Few-Shot Contrastive Predictive Coding: Solving intelligence tests without prior training 4 May 2022 · 0 repositories · arXiv:2205.01924
-
Thompson Sampling for Bandit Learning in Matching Markets 26 Apr 2022 · 1 repository · arXiv:2204.12048
-
Probing phoneme, language and speaker information in unsupervised speech representations 30 Mar 2022 · 0 repositories · arXiv:2203.16193
-
Robust Speaker Recognition with Transformers Using wav2vec 2.0 28 Mar 2022 · 0 repositories · arXiv:2203.15095
-
ETCetera: beyond Event-Triggered Control 3 Mar 2022 · 0 repositories · arXiv:2203.01623
-
Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring 24 Feb 2022 · 3 repositories · arXiv:2202.11929Syntology official (archive's flag): 5 ran · 17 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 5 honoured, 1 violated, 5 with no contract checked; 6 where Syntology's instrument failed) · 2 unverified (of 19 harvested samples) · 1 pointer-only (licence)
-
Formal Analysis of the Sampling Behaviour of Stochastic Event-Triggered Control 21 Feb 2022 · 0 repositories · arXiv:2202.10178
-
VCVTS: Multi-speaker Video-to-Speech synthesis via cross-modal knowledge transfer from voice conversion 18 Feb 2022 · 0 repositories · arXiv:2202.09081
-
Contrastive predictive coding for Anomaly Detection in Multi-variate Time Series Data 8 Feb 2022 · 0 repositories · arXiv:2202.03639
-
Chaos and order in event-triggered control 12 Jan 2022 · 0 repositories · arXiv:2201.04462
-
Detecting Extratropical Cyclones of the Northern Hemisphere with Single Shot Detector 1 Dec 2021 · 0 repositories · arXiv:2112.01283
-
Non-Intrusive Binaural Speech Intelligibility Prediction from Discrete Latent Representations 24 Nov 2021 · 1 repository · arXiv:2111.12531
-
STC speaker recognition systems for the NIST SRE 2021 3 Nov 2021 · 0 repositories · arXiv:2111.02298
-
Contrastive prediction strategies for unsupervised segmentation and categorization of phonemes and words 29 Oct 2021 · 1 repository · arXiv:2110.15909Syntology official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
Bandits with Dynamic Arm-acquisition Costs 23 Oct 2021 · 0 repositories · arXiv:2110.12118
-
Towards High-fidelity Singing Voice Conversion with Acoustic Reference and Contrastive Predictive Coding 10 Oct 2021 · 0 repositories · arXiv:2110.04754
-
SCaLa: Supervised Contrastive Learning for End-to-End Speech Recognition 8 Oct 2021 · 0 repositories · arXiv:2110.04187
-
Unsupervised Speech Segmentation and Variable Rate Representation Learning using Segmental Contrastive Predictive Coding 5 Oct 2021 · 0 repositories · arXiv:2110.02345
-
Computing the average inter-sample time of event-triggered control using quantitative automata 29 Sep 2021 · 0 repositories · arXiv:2109.14391
-
Prototypical Contrastive Predictive Coding 29 Sep 2021 · 0 repositories
-
Memory Augmented Multi-Instance Contrastive Predictive Coding for Sequential Recommendation 1 Sep 2021 · 2 repositories · arXiv:2109.00368
-
Uncertainties and output feedback in rollout event-triggered control 20 Aug 2021 · 0 repositories · arXiv:2108.09125
-
Rollout event-triggered control: reconciling event- and time-triggered control 6 Aug 2021 · 0 repositories · arXiv:2108.02994
-
Debiasing Samples from Online Learning Using Bootstrap 31 Jul 2021 · 0 repositories · arXiv:2108.00236
-
Contrastive Predictive Coding for Anomaly Detection 16 Jul 2021 · 0 repositories · arXiv:2107.07820
-
Model Selection for Generic Contextual Bandits 7 Jul 2021 · 0 repositories · arXiv:2107.03455
-
Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation 3 Jun 2021 · 0 repositories · arXiv:2106.02170
-
Unsupervised Representation Learning from Pathology Images with Multi-directional Contrastive Predictive Coding 11 May 2021 · 1 repository · arXiv:2105.05345
-
Voice Conversion Based Speaker Normalization for Acoustic Unit Discovery 4 May 2021 · 0 repositories · arXiv:2105.01786
-
VQCPC-GAN: Variable-Length Adversarial Audio Synthesis Using Vector-Quantized Contrastive Predictive Coding 4 May 2021 · 1 repository · arXiv:2105.01531
-
The Zero Resource Speech Challenge 2021: Spoken language modelling 29 Apr 2021 · 0 repositories · arXiv:2104.14700
-
Aligned Contrastive Predictive Coding 24 Apr 2021 · 1 repository · arXiv:2104.11946Syntology official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
NoiseVC: Towards High Quality Zero-Shot Voice Conversion 13 Apr 2021 · 0 repositories · arXiv:2104.06074
-
Abstracting the Sampling Behaviour of Stochastic Linear Periodic Event-Triggered Control Systems 25 Mar 2021 · 0 repositories · arXiv:2103.13839
-
Contrastive Unsupervised Learning for Speech Emotion Recognition 12 Feb 2021 · 0 repositories · arXiv:2102.06357
-
The Wireless Control Bus: Enabling Efficient Multi-hop Event-Triggered Control with Concurrent Transmissions 26 Jan 2021 · 0 repositories · arXiv:2101.10961
-
Novelty Detection with Rotated Contrastive Predictive Coding 1 Jan 2021 · 0 repositories
-
RealFormer: Transformer Likes Residual Attention 21 Dec 2020 · 5 repositories · arXiv:2012.11747
-
A comparison of self-supervised speech representations as input features for unsupervised acoustic word embeddings 14 Dec 2020 · 0 repositories · arXiv:2012.07387
-
Contrastive Predictive Coding for Human Activity Recognition 9 Dec 2020 · 0 repositories · arXiv:2012.05333
-
Unsupervised Anomaly Detection From Semantic Similarity Scores 1 Dec 2020 · 0 repositories · arXiv:2012.00461
-
Joint Masked CPC and CTC Training for ASR 30 Oct 2020 · 1 repository · arXiv:2011.00093
-
Improving Perceptual Quality by Phone-Fortified Perceptual Loss using Wasserstein Distance for Speech Enhancement 28 Oct 2020 · 1 repository · arXiv:2010.15174Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Abstracting the Traffic of Nonlinear Event-Triggered Control Systems 23 Oct 2020 · 0 repositories · arXiv:2010.12341
-
Machine Learning in Event-Triggered Control: Recent Advances and Open Issues 27 Sep 2020 · 0 repositories · arXiv:2009.12783
-
Uncovering the structure of clinical EEG signals with self-supervised learning 31 Jul 2020 · 2 repositories · arXiv:2007.16104Syntology 0 ran · 3 unverified (of 3 harvested samples)
-
Multi-label Contrastive Predictive Coding 20 Jul 2020 · 0 repositories · arXiv:2007.09852
-
Analysis of Predictive Coding Models for Phonemic Representation Learning in Small Datasets 8 Jul 2020 · 0 repositories · arXiv:2007.04205
-
Embedding Task Knowledge into 3D Neural Networks via Self-supervised Learning 10 Jun 2020 · 0 repositories · arXiv:2006.05798
-
3D Self-Supervised Methods for Medical Imaging 6 Jun 2020 · 1 repository · arXiv:2006.03829Syntology official (archive's flag): 14 ran · 14 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 16 harvested samples) · 16 pointer-only (licence)
-
Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge 19 May 2020 · 2 repositories · arXiv:2005.09409
-
COBRA: Contrastive Bi-Modal Representation Algorithm 7 May 2020 · 1 repository · arXiv:2005.03687Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples)
-
Region-Based Self-Triggered Control for Perturbed and Uncertain Nonlinear Systems 1 May 2020 · 0 repositories · arXiv:2005.00473
Tasks archive 2025-07-28
20 shown of 137 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections