Papers › TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders

TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders

1 Mar 2023arXiv:2303.00320archive 2025-07-28

Mingyue Cheng, Qi Liu, Zhiding Liu, Hao Zhang, Rujiao Zhang, Enhong Chen

Enhancing the expressive capacity of deep learning-based time series models with self-supervised pre-training has become ever-increasingly prevalent in time series classification. Even though numerous efforts have been devoted to developing self-supervised models for time series data, we argue that the current methods are not sufficient to learn optimal time series representations due to solely unidirectional encoding over sparse point-wise input units. In this work, we propose TimeMAE, a novel self-supervised paradigm for learning transferrable time series representations based on transformer networks. The distinct characteristics of the TimeMAE lie in processing each time series into a sequence of non-overlapping sub-series via window-slicing partitioning, followed by random masking strategies over the semantic units of localized sub-series. Such a simple yet effective setting can help us achieve the goal of killing three birds with one stone, i.e., (1) learning enriched contextual representations of time series with a bidirectional encoding scheme; (2) increasing the information density of basic semantic units; (3) efficiently encoding representations of time series using transformer networks. Nevertheless, it is a non-trivial to perform reconstructing task over such a novel formulated modeling paradigm. To solve the discrepancy issue incurred by newly injected masked embeddings, we design a decoupled autoencoder architecture, which learns the representations of visible (unmasked) positions and masked ones with two different encoder modules, respectively. Furthermore, we construct two types of informative targets to accomplish the corresponding pretext tasks. One is to create a tokenizer module that assigns a codeword to each masked region, allowing the masked codeword classification (MCC) task to be completed effectively...

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

mingyue-cheng/timemae officialmentioned in paperpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Diffusion Auto-regressive Transformer for Effective Self-supervised Time Series Forecasting (arXiv:2410.05711; that paper's own run record: 27 ran (of which 8 constructed an object rather than computing a result; 23 with no instrument failure: 1 honoured, 2 violated, 20 with no contract checked; 4 where Syntology's instrument failed) · 4 unverified) and Self-Supervised Dynamical System Representations for Physiological Time-Series (arXiv:2512.00239; that paper's own run record: 18 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 2 violated, 8 with no contract checked; 8 where Syntology's instrument failed) · 9 unverified); nothing checks that they implement this paper's method. report
RomGai/BrainVis mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to PreScience: A Dataset and Benchmark for Scientific Forecasting (arXiv:2602.20459; that paper's own run record: 22 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 4 honoured, 7 violated, 2 with no contract checked; 9 where Syntology's instrument failed) · 7 unverified); nothing checks that they implement this paper's method. report
mingyue-cheng/convtimenet mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to ConvTimeNet: A Deep Hierarchical Fully Convolutional Model for Multivariate Time... (arXiv:2403.01493; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
mingyue-cheng/crosstimenet mentioned on GitHubpytorchSyntology: not harvested report
mingyue-cheng/formertime mentioned on GitHubpytorchSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Time SeriesTime Series AnalysisTime Series Classification

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections