Papers › FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series...

FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification

20 Feb 2023arXiv:2302.09818archive 2025-07-28

Mingyue Cheng, Qi Liu, Zhiding Liu, Zhi Li, Yucong Luo, Enhong Chen

Deep learning-based algorithms, e.g., convolutional networks, have significantly facilitated multivariate time series classification (MTSC) task. Nevertheless, they suffer from the limitation in modeling long-range dependence due to the nature of convolution operations. Recent advancements have shown the potential of transformers to capture long-range dependence. However, it would incur severe issues, such as fixed scale representations, temporal-invariant and quadratic time complexity, with transformers directly applicable to the MTSC task because of the distinct properties of time series data. To tackle these issues, we propose FormerTime, an hierarchical representation model for improving the classification capacity for the MTSC task. In the proposed FormerTime, we employ a hierarchical network architecture to perform multi-scale feature maps. Besides, a novel transformer encoder is further designed, in which an efficient temporal reduction attention layer and a well-informed contextual positional encoding generating strategy are developed. To sum up, FormerTime exhibits three aspects of merits: (1) learning hierarchical multi-scale representations from time series data, (2) inheriting the strength of both transformers and convolutional networks, and (3) tacking the efficiency challenges incurred by the self-attention mechanism. Extensive experiments performed on 10 publicly available datasets from UEA archive verify the superiorities of the FormerTime compared to previous competitive baselines.

PaperPDFCode

Syntology This paper's arXiv id is not in Syntology's Atlas id list (549,449 ids, as this build read them; the read time is in the build record), so this page links to no Syntology page for it. The list is not the whole graph; Syntology's feed shows papers as they enter the graph.

Code

mingyue-cheng/formertime officialmentioned in paperpytorchSyntology: not harvested report
mingyue-cheng/convtimenet mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to ConvTimeNet: A Deep Hierarchical Fully Convolutional Model for Multivariate Time... (arXiv:2403.01493; that paper's own run record: 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report
mingyue-cheng/timemae mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Diffusion Auto-regressive Transformer for Effective Self-supervised Time Series Forecasting (arXiv:2410.05711; that paper's own run record: 27 ran (of which 8 constructed an object rather than computing a result; 23 with no instrument failure: 1 honoured, 2 violated, 20 with no contract checked; 4 where Syntology's instrument failed) · 4 unverified) and Self-Supervised Dynamical System Representations for Physiological Time-Series (arXiv:2512.00239; that paper's own run record: 18 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 2 violated, 8 with no contract checked; 8 where Syntology's instrument failed) · 9 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Time SeriesTime Series AnalysisTime Series Classification

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Convolution

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections