Papers › Enhanced Transformer architecture for in-context learning of dynamical systems

Enhanced Transformer architecture for in-context learning of dynamical systems

4 Oct 2024arXiv:2410.03291archive 2025-07-28

Matteo Rufolo, Dario Piga, Gabriele Maroni, Marco Forgione

Recently introduced by some of the authors, the in-context identification paradigm aims at estimating, offline and based on synthetic data, a meta-model that describes the behavior of a whole class of systems. Once trained, this meta-model is fed with an observed input/output sequence (context) generated by a real system to predict its behavior in a zero-shot learning fashion. In this paper, we enhance the original meta-modeling framework through three key innovations: by formulating the learning task within a probabilistic framework; by managing non-contiguous context and query windows; and by adopting recurrent patching to effectively handle long context sequences. The efficacy of these modifications is demonstrated through a numerical example focusing on the Wiener-Hammerstein system class, highlighting the model's enhanced performance and scalability.

PaperPDFCode

Syntology This paper's arXiv id is not in Syntology's Atlas id list (549,449 ids, as this build read them; the read time is in the build record), so this page links to no Syntology page for it. The list is not the whole graph; Syntology's feed shows papers as they enter the graph.

Code

mattrufolo/sysid-prob-transformer officialmentioned in paperpytorchSyntology: not harvested report
mattrufolo/sysid-robust-transformer mentioned on GitHubpytorchSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

In-Context LearningZero-Shot Learning

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Patching

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections