Methods › General › Attention Mechanisms › Location Sensitive Attention

Location Sensitive Attention

25 papers tagged archive 2025-07-28

Introduced by Jan Chorowski et al. in Attention-Based Models for Speech Recognition

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

Location Sensitive Attention is an attention mechanism that extends the additive attention mechanism to use cumulative attention weights from previous decoder time steps as an additional feature. This encourages the model to move forward consistently through the input, mitigating potential failure modes where some subsequences are repeated or ignored by the decoder.

Starting with additive attention where h is a sequential representation from a BiRNN encoder and sᵢ₋₁ is the (i − 1)-th state of a recurrent neural network (e.g. a LSTM or GRU):

e_(i, j) = wᵀtanh(Wsᵢ₋₁ + Vhⱼ + b)

where w and b are vectors, W and V are matrices. We extend this to be location-aware by making it take into account the alignment produced at the previous step. First, we extract k vectors f_(i,j) ∈ℝᵏ for every position j of the previous alignment αᵢ₋₁ by convolving it with a matrix F ∈R^(k×r):

fᵢ = F ∗ αᵢ₋₁

These additional vectors f_(i,j) are then used by the scoring mechanism e_(i,j):

e_(i,j) = wᵀtanh(Wsᵢ₋₁ + Vhⱼ + Uf_(i,j) + b)

PaperSourceSee Code · begeekmyfriend/tacotron2

In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.

Near matches the lookup lists, each a different name: Depth-Sensitive Attention; Monotonic Location Attention; Budgeted Attention Allocation; Scale and Location Sensitive (SLS) loss; VC-Attention; TF-ATTENTION; SS-Attention; TC-ATTENTION.

Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.

Papers archive 2025-07-28

archive papers tagged: 25 · with a code link: 14 · where Syntology ran a sample: 4 (3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology

Show: all tagged papersonly where code ran (4 of 25 tagged: 3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 25 of 25, newest first by the archive's date (ties by slug), in archive order.

Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.

Tasks archive 2025-07-28

20 shown of 29 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

TaskPapers
Text to Speech16
text-to-speech16
Speech Synthesis14
Text-To-Speech Synthesis4
Decoder3
Speech Recognition2
Style Transfer2
Voice Cloning2
Acoustic Modelling1
All1
Audio Synthesis1
CPU1
Classification1
Disentanglement1
Form1
GPU1
General Classification1
Machine Translation1
Meta-Learning1
Phoneme Recognition1

Usage over time archive 2025-07-28

Papers per year tagged with Location Sensitive Attention: 2015 to 2024, peak 8 8 0 2015: 1 paper 2015 2016: 0 papers 2016 2017: 1 paper 2017 2018: 0 papers 2018 2019: 2 papers 2019 2020: 8 papers 2020 2021: 6 papers 2021 2022: 2 papers 2022 2023: 4 papers 2023 2024: 1 paper 2024
Papers per year the archive tags with this method, by the paper's archive date (25 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Attention Mechanisms

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections