{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/integrating-attention-feedback-into-the","title":"Integrating Attention Feedback into the Recurrent Neural Network","arxiv_id":null,"date":"2021-09-29","proceeding":null,"authors":["Heng Li"],"abstract":"In this paper, an improved long short-term memory (LSTM) structure, called hidden attention long short-term memory (HA-LSTM), is proposed to reduce the long-term memory loss when updating the LSTM neural network at each time step. The HA-LSTM structure is different from the standard LSTM structure because a scaled dot-product attention-based sliding controller is introduced to the LSTM structure. The design of the sliding attention controller provides the traditional attention mechanism with a time-varying property, which makes the attention mechanism more suitable for time-series analysis tasks. Traditionally, the inputs to the attention mechanism are the hidden state vectors at all time steps. In contrast, the length of the inputs to the sliding attention controller is range-limited, which means it uses less memory than the traditional attention mechanism. In addition, the HA-LSTM structure integrates the attention mechanism into the standard LSTM structure, which provides the structure with advantages over both the standard LSTM structure and the attention mechanism. Different from most works, which perform unilateral computations on the attention mechanism after collecting the hidden state vectors at all time steps, the information of the gate vectors and the cell state vector of the HA-LSTM structure are based on feedback from the attention mechanism. In other words, the HA-LSTM structure's feedback property helps the cell state vector retain valuable information. To evaluate the performance of the HA-LSTM structure, four text benchmark datasets are used in experiments for text classification tasks. The model presented here is compared with two classic models and with a state-of-the-art model presented in recent years. Most outcomes indicate that the HA-LSTM structure is superior to the other structures.","url_abs":"https://openreview.net/forum?id=JLxrtqUaVe","url_pdf":"https://openreview.net/pdf?id=JLxrtqUaVe","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"integrating-attention-feedback-into-the","repo_url":"https://github.com/zz2232/ha-lstm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"time-series","task_name":"Time Series Analysis"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"no_arxiv_id","checked_against":null,"by_repo":[{"repo":"zz2232/ha-lstm","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"20bbec2acda30573eb0d32a73bdbd6a57653923c89fa6904097ef5ce2ed956ef","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}