{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/highway-networks","slug":"highway-networks","name":"Highway networks","full_name":"Highway networks","full_name_withheld":false,"description_markdown":"There is plenty of theoretical and empirical evidence that depth of neural networks is a crucial ingredient for their success. However, network training becomes more difficult with increasing depth and training of very deep networks remains an open problem. In this extended abstract, we introduce a new architecture designed to ease gradient-based training of very deep networks. We refer to networks with this architecture as highway networks, since they allow unimpeded information flow across several layers on \"information highways\". The architecture is characterized by the use of gating units which learn to regulate the flow of information through a network. Highway networks with hundreds of layers can be trained directly using stochastic gradient descent and with a variety of activation functions, opening up the possibility of studying extremely deep and efficient architectures.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Highway Networks","paper":"/paper/highway-networks","first_author":"Rupesh Kumar Srivastava","n_authors":3,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/highway-networks"},"source":{"url":"http://arxiv.org/abs/1505.00387v2","title":"Highway Networks","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Attention Mechanisms","url":"/methods/category/attention-mechanisms","pwc_aliases":["attention-mechanisms-1"]}],"n_papers_tagged":24,"archive_num_papers":24,"papers_newest_first":[{"paper":null,"slug":"to-stay-or-to-bypass-unraveling-mainline","title":"To Stay or to Bypass: Unraveling Mainline Vehicles' Aggregate Strategic Decision-Making at Highway Weaving Ramps","date":"2025-05-13","arxiv_id":"2505.08965","n_code_links":0,"syntology":null},{"paper":"/paper/gated-attention-for-large-language-models-non","slug":"gated-attention-for-large-language-models-non","title":"Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free","date":"2025-05-10","arxiv_id":"2505.06708","n_code_links":1,"syntology":{"ran":3,"of":3,"n_ran_checked":0,"n_instrument":3,"unverified":0,"pointer_only":0,"phrase":"3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified","official":{"repos":["qiuzh20/gated_attention"],"state":"official (archive's flag): 3 ran","n_ran":3,"n_constructed":0,"n_ran_no_instrument_failure":0,"n_unverified":0,"ran_from_kinds":["official"]}}},{"paper":null,"slug":"deconstructing-recurrence-attention-and","title":"Deconstructing Recurrence, Attention, and Gating: Investigating the transferability of Transformers and Gated Recurrent Neural Networks in forecasting of dynamical systems","date":"2024-10-03","arxiv_id":"2410.02654","n_code_links":0,"syntology":null},{"paper":"/paper/highway-networks-for-improved-surface","slug":"highway-networks-for-improved-surface","title":"Highway Networks for Improved Surface Reconstruction: The Role of Residuals and Weight Updates","date":"2024-07-11","arxiv_id":"2407.08134","n_code_links":1,"syntology":null},{"paper":null,"slug":"computer-vision-based-model-for-detecting","title":"Computer vision-based model for detecting turning lane features on Florida's public roadways","date":"2024-06-13","arxiv_id":"2406.08822","n_code_links":0,"syntology":null},{"paper":null,"slug":"deep-learning-based-vehicle-speed-prediction","title":"Deep Learning-Based Vehicle Speed Prediction for Ecological Adaptive Cruise Control in Urban and Highway Scenarios","date":"2022-11-30","arxiv_id":"2212.00149","n_code_links":0,"syntology":null},{"paper":null,"slug":"multi-task-recommendation-system-for","title":"Multi-task recommendation system for scientific papers with high-way networks","date":"2022-04-21","arxiv_id":"2204.09930","n_code_links":0,"syntology":null},{"paper":"/paper/transfer-learning-with-graph-neural-networks","slug":"transfer-learning-with-graph-neural-networks","title":"Transfer Learning with Graph Neural Networks for Short-Term Highway Traffic Forecasting","date":"2020-04-17","arxiv_id":"2004.08038","n_code_links":2,"syntology":null},{"paper":"/paper/graph-partitioning-based-diffusion","slug":"graph-partitioning-based-diffusion","title":"Graph-Partitioning-Based Diffusion Convolutional Recurrent Neural Network for Large-Scale Traffic Forecasting","date":"2019-09-24","arxiv_id":"1909.11197","n_code_links":2,"syntology":null},{"paper":"/paper/investigating-the-effect-of-residual-and","slug":"investigating-the-effect-of-residual-and","title":"Investigating the effect of residual and highway connections in speech enhancement models","date":"2018-10-22","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/batch-normalized-recurrent-highway-networks","slug":"batch-normalized-recurrent-highway-networks","title":"Batch-normalized Recurrent Highway Networks","date":"2018-09-26","arxiv_id":"1809.10271","n_code_links":1,"syntology":null},{"paper":null,"slug":"semi-tied-units-for-efficient-gating-in-lstm","title":"Semi-tied Units for Efficient Gating in LSTM and Highway Networks","date":"2018-06-18","arxiv_id":"1806.06513","n_code_links":0,"syntology":null},{"paper":null,"slug":"humor-recognition-using-deep-learning","title":"Humor Recognition Using Deep Learning","date":"2018-06-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"avoiding-degradation-in-deep-feed-forward","title":"Avoiding degradation in deep feed-forward networks by phasing out skip-connections","date":"2018-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"language-modeling-with-recurrent-highway","title":"Language Modeling with Recurrent Highway Hypernetworks","date":"2017-12-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"slug":"exploiting-nontrivial-connectivity-for","title":"Exploiting Nontrivial Connectivity for Automatic Speech Recognition","date":"2017-11-28","arxiv_id":"1711.10271","n_code_links":0,"syntology":null},{"paper":"/paper/lattice-recurrent-unit-improving-convergence","slug":"lattice-recurrent-unit-improving-convergence","title":"Lattice Recurrent Unit: Improving Convergence and Statistical Efficiency for Sequence Modeling","date":"2017-10-06","arxiv_id":"1710.02254","n_code_links":3,"syntology":null},{"paper":"/paper/application-of-a-hybrid-bi-lstm-crf-model-to","slug":"application-of-a-hybrid-bi-lstm-crf-model-to","title":"Application of a Hybrid Bi-LSTM-CRF model to the task of Russian Named Entity Recognition","date":"2017-09-27","arxiv_id":"1709.09686","n_code_links":2,"syntology":null},{"paper":null,"slug":"language-modeling-with-highway-lstm","title":"Language Modeling with Highway LSTM","date":"2017-09-19","arxiv_id":"1709.06436","n_code_links":0,"syntology":null},{"paper":null,"slug":"early-improving-recurrent-elastic-highway","title":"Early Improving Recurrent Elastic Highway Network","date":"2017-08-14","arxiv_id":"1708.04116","n_code_links":0,"syntology":null},{"paper":null,"slug":"faster-training-of-very-deep-networks-via-p","title":"Faster Training of Very Deep Networks Via p-Norm Gates","date":"2016-08-11","arxiv_id":"1608.03639","n_code_links":0,"syntology":null},{"paper":null,"slug":"choice-by-elimination-via-deep-neural","title":"Choice by Elimination via Deep Neural Networks","date":"2016-02-17","arxiv_id":"1602.05285","n_code_links":0,"syntology":null},{"paper":"/paper/training-very-deep-networks","slug":"training-very-deep-networks","title":"Training Very Deep Networks","date":"2015-07-22","arxiv_id":"1507.06228","n_code_links":3,"syntology":null},{"paper":"/paper/highway-networks","slug":"highway-networks","title":"Highway Networks","date":"2015-05-03","arxiv_id":"1505.00387","n_code_links":4,"syntology":null}],"papers_shown":24,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":24,"with_a_code_link":10,"where_syntology_ran_a_sample":1,"not_listed_spam_title":0,"listed":24,"listed_where_code_ran":1,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":0,"every_run_a_failure_of_syntologys_instrument":1,"listed_with_a_run_with_no_instrument_failure":0,"listed_every_run_a_failure_of_syntologys_instrument":1,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":1,"papers_where_code_ran_first_page":"/method/highway-networks/papers/ran/1","papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[{"order":"ran","page":1,"of":1,"url":"/method/highway-networks/papers/ran/1","json":"/method/highway-networks/papers/ran/1.json","rows":1}],"tasks":[{"task":"/task/language-modelling","name":"Language Modelling","papers":4},{"task":"/task/language-modeling","name":"Language Modeling","papers":3},{"task":"/task/speech-recognition","name":"Speech Recognition","papers":3},{"task":"/task/speech-recognition-1","name":"speech-recognition","papers":3},{"task":"/task/automatic-speech-recognition-2","name":"Automatic Speech Recognition","papers":2},{"task":"/task/automatic-speech-recognition","name":"Automatic Speech Recognition (ASR)","papers":2},{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/management","name":"Management","papers":2},{"task":"/task/time-series","name":"Time Series Analysis","papers":2},{"task":"/task/activity-recognition","name":"Activity Recognition","papers":1},{"task":"/task/attribute","name":"Attribute","papers":1},{"task":"/task/decision-making","name":"Decision Making","papers":1},{"task":"/task/deep-learning","name":"Deep Learning","papers":1},{"task":"/task/denoising","name":"Denoising","papers":1},{"task":"/task/classification","name":"General Classification","papers":1},{"task":"/task/graph-neural-network","name":"Graph Neural Network","papers":1},{"task":"/task/human-activity-recognition","name":"Human Activity Recognition","papers":1},{"task":"/task/image-captioning","name":"Image Captioning","papers":1},{"task":"/task/learning-to-rank","name":"Learning-To-Rank","papers":1},{"task":"/task/machine-translation","name":"Machine Translation","papers":1}],"tasks_shown":20,"n_tasks":36,"usage_by_year":[{"year":"2015","papers":2},{"year":"2016","papers":2},{"year":"2017","papers":6},{"year":"2018","papers":5},{"year":"2019","papers":1},{"year":"2020","papers":1},{"year":"2022","papers":2},{"year":"2024","papers":3},{"year":"2025","papers":2}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/highway-networks"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=Highway%20networks","read_at":"2026-10-05T00:57:34+00:00","query":"Highway networks","kind":"paper","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"arxiv_id":"1505.00387"},"resolution":"paper","resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"same_paper","archive_paper_arxiv_id":"1505.00387","repositories":{"listed":4,"in_graph":4,"list_truncated":false,"strongest_link":"mentioned_in_github"},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"1505.00387","citations":null},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'Highway networks'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[],"code_nearby_caption":null,"origin_paper":{"arxiv_id":"1505.00387","page":"/paper/highway-networks","syntology_url":"https://syntology.ai/paper/1505.00387","in_graph_layer":false,"samples":0},"line":"repository known, no verified implementation (declared_code; repositories: 4 listed of 4 in the graph, strongest link: mentioned on GitHub). The lookup matched a paper by its title, arXiv:1505.00387, not a method entry. This answer is about arXiv:1505.00387, the paper the archive credits with this method.","level_legend":null},"syntology_have_state":"loaded","record_sha256":"d7c94004e04654773ec7682555f0113053c941bca088849f0c655ed055e2cd86","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}