{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/a-new-deep-learning-architecture","title":"A New Deep Learning Architecture withInductive Bias Balance for Transformer Oil Temperature Forecasting","arxiv_id":null,"date":"2023-05-28","proceeding":"Journal of Big Data 2023 5","authors":["Manuel Jesús Jiménez-Navarro","María Martínez-Ballesteros","Francisco Martínez-Álvarez & Gualberto Asencio-Cortés"],"abstract":"Ensuring optimal performance of power transformers is a laborious task, where the insulation system is essential to decrease their deterioration. The insulation system uses the insulate oil required to control temperature. High temperatures may reduce the lifetime of transformers, leading to expensive maintenance. Deep learning architectures have been shown to obtain remarkable results in a wide range of fields. However, this improvement usually comes with an increase in computing resources, which increases the carbon footprint and hinders the optimization of the architectures. In this work, we develop a new deep learning architecture that obtain an efficacy which compete with the best current architectures in transformer oil temperature forecasting while improve the efficacy. Effective forecasting can help prevent high temperatures and monitor the future condition of power transformers, avoiding unnecessary waste. We attempt to balance the inductive bias included in our architecture through the proposed Smooth Residual Block. This mechanism divides the original problem into multiple subproblems, obtaining different representations of the time series, which collaboratively obtain the final forecasting. Our architecture is applied to the Electricity Transformer datasets, which obtain transformer insulate oil temperature measures from two transformers in China. The results achieve a 13% improvement in MSE and a 57% improvement in performance compared to, as far as we know, the best current architectures. Additionally, we analyze the behavior learned by this architecture to obtain an intuitive interpretation of the achieved solution.","url_abs":"https://journalofbigdata.springeropen.com/articles/10.1186/s40537-023-00745-0","url_pdf":"https://journalofbigdata.springeropen.com/counter/pdf/10.1186/s40537-023-00745-0.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-new-deep-learning-architecture","repo_url":"https://github.com/manjimnav/SRCNet","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series-forecasting","task_name":"Time Series Forecasting"},{"task_slug":"univariate-time-series-forecasting","task_name":"Univariate Time Series Forecasting"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/time-series-forecasting-on-etth1-24-2","task":"Time Series Forecasting","dataset":"ETTh1 (24) Univariate","model":"SRCNet","rank_in_archive_order":5,"of":5,"metrics":{"MAE":"0.129"},"uses_additional_data":false},{"leaderboard":"/sota/time-series-forecasting-on-etth2-24-2","task":"Time Series Forecasting","dataset":"ETTh2 (24) Univariate","model":"SRCNet","rank_in_archive_order":5,"of":5,"metrics":{"MAE":"0.193"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"no_arxiv_id","checked_against":null,"by_repo":[{"repo":"manjimnav/SRCNet","kind":"named_in_paper","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"99be9b156802901e35e2468460ddc360eedb2167635a0081f298cfed3b33624a","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}