{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/dawn-of-the-transformer-era-in-speech-emotion","title":"Dawn of the transformer era in speech emotion recognition: closing the valence gap","arxiv_id":"2203.07378","date":"2022-03-14","proceeding":null,"authors":["Johannes Wagner","Andreas Triantafyllopoulos","Hagen Wierstorf","Maximilian Schmitt","Felix Burkhardt","Florian Eyben","Björn W. Schuller"],"abstract":"Recent advances in transformer-based architectures which are pre-trained in self-supervised manner have shown great promise in several machine learning tasks. In the audio domain, such architectures have also been successfully utilised in the field of speech emotion recognition (SER). However, existing works have not evaluated the influence of model size and pre-training data on downstream performance, and have shown limited attention to generalisation, robustness, fairness, and efficiency. The present contribution conducts a thorough analysis of these aspects on several pre-trained variants of wav2vec 2.0 and HuBERT that we fine-tuned on the dimensions arousal, dominance, and valence of MSP-Podcast, while additionally using IEMOCAP and MOSI to test cross-corpus generalisation. To the best of our knowledge, we obtain the top performance for valence prediction without use of explicit linguistic information, with a concordance correlation coefficient (CCC) of .638 on MSP-Podcast. Furthermore, our investigations reveal that transformer-based architectures are more robust to small perturbations compared to a CNN-based baseline and fair with respect to biological sex groups, but not towards individual speakers. Finally, we are the first to show that their extraordinary success on valence is based on implicit linguistic information learnt during fine-tuning of the transformer layers, which explains why they perform on-par with recent multimodal approaches that explicitly utilise textual information. Our findings collectively paint the following picture: transformer-based architectures constitute the new state-of-the-art in SER, but further advances are needed to mitigate remaining robustness and individual speaker issues. To make our findings reproducible, we release the best performing model to the community.","url_abs":"https://arxiv.org/abs/2203.07378v4","url_pdf":"https://arxiv.org/pdf/2203.07378v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dawn-of-the-transformer-era-in-speech-emotion","repo_url":"https://github.com/audeering/w2v2-how-to","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"cross-corpus","task_name":"Cross-corpus"},{"task_slug":"emotion-recognition","task_name":"Emotion Recognition"},{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"speech-emotion-recognition","task_name":"Speech Emotion Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/emotion-recognition-on-msp-podcast","task":"Emotion Recognition","dataset":"MSP-Podcast","model":"w2v2-L-robust-12","rank_in_archive_order":1,"of":1,"metrics":{"Concordance correlation coefficient (CCC)":"0.638"},"uses_additional_data":false},{"leaderboard":"/sota/speech-emotion-recognition-on-msp-podcast-1","task":"Speech Emotion Recognition","dataset":"MSP-Podcast (Activation)","model":"w2v2-L-robust-12","rank_in_archive_order":3,"of":4,"metrics":{"CCC":"0.745"},"uses_additional_data":false},{"leaderboard":"/sota/speech-emotion-recognition-on-msp-podcast-2","task":"Speech Emotion Recognition","dataset":"MSP-Podcast (Dominance)","model":"w2v2-L-robust-12","rank_in_archive_order":3,"of":4,"metrics":{"CCC":"0.655"},"uses_additional_data":false},{"leaderboard":"/sota/speech-emotion-recognition-on-msp-podcast","task":"Speech Emotion Recognition","dataset":"MSP-Podcast (Valence)","model":"w2v2-L-robust-12","rank_in_archive_order":3,"of":4,"metrics":{"CCC":"0.638"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2203.07378","atlas_url":"https://app.syntology.ai/?focus=2203.07378","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2203.07378"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2203.07378\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2203.07378"},"call":"get_concepts_for_paper(arxiv_id=\"2203.07378\")"}],"by_repo":[{"repo":"audeering/w2v2-how-to","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"641fe7767b872cba24403fcecbfd099dc30618a109aed2776ab41e72c8dc82ff","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}