{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/frage-frequency-agnostic-word-representation","title":"FRAGE: Frequency-Agnostic Word Representation","arxiv_id":"1809.06858","date":"2018-09-18","proceeding":"NeurIPS 2018 12","authors":["Chengyue Gong","Di He","Xu Tan","Tao Qin","Li-Wei Wang","Tie-Yan Liu"],"abstract":"Continuous word representation (aka word embedding) is a basic building block in many neural network-based models used in natural language processing tasks. Although it is widely accepted that words with similar semantics should be close to each other in the embedding space, we find that word embeddings learned in several tasks are biased towards word frequency: the embeddings of high-frequency and low-frequency words lie in different subregions of the embedding space, and the embedding of a rare word and a popular word can be far from each other even if they are semantically similar. This makes learned word embeddings ineffective, especially for rare words, and consequently limits the performance of these neural network models. In this paper, we develop a neat, simple yet effective way to learn \\emph{FRequency-AGnostic word Embedding} (FRAGE) using adversarial training. We conducted comprehensive studies on ten datasets across four natural language processing tasks, including word similarity, language modeling, machine translation and text classification. Results show that with FRAGE, we achieve higher performance than the baselines in all tasks.","url_abs":"https://arxiv.org/abs/1809.06858v2","url_pdf":"https://arxiv.org/pdf/1809.06858v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"frage-frequency-agnostic-word-representation","repo_url":"https://github.com/ChengyueGongR/FrequencyAgnostic","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"frage-frequency-agnostic-word-representation","repo_url":"https://github.com/JakubStefko/w2vf","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"},{"task_slug":"word-similarity","task_name":"Word Similarity"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/language-modelling-on-penn-treebank-word","task":"Language Modelling","dataset":"Penn Treebank (Word Level)","model":"FRAGE + AWD-LSTM-MoS + dynamic eval","rank_in_archive_order":7,"of":43,"metrics":{"Params":"22M","Test perplexity":"46.54","Validation perplexity":"47.38"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-wikitext-2","task":"Language Modelling","dataset":"WikiText-2","model":"FRAGE + AWD-LSTM-MoS + dynamic eval","rank_in_archive_order":13,"of":38,"metrics":{"Number of params":"35M","Test perplexity":"39.14","Validation perplexity":"40.85"},"uses_additional_data":false},{"leaderboard":"/sota/machine-translation-on-iwslt2015-german","task":"Machine Translation","dataset":"IWSLT2015 German-English","model":"Transformer with FRAGE","rank_in_archive_order":3,"of":15,"metrics":{"BLEU score":"33.97"},"uses_additional_data":false},{"leaderboard":"/sota/machine-translation-on-wmt2014-english-german","task":"Machine Translation","dataset":"WMT2014 English-German","model":"Transformer Big with FRAGE","rank_in_archive_order":31,"of":91,"metrics":{"BLEU score":"29.11"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1809.06858","atlas_url":"https://app.syntology.ai/?focus=1809.06858","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1809.06858"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1809.06858\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1809.06858"},"call":"get_concepts_for_paper(arxiv_id=\"1809.06858\")"}],"by_repo":[{"repo":"ChengyueGongR/FrequencyAgnostic","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"JakubStefko/w2vf","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"0a74bb15c7e15269df0e4c80836becea002dd2012d1f406cd70255b67b0937bd","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}