{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/muld-the-multitask-long-document-benchmark","title":"MuLD: The Multitask Long Document Benchmark","arxiv_id":"2202.07362","date":"2022-02-15","proceeding":"LREC 2022 6","authors":["G Thomas Hudson","Noura Al Moubayed"],"abstract":"The impressive progress in NLP techniques has been driven by the development of multi-task benchmarks such as GLUE and SuperGLUE. While these benchmarks focus on tasks for one or two input sentences, there has been exciting work in designing efficient techniques for processing much longer inputs. In this paper, we present MuLD: a new long document benchmark consisting of only documents over 10,000 tokens. By modifying existing NLP tasks, we create a diverse benchmark which requires models to successfully model long-term dependencies in the text. We evaluate how existing models perform, and find that our benchmark is much more challenging than their `short document' equivalents. Furthermore, by evaluating both regular and efficient transformers, we show that models with increased context length are better able to solve the tasks presented, suggesting that future improvements in these models are vital for solving similar long document problems. We release the data and code for baselines to encourage further research on efficient NLP models.","url_abs":"https://arxiv.org/abs/2202.07362v1","url_pdf":"https://arxiv.org/pdf/2202.07362v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"muld-the-multitask-long-document-benchmark","repo_url":"https://github.com/ghomashudson/muld","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"style-change-detection","task_name":"Style change detection"},{"task_slug":"summarization","task_name":"Summarization"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[{"slug":"muld","name":"MuLD","full_name":"Multitask Long Document Benchmark"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/question-answering-on-muld-hotpotqa","task":"Question Answering","dataset":"MuLD (HotpotQA)","model":"Longformer","rank_in_archive_order":1,"of":2,"metrics":{"BLEU-1":"30.38","BLEU-4":"16.76","METEOR":"4.98","Rouge-L":"30.49"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-muld-hotpotqa","task":"Question Answering","dataset":"MuLD (HotpotQA)","model":"T5","rank_in_archive_order":2,"of":2,"metrics":{"BLEU-1":"28.11","BLEU-4":"13.63","METEOR":"4.46","Rouge-L":"27.61"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-muld-narrativeqa","task":"Question Answering","dataset":"MuLD (NarrativeQA)","model":"Longformer","rank_in_archive_order":1,"of":2,"metrics":{"BLEU-1":"19.84","BLEU-4":"62","METEOR":"4.52","Rouge-L":"22.09"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-muld-narrativeqa","task":"Question Answering","dataset":"MuLD (NarrativeQA)","model":"T5","rank_in_archive_order":2,"of":2,"metrics":{"BLEU-1":"17.67","BLEU-4":"55","METEOR":"3.36","Rouge-L":"19.03"},"uses_additional_data":false},{"leaderboard":"/sota/summarization-on-muld-vlsp","task":"Summarization","dataset":"MuLD (VLSP)","model":"Longformer","rank_in_archive_order":1,"of":2,"metrics":{"BLEU-1":"46.74","BLEU-4":"3.05","METEOR":"9.58","Rouge-L":"19.52"},"uses_additional_data":false},{"leaderboard":"/sota/summarization-on-muld-vlsp","task":"Summarization","dataset":"MuLD (VLSP)","model":"T5","rank_in_archive_order":2,"of":2,"metrics":{"BLEU-1":"28.85","BLEU-4":"84","METEOR":"7.98","Rouge-L":"16.55"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-muld-character-type","task":"Text Classification","dataset":"MuLD (Character Type)","model":"Longformer","rank_in_archive_order":1,"of":2,"metrics":{"F1":"82.58"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-muld-character-type","task":"Text Classification","dataset":"MuLD (Character Type)","model":"T5","rank_in_archive_order":2,"of":2,"metrics":{"F1":"54.01"},"uses_additional_data":false},{"leaderboard":"/sota/translation-on-muld-opensubtitles","task":"Translation","dataset":"MuLD (OpenSubtitles)","model":"T5","rank_in_archive_order":1,"of":2,"metrics":{"BLEU-1":"34.07","BLEU-4":"1.63","METEOR":"38.53","Rouge-L":"35.35"},"uses_additional_data":false},{"leaderboard":"/sota/translation-on-muld-opensubtitles","task":"Translation","dataset":"MuLD (OpenSubtitles)","model":"Longformer","rank_in_archive_order":2,"of":2,"metrics":{"BLEU-1":"22.74","BLEU-4":"20","METEOR":"22.95","Rouge-L":"22.17"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2202.07362","atlas_url":"https://app.syntology.ai/?focus=2202.07362","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2202.07362"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2202.07362\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2202.07362"},"call":"get_concepts_for_paper(arxiv_id=\"2202.07362\")"}],"by_repo":[{"repo":"ghomashudson/muld","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"48e10daf273a8550c26332f2d26fd3de52ea92271c6cfdfe81428229d9ab844f","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}