{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/on-path-to-multimodal-historical-reasoning","title":"On Path to Multimodal Historical Reasoning: HistBench and HistAgent","arxiv_id":"2505.20246","date":"2025-05-26","proceeding":null,"authors":["Jiahao Qiu","Fulian Xiao","Yimin Wang","Yuchen Mao","Yijia Chen","Xinzhe Juan","Siran Wang","Xuan Qi","Tongcheng Zhang","Zixin Yao","Jiacheng Guo","Yifu Lu","Charles Argon","Jundi Cui","Daixin Chen","Junran Zhou","Shuyao Zhou","Zhanpeng Zhou","Ling Yang","Shilong Liu","Hongru Wang","Kaixuan Huang","Xun Jiang","Yuming Cao","Yue Chen","Yunfei Chen","Zhengyi Chen","Ruowei Dai","Mengqiu Deng","Jiye Fu","Yunting Gu","Zijie Guan","Zirui Huang","Xiaoyan Ji","Yumeng Jiang","Delong Kong","Haolong Li","Jiaqi Li","Ruipeng Li","Tianze Li","Zhuoran Li","Haixia Lian","Mengyue Lin","Xudong Liu","Jiayi Lu","Jinghan Lu","Wanyu Luo","Ziyue Luo","Zihao Pu","Zhi Qiao","Ruihuan Ren","Liang Wan","Ruixiang Wang","Tianhui Wang","Yang Wang","Zeyu Wang","Zihua Wang","Yujia Wu","Zhaoyi Wu","Hao Xin","Weiao Xing","Ruojun Xiong","Weijie Xu","Yao Shu","Xiao Yao","Xiaorui Yang","Yuchen Yang","Nan Yi","Jiadong Yu","Yangyuxuan Yu","Huiting Zeng","Danni Zhang","Yunjie Zhang","Zhaoyu Zhang","Zhiheng Zhang","Xiaofeng Zheng","Peirong Zhou","Linyan Zhong","Xiaoyin Zong","Ying Zhao","Zhenxin Chen","Lin Ding","Xiaoyu Gao","Bingbing Gong","Yichao Li","Yang Liao","Guang Ma","Tianyuan Ma","Xinrui Sun","Tianyi Wang","Han Xia","Ruobing Xian","Gen Ye","Tengfei Yu","Wentao Zhang","Yuxi Wang","Xi Gao","Mengdi Wang"],"abstract":"Recent advances in large language models (LLMs) have led to remarkable progress across domains, yet their capabilities in the humanities, particularly history, remain underexplored. Historical reasoning poses unique challenges for AI, involving multimodal source interpretation, temporal inference, and cross-linguistic analysis. While general-purpose agents perform well on many existing benchmarks, they lack the domain-specific expertise required to engage with historical materials and questions. To address this gap, we introduce HistBench, a new benchmark of 414 high-quality questions designed to evaluate AI's capacity for historical reasoning and authored by more than 40 expert contributors. The tasks span a wide range of historical problems-from factual retrieval based on primary sources to interpretive analysis of manuscripts and images, to interdisciplinary challenges involving archaeology, linguistics, or cultural history. Furthermore, the benchmark dataset spans 29 ancient and modern languages and covers a wide range of historical periods and world regions. Finding the poor performance of LLMs and other agents on HistBench, we further present HistAgent, a history-specific agent equipped with carefully designed tools for OCR, translation, archival search, and image understanding in History. On HistBench, HistAgent based on GPT-4o achieves an accuracy of 27.54% pass@1 and 36.47% pass@2, significantly outperforming LLMs with online search and generalist agents, including GPT-4o (18.60%), DeepSeek-R1(14.49%) and Open Deep Research-smolagents(20.29% pass@1 and 25.12% pass@2). These results highlight the limitations of existing LLMs and generalist agents and demonstrate the advantages of HistAgent for historical reasoning.","url_abs":"https://arxiv.org/abs/2505.20246v1","url_pdf":"https://arxiv.org/pdf/2505.20246v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-path-to-multimodal-historical-reasoning","repo_url":"https://github.com/charlesq9/histagent","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"on-path-to-multimodal-historical-reasoning","repo_url":"https://github.com/charlesq9/alita","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"optical-character-recognition","task_name":"Optical Character Recognition (OCR)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.20246","atlas_url":"https://app.syntology.ai/?focus=2505.20246","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"2505.20246"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"2505.20246\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"2505.20246"},"call":"get_concepts_for_paper(arxiv_id=\"2505.20246\")"}],"by_repo":[{"repo":"charlesq9/histagent","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"charlesq9/alita","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"40e141bd85d4a044b22d8eb601df6f24a04c2380dfea037d41e81a3cf08d2491","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}