{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/evaluating-and-enhancing-llms-for-multi-turn","title":"Evaluating and Enhancing LLMs for Multi-turn Text-to-SQL with Multiple Question Types","arxiv_id":"2412.17867","date":"2024-12-21","proceeding":null,"authors":["Ziming Guo","Chao Ma","Yinggang Sun","Tiancheng Zhao","Guangyao Wang","Hai Huang"],"abstract":"Recent advancements in large language models (LLMs) have significantly advanced text-to-SQL systems. However, most LLM-based methods often narrowly focus on SQL generation, neglecting the complexities of real-world conversational queries. This oversight can lead to unreliable responses, particularly for ambiguous questions that cannot be directly addressed with SQL. To bridge this gap, we propose MMSQL, a comprehensive test suite designed to evaluate the question classification and SQL generation capabilities of LLMs by simulating real-world scenarios with diverse question types and multi-turn Q\\&A interactions. Using MMSQL, we assessed the performance of popular LLMs, including both open-source and closed-source models, and identified key factors impacting their performance in such scenarios. Moreover, we introduce an LLM-based multi-agent framework that employs specialized agents to identify question types and determine appropriate answering strategies. Our experiments demonstrate that this approach significantly enhances the model's ability to navigate the complexities of conversational dynamics, effectively handling the diverse and complex nature of user queries.","url_abs":"https://arxiv.org/abs/2412.17867v1","url_pdf":"https://arxiv.org/pdf/2412.17867v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"evaluating-and-enhancing-llms-for-multi-turn","repo_url":"https://github.com/mcxiaoxiao/MMSQL","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"mmsql-performance","task_name":"MMSQL performance"},{"task_slug":"navigate","task_name":"Navigate"},{"task_slug":"text-to-sql","task_name":"Text to SQL"},{"task_slug":"text-to-sql","task_name":"Text-To-SQL"}],"methods":[{"method_slug":"cot-prompting","method_name":"CoT Prompting"},{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[{"slug":"mmsql","name":"MMSQL","full_name":"Multi-Turn Multi-Type Text-to-SQL test suit"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/mmsql-performance-on-mmsql","task":"MMSQL performance","dataset":"MMSQL","model":"GPT-4 Turbo","rank_in_archive_order":1,"of":6,"metrics":{"TDEX":"67.0"},"uses_additional_data":false},{"leaderboard":"/sota/mmsql-performance-on-mmsql","task":"MMSQL performance","dataset":"MMSQL","model":"Gemini-1.5 Flash","rank_in_archive_order":2,"of":6,"metrics":{"TDEX":"65.8"},"uses_additional_data":false},{"leaderboard":"/sota/mmsql-performance-on-mmsql","task":"MMSQL performance","dataset":"MMSQL","model":"GPT-3.5 Turbo","rank_in_archive_order":3,"of":6,"metrics":{"TDEX":"64.1"},"uses_additional_data":false},{"leaderboard":"/sota/mmsql-performance-on-mmsql","task":"MMSQL performance","dataset":"MMSQL","model":"Llama3-8B","rank_in_archive_order":4,"of":6,"metrics":{"TDEX":"64.0"},"uses_additional_data":false},{"leaderboard":"/sota/mmsql-performance-on-mmsql","task":"MMSQL performance","dataset":"MMSQL","model":"Llama3-70B","rank_in_archive_order":5,"of":6,"metrics":{"TDEX":"62.8"},"uses_additional_data":false},{"leaderboard":"/sota/mmsql-performance-on-mmsql","task":"MMSQL performance","dataset":"MMSQL","model":"SQLCoder-8B","rank_in_archive_order":6,"of":6,"metrics":{"TDEX":"30.7"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"mcxiaoxiao/MMSQL","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"3a12d316112b9b833106acd67427489ce59bd0ff6b4b97bcef3fbee3c03b8d78","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}