{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/ask-the-right-questions-active-question","title":"Ask the Right Questions: Active Question Reformulation with Reinforcement Learning","arxiv_id":"1705.07830","date":"2017-05-22","proceeding":"ICLR 2018 1","authors":["Christian Buck","Jannis Bulian","Massimiliano Ciaramita","Wojciech Gajewski","Andrea Gesmundo","Neil Houlsby","Wei Wang"],"abstract":"We frame Question Answering (QA) as a Reinforcement Learning task, an\napproach that we call Active Question Answering. We propose an agent that sits\nbetween the user and a black box QA system and learns to reformulate questions\nto elicit the best possible answers. The agent probes the system with,\npotentially many, natural language reformulations of an initial question and\naggregates the returned evidence to yield the best answer. The reformulation\nsystem is trained end-to-end to maximize answer quality using policy gradient.\nWe evaluate on SearchQA, a dataset of complex questions extracted from\nJeopardy!. The agent outperforms a state-of-the-art base model, playing the\nrole of the environment, and other benchmarks. We also analyze the language\nthat the agent has learned while interacting with the question answering\nsystem. We find that successful question reformulations look quite different\nfrom natural language paraphrases. The agent is able to discover non-trivial\nreformulation strategies that resemble classic information retrieval techniques\nsuch as term re-weighting (tf-idf) and stemming.","url_abs":"http://arxiv.org/abs/1705.07830v3","url_pdf":"http://arxiv.org/pdf/1705.07830v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ask-the-right-questions-active-question","repo_url":"https://github.com/BPYap/Question-Generation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"ask-the-right-questions-active-question","repo_url":"https://github.com/google/active-qa","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1705.07830","atlas_url":"https://app.syntology.ai/?focus=1705.07830","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1705.07830"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1705.07830\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1705.07830"},"call":"get_concepts_for_paper(arxiv_id=\"1705.07830\")"}],"by_repo":[{"repo":"BPYap/Question-Generation","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0},{"repo":"google/active-qa","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"db1f1fade95d51790fb480238b34fc4d540c66c437849c00375eadc072fc6a80","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}