{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/challenges-in-the-automatic-analysis-of","title":"Challenges in the Automatic Analysis of Students' Diagnostic Reasoning","arxiv_id":"1811.10550","date":"2018-11-26","proceeding":null,"authors":["Claudia Schulz","Christian M. Meyer","Michael Sailer","Jan Kiesewetter","Elisabeth Bauer","Frank Fischer","Martin R. Fischer","Iryna Gurevych"],"abstract":"Diagnostic reasoning is a key component of many professions. To improve\nstudents' diagnostic reasoning skills, educational psychologists analyse and\ngive feedback on epistemic activities used by these students while diagnosing,\nin particular, hypothesis generation, evidence generation, evidence evaluation,\nand drawing conclusions. However, this manual analysis is highly\ntime-consuming. We aim to enable the large-scale adoption of diagnostic\nreasoning analysis and feedback by automating the epistemic activity\nidentification. We create the first corpus for this task, comprising diagnostic\nreasoning self-explanations of students from two domains annotated with\nepistemic activities. Based on insights from the corpus creation and the task's\ncharacteristics, we discuss three challenges for the automatic identification\nof epistemic activities using AI methods: the correct identification of\nepistemic activity spans, the reliable distinction of similar epistemic\nactivities, and the detection of overlapping epistemic activities. We propose a\nseparate performance metric for each challenge and thus provide an evaluation\nframework for future research. Indeed, our evaluation of various\nstate-of-the-art recurrent neural network architectures reveals that current\ntechniques fail to address some of these challenges.","url_abs":"http://arxiv.org/abs/1811.10550v1","url_pdf":"http://arxiv.org/pdf/1811.10550v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"challenges-in-the-automatic-analysis-of","repo_url":"https://github.com/UKPLab/aaai19-diagnostic-reasoning","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"diagnostic","task_name":"Diagnostic"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1811.10550","atlas_url":"https://app.syntology.ai/?focus=1811.10550","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1811.10550"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1811.10550\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1811.10550"},"call":"get_concepts_for_paper(arxiv_id=\"1811.10550\")"}],"by_repo":[{"repo":"UKPLab/aaai19-diagnostic-reasoning","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"a1676c4426f3a5835a648c77dafe3f918c3778df09e998c5d5d914ced51096a7","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}