{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/model-free-adaptive-optimal-control-of","title":"Model-Free Adaptive Optimal Control of Episodic Fixed-Horizon Manufacturing Processes using Reinforcement Learning","arxiv_id":"1809.06646","date":"2018-09-18","proceeding":null,"authors":["Johannes Dornheim","Norbert Link","Peter Gumbsch"],"abstract":"A self-learning optimal control algorithm for episodic fixed-horizon\nmanufacturing processes with time-discrete control actions is proposed and\nevaluated on a simulated deep drawing process. The control model is built\nduring consecutive process executions under optimal control via reinforcement\nlearning, using the measured product quality as reward after each process\nexecution. Prior model formulation, which is required by state-of-the-art\nalgorithms from model predictive control and approximate dynamic programming,\nis therefore obsolete. This avoids several difficulties namely in system\nidentification, accurate modelling, and runtime complexity, that arise when\ndealing with processes subject to nonlinear dynamics and stochastic influences.\nInstead of using pre-created process and observation models, value\nfunction-based reinforcement learning algorithms build functions of expected\nfuture reward, which are used to derive optimal process control decisions. The\nexpectation functions are learned online, by interacting with the process. The\nproposed algorithm takes stochastic variations of the process conditions into\naccount and is able to cope with partial observability. A Q-learning-based\nmethod for adaptive optimal control of partially observable episodic\nfixed-horizon manufacturing processes is developed and studied. The resulting\nalgorithm is instantiated and evaluated by applying it to a simulated\nstochastic optimal control problem in metal sheet deep drawing.","url_abs":"http://arxiv.org/abs/1809.06646v3","url_pdf":"http://arxiv.org/pdf/1809.06646v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"model-free-adaptive-optimal-control-of","repo_url":"https://github.com/johannes-dornheim/Reinforce-FE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"model-predictive-control","task_name":"Model Predictive Control"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"self-learning","task_name":"Self-Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"johannes-dornheim/Reinforce-FE","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"45502eb87a451b5a3cba8970a8e641834a861d969eba075e50a9b5343e9728ea","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}