{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/efficient-hyperparameter-optimization-of-deep","title":"Efficient Hyperparameter Optimization of Deep Learning Algorithms Using Deterministic RBF Surrogates","arxiv_id":"1607.08316","date":"2016-07-28","proceeding":null,"authors":["Ilija Ilievski","Taimoor Akhtar","Jiashi Feng","Christine Annette Shoemaker"],"abstract":"Automatically searching for optimal hyperparameter configurations is of\ncrucial importance for applying deep learning algorithms in practice. Recently,\nBayesian optimization has been proposed for optimizing hyperparameters of\nvarious machine learning algorithms. Those methods adopt probabilistic\nsurrogate models like Gaussian processes to approximate and minimize the\nvalidation error function of hyperparameter values. However, probabilistic\nsurrogates require accurate estimates of sufficient statistics (e.g.,\ncovariance) of the error distribution and thus need many function evaluations\nwith a sizeable number of hyperparameters. This makes them inefficient for\noptimizing hyperparameters of deep learning algorithms, which are highly\nexpensive to evaluate. In this work, we propose a new deterministic and\nefficient hyperparameter optimization method that employs radial basis\nfunctions as error surrogates. The proposed mixed integer algorithm, called\nHORD, searches the surrogate for the most promising hyperparameter values\nthrough dynamic coordinate search and requires many fewer function evaluations.\nHORD does well in low dimensions but it is exceptionally better in higher\ndimensions. Extensive evaluations on MNIST and CIFAR-10 for four deep neural\nnetworks demonstrate HORD significantly outperforms the well-established\nBayesian optimization methods such as GP, SMAC, and TPE. For instance, on\naverage, HORD is more than 6 times faster than GP-EI in obtaining the best\nconfiguration of 19 hyperparameters.","url_abs":"http://arxiv.org/abs/1607.08316v2","url_pdf":"http://arxiv.org/pdf/1607.08316v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-hyperparameter-optimization-of-deep","repo_url":"https://github.com/jekyllstein/HORDOpt.jl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"bayesian-optimization","task_name":"Bayesian Optimization"},{"task_slug":"gaussian-processes","task_name":"Gaussian Processes"},{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"},{"task_slug":"smac","task_name":"SMAC"},{"task_slug":"smac-1","task_name":"SMAC+"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"jekyllstein/HORDOpt.jl","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"40cff725aa5dbf1b657ee6f6ad0b52ad769ddcb1e105c1c852c8f6c502b4a3b0","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}