{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/method/automl-zero","slug":"automl-zero","name":"AutoML-Zero","full_name":"AutoML-Zero","full_name_withheld":false,"description_markdown":"**AutoML-Zero** is an AutoML technique that aims to search a fine-grained space simultaneously for the model, optimization procedure, initialization, and so on, permitting much less human-design and even allowing the discovery of non-neural network algorithms. It represents ML algorithms as computer programs comprised of three component functions, Setup, Predict, and Learn, that performs initialization, prediction and learning. The instructions in these functions apply basic mathematical operations on a small memory. The operation and memory addresses used by each instruction are free parameters in the search space, as is the size of the component functions. While this reduces expert design, the consequent sparsity means that [random search](https://paperswithcode.com/method/random-search) cannot make enough progress. To overcome this difficulty, the authors use small proxy tasks and migration techniques to build an optimized infrastructure capable of searching through 10,000 models/second/cpu core.\r\n\r\nEvolutionary methods can find solutions in the AutoML-Zero search space despite its enormous\r\nsize and sparsity. The authors show that by randomly modifying the programs and periodically selecting the best performing ones on given tasks/datasets, AutoML-Zero discovers reasonable algorithms. They start from empty programs and using data labeled by “teacher” neural networks with random weights, and demonstrate  evolution can discover neural networks trained by gradient descent. Following this, they minimize bias toward known algorithms by switching to binary classification tasks extracted from CIFAR-10 and allowing a larger set of possible operations. This discovers interesting techniques like multiplicative interactions, normalized gradient and weight averaging. Finally, they show it is possible for evolution to adapt the algorithm to the type of task provided. For example, [dropout](https://paperswithcode.com/method/dropout)-like operations emerge when the task needs regularization and learning rate decay appears when the task requires faster convergence.","description_state":"present","introduced_year":null,"introduced_by":{"title":"AutoML-Zero: Evolving Machine Learning Algorithms From Scratch","paper":"/paper/automl-zero-evolving-machine-learning","first_author":"Esteban Real","n_authors":4,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/automl-zero-evolving-machine-learning"},"source":{"url":"https://arxiv.org/abs/2003.03384v2","title":"AutoML-Zero: Evolving Machine Learning Algorithms From Scratch","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"AutoML","url":"/methods/category/automl","pwc_aliases":[]}],"n_papers_tagged":2,"archive_num_papers":2,"papers_newest_first":[{"paper":null,"slug":"discovering-adaptable-symbolic-algorithms","title":"Discovering Adaptable Symbolic Algorithms from Scratch","date":"2023-07-31","arxiv_id":"2307.16890","n_code_links":0,"syntology":null},{"paper":"/paper/automl-zero-evolving-machine-learning","slug":"automl-zero-evolving-machine-learning","title":"AutoML-Zero: Evolving Machine Learning Algorithms From Scratch","date":"2020-03-06","arxiv_id":"2003.03384","n_code_links":1,"syntology":null}],"papers_shown":2,"papers_order":"date (newest first), then slug","paper_counts":{"archive_papers_tagged":2,"with_a_code_link":1,"where_syntology_ran_a_sample":0,"not_listed_spam_title":0,"listed":2,"listed_where_code_ran":0,"where_syntology_ran_a_sample_split":{"with_a_run_with_no_instrument_failure":0,"every_run_a_failure_of_syntologys_instrument":0,"listed_with_a_run_with_no_instrument_failure":0,"listed_every_run_a_failure_of_syntologys_instrument":0,"filter":{"states":["a run with no instrument failure","any run, instrument failures included"],"default":"a run with no instrument failure","note":"on the 'only where code ran' pages the default hides, in the browser, the rows where every run was a failure of Syntology's instrument; the second state shows them again. Rows are hidden, never re-ordered; these twins list every row"}},"definition":"distinct papers the archive tags; 'where Syntology ran a sample' counts papers with at least one harvested sample that ran, which is not a correctness claim"},"papers_where_code_ran":0,"papers_where_code_ran_first_page":null,"papers_where_code_ran_caption":"We ran code from the paper's repository; we did not isolate this method inside it.","pages":[],"tasks":[{"task":"/task/automl","name":"AutoML","papers":2},{"task":"/task/machine-learning","name":"BIG-bench Machine Learning","papers":1}],"tasks_shown":2,"n_tasks":2,"usage_by_year":[{"year":"2020","papers":1},{"year":"2023","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/automl-zero"},"syntology_read_at":"2026-10-05T00:36:11+00:00","syntology":{"source":"Syntology's have check for this name (the free coverage lookup), read from Syntology's graph","have_url":"https://syntology.ai/api/graph/have?x=AutoML-Zero","read_at":"2026-10-05T00:52:54+00:00","query":"AutoML-Zero","kind":"method","have":"declared_code","tier":"repo_known","verified_implementation":false,"level":null,"level_meaning":"the served verification level, per answer; never summed across methods","resolved_to":{"method":"AutoML-Zero","origin_arxiv_id":"2003.03384"},"resolution":"exact","resolved_name_differs":false,"absence_asserted":false,"absence_unchecked":null,"no_code_is_about_this_method":false,"subject":"same_paper","archive_paper_arxiv_id":"2003.03384","repositories":{"listed":1,"in_graph":1,"list_truncated":false,"strongest_link":"official"},"retrieve_with":null,"name_competition":{"verdict":"sole_bearer","may_be_wrong_sense":false,"bearers_found":1,"served":{"arxiv_id":"2003.03384","citations":290},"leading_other_bearer":null,"n_other_bearers_listed":0,"note":"Nothing else in this corpus is named 'AutoML-Zero'. That is a statement about this corpus and not about the field: short names are reused, and a sense we never collected cannot show up here. Confirm `served.arxiv_id` if you had a specific paper in mind."},"name_competition_text":null,"near_matches":null,"code_nearby":[{"name":"Zero-Shot Tokenizer Transfer (ZeTT)","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":2,"retrieve_with":"get_reference_implementation(name='Zero-Shot Tokenizer Transfer (ZeTT)')","origin_arxiv_id":"2405.07883"},{"name":"Simple Zero-shot Prompt Weighting Technique","relation":"shares_name_tokens","relation_means":"shares name tokens with this name; a different method","level":3,"retrieve_with":"get_reference_implementation(name='Simple Zero-shot Prompt Weighting Technique')","origin_arxiv_id":"2302.06235"}],"code_nearby_caption":"Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.","origin_paper":{"arxiv_id":"2003.03384","page":"/paper/automl-zero-evolving-machine-learning","syntology_url":"https://syntology.ai/paper/2003.03384","in_graph_layer":false,"samples":0},"line":"repository known, no verified implementation (declared_code; repositories: 1 listed of 1 in the graph, strongest link: official). This answer is about arXiv:2003.03384, the paper the archive credits with this method.","level_legend":{"state":"quoted","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","card_sha256":"479785a84658dd682a11b10237b3620d651d7f87799f5dc7add970aaa83f6d39","note":"Syntology's served verification levels, quoted verbatim from its MCP server card; a row is split where the card separates clauses with a double hyphen, and the separator is not reproduced","levels":[{"level":"V0","quoted":["imports"]},{"level":"V1","quoted":["executes against a synthesized fixture and returns a value of the recorded type"]},{"level":"V2","quoted":["two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs","neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)"]},{"level":"V3","quoted":["paper-derived property tests pass on both of those same two generations."]}],"no_rung":"No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results."}},"syntology_have_state":"loaded","record_sha256":"c6b94464ad1fb21d1206e616fc32160e78c0e4fdbb3289b20fb6175a8c7cc26e","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}