Papers › Searching for A Robust Neural Architecture in Four GPU Hours

Searching for A Robust Neural Architecture in Four GPU Hours

10 Oct 2019CVPR 2019 6arXiv:1910.04465archive 2025-07-28

Xuanyi Dong, Yi Yang

Conventional neural architecture search (NAS) approaches are based on reinforcement learning or evolutionary strategy, which take more than 3000 GPU hours to find a good model on CIFAR-10. We propose an efficient NAS approach learning to search by gradient descent. Our approach represents the search space as a directed acyclic graph (DAG). This DAG contains billions of sub-graphs, each of which indicates a kind of neural architecture. To avoid traversing all the possibilities of the sub-graphs, we develop a differentiable sampler over the DAG. This sampler is learnable and optimized by the validation loss after training the sampled architecture. In this way, our approach can be trained in an end-to-end fashion by gradient descent, named Gradient-based search using Differentiable Architecture Sampler (GDAS). In experiments, we can finish one searching procedure in four GPU hours on CIFAR-10, and the discovered model obtains a test error of 2.82% with only 2.5M parameters, which is on par with the state-of-the-art. Code is publicly available on GitHub: https://github.com/D-X-Y/NAS-Projects.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

D-X-Y/NAS-Projects officialmentioned in papermentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search (arXiv:2001.00326; that paper's own run record: 15 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 2 violated, 9 with no contract checked; 4 where Syntology's instrument failed) · 5 unverified); nothing checks that they implement this paper's method. report
D-X-Y/AutoDL-Projects mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets (arXiv:2107.00860; that paper's own run record: 12 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 1 honoured, 0 violated, 10 with no contract checked; 1 where Syntology's instrument failed) · 7 unverified); nothing checks that they implement this paper's method. report
D-X-Y/GDAS mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to One-Shot Neural Architecture Search via Self-Evaluated Template Network (arXiv:1910.05733; that paper's own run record: 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
epfl-ml-reproducers/subspace-attack-reproduction mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Wide Residual Networks (arXiv:1605.07146; that paper's own run record: 66 ran (of which 31 constructed an object rather than computing a result; 47 with no instrument failure: 0 honoured, 0 violated, 47 with no contract checked; 19 where Syntology's instrument failed) · 30 unverified); nothing checks that they implement this paper's method. report
rwbfd/OpenCompetitionV2 mentioned on GitHubpytorchGPL-3.0Syntology: not harvested report
xxlya/COS598D_Assignment1 mentioned on GitHubpytorchSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Neural Architecture SearchReinforcement Learning

1 archive task tag without a task page not shown.

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Neural Architecture Search CIFAR-10 GDAS (FRC) Search Time (GPU days) 0.17 #18 of 41 Archive leaderboard report
Neural Architecture Search CIFAR-10 GDAS (FRC) Top-1 Error Rate 2.5% #18 of 41 Archive leaderboard report
Neural Architecture Search CIFAR-10 GDAS Search Time (GPU days) 0.21 #40 of 41 Archive leaderboard report
Neural Architecture Search CIFAR-10 GDAS Top-1 Error Rate 3.4% #40 of 41 Archive leaderboard report
Neural Architecture Search NAS-Bench-201, CIFAR-10 GDAS Accuracy (Test) 93.61 #22 of 37 Archive leaderboard report
Neural Architecture Search NAS-Bench-201, CIFAR-10 GDAS Accuracy (Val) 89.89 #22 of 37 Archive leaderboard report
Neural Architecture Search NAS-Bench-201, CIFAR-10 GDAS Search time (s) 28926 #22 of 37 Archive leaderboard report
Neural Architecture Search NAS-Bench-201, CIFAR-100 GDAS Accuracy (Test) 70.70 #26 of 40 Archive leaderboard report
Neural Architecture Search NAS-Bench-201, CIFAR-100 GDAS Accuracy (Val) 71.34 #26 of 40 Archive leaderboard report
Neural Architecture Search NAS-Bench-201, CIFAR-100 GDAS Search time (s) 28926 #26 of 40 Archive leaderboard report
Neural Architecture Search NAS-Bench-201, ImageNet-16-120 GDAS Accuracy (Test) 41.71 #36 of 49 Archive leaderboard report
Neural Architecture Search NAS-Bench-201, ImageNet-16-120 GDAS Search time (s) 28926 #36 of 49 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

LSTMSigmoid ActivationSoftmaxTanh ActivationTest

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections