{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/unreasonable-effectiveness-of-greedy","title":"Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms","arxiv_id":null,"date":"2020-12-01","proceeding":"NeurIPS 2020 12","authors":["Mohsen Bayati","Nima Hamidi","Ramesh Johari","Khashayar Khosravi"],"abstract":"We study the structure of regret-minimizing policies in the {\\em many-armed} Bayesian multi-armed bandit problem: in particular, with $k$ the number of arms and $T$ the time horizon, we consider the case where $k \\geq \\sqrt{T}$.\nWe first show that {\\em subsampling} is a critical step for designing optimal policies.  In particular, the standard UCB algorithm leads to sub-optimal regret bounds in the many-armed regime.  However, a subsampled UCB (SS-UCB), which samples $\\Theta(\\sqrt{T})$ arms and executes UCB only on that subset, is rate-optimal.  \nDespite theoretically optimal regret, even SS-UCB performs poorly due to excessive exploration of suboptimal arms.  In particular, in numerical experiments SS-UCB performs worse than a simple greedy algorithm (and its subsampled version) that pulls the current empirical best arm at every time period.\nWe show that these insights hold even in a contextual setting, using real-world data.\nThese empirical results suggest a novel form of {\\em free exploration} in the many-armed regime that benefits greedy algorithms. We theoretically study this new source of free exploration and find that it is deeply connected to the distribution of a certain tail event for the prior distribution of arm rewards.  This is a fundamentally distinct phenomenon from free exploration as discussed in the recent literature on contextual bandits, where free exploration arises due to variation in contexts.\nWe use this insight to prove that the subsampled greedy algorithm is rate-optimal for Bernoulli bandits when $k > \\sqrt{T}$, and achieves sublinear regret with more general distributions.  This is a case where theoretical rate optimality does not tell the whole story: when complemented by the empirical observations of our paper, the power of greedy algorithms becomes quite evident. Taken together, from a practical standpoint, our results suggest that in applications it may be preferable to use a variant of the greedy algorithm in the many-armed regime.","url_abs":"http://proceedings.neurips.cc/paper/2020/hash/12d16adf4a9355513f9d574b76087a08-Abstract.html","url_pdf":"http://proceedings.neurips.cc/paper/2020/file/12d16adf4a9355513f9d574b76087a08-Paper.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unreasonable-effectiveness-of-greedy","repo_url":"https://github.com/khashayarkhv/many-armed-bandit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"multi-armed-bandits","task_name":"Multi-Armed Bandits"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"no_arxiv_id","checked_against":null,"by_repo":[{"repo":"khashayarkhv/many-armed-bandit","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0,"harvested_for_other_papers":1,"harvested_for":["2002.10121"],"harvested_for_calls":[{"arxiv_id":"2002.10121","tool":"get_harvested_code_for_paper","arguments":{"arxiv_id":"2002.10121"},"call":"get_harvested_code_for_paper(arxiv_id=\"2002.10121\")"}]}],"by_repo_harvested_for_is":"harvested_for: the arXiv ids (as Syntology's graph holds them) of every paper whose samples Syntology's graph links to this repository, in arXiv-id order, complete; the page names the first 3 and says how many more. A link only: it does not say the papers are the same work, that one copied the other, or that the code implements this paper's method; harvested_for_calls gives the get_harvested_code_for_paper call for each arXiv-shaped id among them","by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"c1934a0da69299caa93632c616033b8795d4de1968e49aedca770826c3c08c5f","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}