{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/reinforcement-learning-to-rank-in-e-commerce","title":"Reinforcement Learning to Rank in E-Commerce Search Engine: Formalization, Analysis, and Application","arxiv_id":"1803.00710","date":"2018-03-02","proceeding":null,"authors":["Yujing Hu","Qing Da","An-Xiang Zeng","Yang Yu","Yinghui Xu"],"abstract":"In e-commerce platforms such as Amazon and TaoBao, ranking items in a search\nsession is a typical multi-step decision-making problem. Learning to rank (LTR)\nmethods have been widely applied to ranking problems. However, such methods\noften consider different ranking steps in a session to be independent, which\nconversely may be highly correlated to each other. For better utilizing the\ncorrelation between different ranking steps, in this paper, we propose to use\nreinforcement learning (RL) to learn an optimal ranking policy which maximizes\nthe expected accumulative rewards in a search session. Firstly, we formally\ndefine the concept of search session Markov decision process (SSMDP) to\nformulate the multi-step ranking problem. Secondly, we analyze the property of\nSSMDP and theoretically prove the necessity of maximizing accumulative rewards.\nLastly, we propose a novel policy gradient algorithm for learning an optimal\nranking policy, which is able to deal with the problem of high reward variance\nand unbalanced reward distribution of an SSMDP. Experiments are conducted in\nsimulation and TaoBao search engine. The results demonstrate that our algorithm\nperforms much better than online LTR methods, with more than 40% and 30% growth\nof total transaction amount in the simulation and the real application,\nrespectively.","url_abs":"http://arxiv.org/abs/1803.00710v3","url_pdf":"http://arxiv.org/pdf/1803.00710v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reinforcement-learning-to-rank-in-e-commerce","repo_url":"https://github.com/UnibucProjects/DeepRLRecommenderSystem","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"learning-to-rank","task_name":"Learning-To-Rank"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1803.00710","atlas_url":"https://app.syntology.ai/?focus=1803.00710","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1803.00710"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1803.00710\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1803.00710"},"call":"get_concepts_for_paper(arxiv_id=\"1803.00710\")"}],"by_repo":[{"repo":"UnibucProjects/DeepRLRecommenderSystem","kind":"listed","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"47ac34839a3141153ca7ef38b3ba46939cd3f22b828b509b9af03bfeff2c117c","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}