{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/code/ndcg","entry":"ndcg","source":"Syntology graph, per-sample; not an archive number","read_at":"2026-10-05T00:36:11+00:00","claim":"Names are grouped by exact entry-name string. Same-named routines are NOT asserted to be equivalent; 'ran' means executed on a synthesized fixture, not correctness. n_samples_ran = sum of by_status over every status except 'unverified' (ran_draft_wrong and ran_fixture are failures of Syntology's instrument, not of the code); n_papers_ran = papers with at least one such sample = n_papers_ran_no_instrument_failure (papers with a run with no instrument failure) + n_papers_ran_instrument_only (papers where every run was a failure of Syntology's instrument); n_samples_ran = n_samples_ran_checked (honoured, violated, no contract) + n_samples_instrument (failures of Syntology's instrument, not of the code).","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"},"n_papers":16,"n_papers_ran":12,"n_papers_ran_no_instrument_failure":7,"n_papers_ran_instrument_only":5,"units":"n_samples, n_samples_ran, n_samples_fingerprinted and by_status count distinct code bodies (code_sha256); n_places and n_places_pointer_only count places, one per (paper, code body) pair, which is also the unit of the samples list","n_samples":15,"n_samples_ran":12,"n_samples_constructed":0,"n_samples_ran_checked":7,"n_samples_instrument":5,"n_samples_unverified":3,"n_samples_fingerprinted":4,"n_places":16,"n_places_pointer_only":8,"n_places_pointer_only_link_open":1,"inline_ok_is":"inline_ok is true only when both licence gates allow Syntology to serve this code's text: this paper's link (inline_ok_link, from the licence of this paper's copy) and the stored copy of the same code that Syntology serves, which may have been first harvested from another repository. withheld_clause is Syntology's own wording of which gate keeps the text a pointer and why, and is null when inline_ok is true. This site shows no code text either way; n_places_pointer_only_link_open counts the places where this paper's link allows the text and the stored copy does not","by_status":{"ran_honours":0,"ran_violates":0,"ran_draft_wrong":0,"ran_fixture":5,"ran":7,"unverified":3},"key_notes":{"n_samples_ran_checked":"legacy name, kept unchanged so existing readers do not break: it counts the samples that ran with no instrument failure (honoured, violated, and ran with no contract checked); it does not mean a contract was checked, and the pages print it as 'K with no instrument failure', not 'K checked'","n_samples_constructed":"a sub-count of the samples that ran, never subtracted from them and never a failure: an executed sample whose run returned an instance of its own class (fixture_out_type equals the entry name): the run built an object and did not compute a result (Syntology's RAN record, counts.constructed)"},"syntology":{"atlas_url":null,"mcp":null,"mcp_per_sample":{"tool":"get_code","arguments_in":"samples[].mcp_get_code"},"developers":"https://syntology.ai/developers"},"samples":[{"arxiv_id":"2608.22920","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2608-22920","title":"Beyond Observed Auxiliary Relations: Environment-Conditioned Modeling for Multi-Behavior Recommendation","date":null,"month_inferred_from_arxiv_id":"2026-08","title_source":"syntology","repo":"LSH0411/BOAR","path":"src/model/trainer.py","file_url":"https://github.com/LSH0411/BOAR/blob/HEAD/src/model/trainer.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, LSH0411/BOAR when it was checked","code_sha256":"dbb00da12cf9c374","code_sha256_prefix":"dbb00da12cf9c374","mcp_get_code":{"code_sha256":"dbb00da12cf9c374"}},{"arxiv_id":"2606.15903","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2606-15903","title":"Control-Plane Placement Shapes Forgetting: An Architectural Study of Agent Memory Across Thirteen System Configurations","date":null,"month_inferred_from_arxiv_id":"2026-06","title_source":"syntology","repo":"deeplethe/lethe","path":"bench/mempalace_bench.py","file_url":"https://github.com/deeplethe/lethe/blob/HEAD/bench/mempalace_bench.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"9257acfb3b218b9c","code_sha256_prefix":"9257acfb3b218b9c","mcp_get_code":{"code_sha256":"9257acfb3b218b9c"}},{"arxiv_id":"2601.06966","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2601-06966","title":"RealMem: Benchmarking LLMs in Real-World Memory-Driven Interaction","date":null,"month_inferred_from_arxiv_id":"2026-01","title_source":"syntology","repo":"AvatarMemory/RealMemBench","path":"eval/compute_auto_metrics_for_realmem.py","file_url":"https://github.com/AvatarMemory/RealMemBench/blob/HEAD/eval/compute_auto_metrics_for_realmem.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"71b9778fa0d631c8","code_sha256_prefix":"71b9778fa0d631c8","mcp_get_code":{"code_sha256":"71b9778fa0d631c8"}},{"arxiv_id":"2601.01280","arxiv_id_is_arxiv":true,"paper":"/paper/arxiv-2601-01280","title":"Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memory","date":null,"month_inferred_from_arxiv_id":"2026-01","title_source":"syntology","repo":"AvatarMemory/UnifiedMem","path":"evals/lme_eval_utils.py","file_url":"https://github.com/AvatarMemory/UnifiedMem/blob/HEAD/evals/lme_eval_utils.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, AvatarMemory/UnifiedMem when it was checked","code_sha256":"878c3d776ad1928f","code_sha256_prefix":"878c3d776ad1928f","mcp_get_code":{"code_sha256":"878c3d776ad1928f"}},{"arxiv_id":"2410.20745","arxiv_id_is_arxiv":true,"paper":"/paper/shopping-mmlu-a-massive-multi-task-online","title":"Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models","date":"2024-10-28","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"KL4805/ShoppingMMLU","path":"task_wise_eval/hf_ranking.py","file_url":"https://github.com/KL4805/ShoppingMMLU/blob/HEAD/task_wise_eval/hf_ranking.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"8d9104b9e41ddb8b","code_sha256_prefix":"8d9104b9e41ddb8b","mcp_get_code":{"code_sha256":"8d9104b9e41ddb8b"}},{"arxiv_id":"2410.10813","arxiv_id_is_arxiv":true,"paper":"/paper/longmemeval-benchmarking-chat-assistants-on","title":"LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory","date":"2024-10-14","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"xiaowu0162/longmemeval","path":"src/retrieval/eval_utils.py","file_url":"https://github.com/xiaowu0162/longmemeval/blob/HEAD/src/retrieval/eval_utils.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"inline_ok_link":true,"withheld_clause":"this paper's link allows the body inline (MIT), but get_code serves the stored copy, first harvested from AvatarMemory/UnifiedMem, and GitHub detected no licence in that repository when it was checked","code_sha256":"878c3d776ad1928f","code_sha256_prefix":"878c3d776ad1928f","mcp_get_code":{"code_sha256":"878c3d776ad1928f"}},{"arxiv_id":"2410.06581","arxiv_id_is_arxiv":true,"paper":"/paper/enhancing-legal-case-retrieval-via-scaling","title":"Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs","date":"2024-10-09","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"thunlp/LEAD","path":"LeCaRD/metrics.py","file_url":"https://github.com/thunlp/LEAD/blob/HEAD/LeCaRD/metrics.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"a6887cb2464e843e","code_sha256_prefix":"a6887cb2464e843e","mcp_get_code":{"code_sha256":"a6887cb2464e843e"}},{"arxiv_id":"2408.14393","arxiv_id_is_arxiv":true,"paper":"/paper/cure4rec-a-benchmark-for-recommendation","title":"CURE4Rec: A Benchmark for Recommendation Unlearning with Deeper Influence","date":"2024-08-26","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"xiye7lai/CURE4Rec","path":"utils.py","file_url":"https://github.com/xiye7lai/CURE4Rec/blob/HEAD/utils.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, xiye7lai/CURE4Rec when it was checked","code_sha256":"7f429335b5cc6bd3","code_sha256_prefix":"7f429335b5cc6bd3","mcp_get_code":{"code_sha256":"7f429335b5cc6bd3"}},{"arxiv_id":"2408.08931","arxiv_id_is_arxiv":true,"paper":"/paper/personalized-federated-collaborative","title":"Personalized Federated Collaborative Filtering: A Variational AutoEncoder Approach","date":"2024-08-16","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"mtics/feddae","path":"utils/evaluation.py","file_url":"https://github.com/mtics/feddae/blob/HEAD/utils/evaluation.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, mtics/feddae when it was checked","code_sha256":"f9bdf014fdeae23f","code_sha256_prefix":"f9bdf014fdeae23f","mcp_get_code":{"code_sha256":"f9bdf014fdeae23f"}},{"arxiv_id":"2403.13574","arxiv_id_is_arxiv":true,"paper":"/paper/a-large-language-model-enhanced-sequential","title":"A Large Language Model Enhanced Sequential Recommender for Joint Video and Comment Recommendation","date":"2024-03-20","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"rucaibox/lsvcr","path":"metrics.py","file_url":"https://github.com/rucaibox/lsvcr/blob/HEAD/metrics.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, rucaibox/lsvcr when it was checked","code_sha256":"b2215adec6371d49","code_sha256_prefix":"b2215adec6371d49","mcp_get_code":{"code_sha256":"b2215adec6371d49"}},{"arxiv_id":"2309.01103","arxiv_id_is_arxiv":true,"paper":"/paper/multi-relational-contrastive-learning-for","title":"Multi-Relational Contrastive Learning for Recommendation","date":"2023-09-03","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"HKUDS/RCL","path":"evaluate.py","file_url":"https://github.com/HKUDS/RCL/blob/HEAD/evaluate.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, HKUDS/RCL when it was checked","code_sha256":"334fbf3e8871bb11","code_sha256_prefix":"334fbf3e8871bb11","mcp_get_code":{"code_sha256":"334fbf3e8871bb11"}},{"arxiv_id":"2308.14029","arxiv_id_is_arxiv":true,"paper":"/paper/text-matching-improves-sequential","title":"Text Matching Improves Sequential Recommendation by Reducing Popularity Biases","date":"2023-08-27","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"openmatch/taste","path":"utils/rec_metrics.py","file_url":"https://github.com/openmatch/taste/blob/HEAD/utils/rec_metrics.py","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"e352eaf812952701","code_sha256_prefix":"e352eaf812952701","mcp_get_code":{"code_sha256":"e352eaf812952701"}},{"arxiv_id":"2206.04789","arxiv_id_is_arxiv":true,"paper":"/paper/comprehensive-fair-meta-learned-recommender","title":"Comprehensive Fair Meta-learned Recommender System","date":"2022-06-09","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"weitianxin/CLOVER","path":"ML1M/maml.py","file_url":"https://github.com/weitianxin/CLOVER/blob/HEAD/ML1M/maml.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"inline_ok_link":false,"withheld_clause":"GitHub detected no licence in this paper's repository, weitianxin/CLOVER when it was checked","code_sha256":"fd2a56eaa1ac6a6b","code_sha256_prefix":"fd2a56eaa1ac6a6b","mcp_get_code":{"code_sha256":"fd2a56eaa1ac6a6b"}},{"arxiv_id":"1909.02050","arxiv_id_is_arxiv":true,"paper":"/paper/tiger-text-to-image-grounding-for-image","title":"TIGEr: Text-to-Image Grounding for Image Caption Evaluation","date":"2019-09-04","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"SeleenaJM/CapEval","path":"tiger.py","file_url":"https://github.com/SeleenaJM/CapEval/blob/HEAD/tiger.py","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"f8a275b3e4999aca","code_sha256_prefix":"f8a275b3e4999aca","mcp_get_code":{"code_sha256":"f8a275b3e4999aca"}},{"arxiv_id":"1802.05814","arxiv_id_is_arxiv":true,"paper":"/paper/variational-autoencoders-for-collaborative","title":"Variational Autoencoders for Collaborative Filtering","date":"2018-02-16","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"mkfilipiuk/VAE-CF","path":"vae/metrics/ndcg.py","file_url":"https://github.com/mkfilipiuk/VAE-CF/blob/HEAD/vae/metrics/ndcg.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"4026df4b925b1a11","code_sha256_prefix":"4026df4b925b1a11","mcp_get_code":{"code_sha256":"4026df4b925b1a11"}},{"arxiv_id":"0705.2011","arxiv_id_is_arxiv":true,"paper":"/paper/multi-dimensional-recurrent-neural-networks","title":"Multi-Dimensional Recurrent Neural Networks","date":"2007-05-14","month_inferred_from_arxiv_id":null,"title_source":"archive","repo":"jzbjyb/rri_match","path":"metric.py","file_url":"https://github.com/jzbjyb/rri_match/blob/HEAD/metric.py","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"inline_ok_link":true,"withheld_clause":null,"code_sha256":"e41abfe79a1abfca","code_sha256_prefix":"e41abfe79a1abfca","mcp_get_code":{"code_sha256":"e41abfe79a1abfca"}}],"record_sha256":"96a3c92e1633f0717a524bdbbfb867852eff2cd4656e53694fea42e2b2d5ea36","record_changed_at":"2026-10-05","record_changed_at_basis":"changed"}