{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/a-crowdsourced-implementation-of-itu-t-p-910","title":"A crowdsourcing approach to video quality assessment","arxiv_id":"2204.06784","date":"2022-04-14","proceeding":null,"authors":["Babak Naderi","Ross Cutler"],"abstract":"The gold standard for measuring video quality is the subjective test, and the most prevalent is the ITU-T Rec. P.910, a lab-based subjective standard in use for the past two decades. However, in practice using ITU-T Rec. P.910 is slow, expensive, and requires a lab, which all create barriers to usage. As a result, most research papers that need to measure video quality don't use ITU-T Rec. P.910 but rather metrics that are not well correlated to subjective opinion. We provide an open-source extension of ITU-T Rec. P.910 based on crowdsourcing principles which address the speed, usage cost, and barrier to usage issues. We implement Absolute Category Rating (ACR), ACR with hidden reference (ACR-HR), Degradation Category Rating (DCR), and Comparison Category Rating (CCR). It includes rater, environment, hardware, and network qualifications, as well as gold and trapping questions to ensure quality. We have validated that the implementation is both accurate and highly reproducible compared to existing ITU-T Rec. P.910 lab studies.","url_abs":"https://arxiv.org/abs/2204.06784v3","url_pdf":"https://arxiv.org/pdf/2204.06784v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-crowdsourced-implementation-of-itu-t-p-910","repo_url":"https://github.com/microsoft/p.910","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"video-quality-assessment","task_name":"Video Quality Assessment"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"arxiv_id_not_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"by_repo":[{"repo":"microsoft/p.910","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"58b8e987a75bb2ccf8fb12712c5f1b3438a90314bb0000c79fcdc5a59c91afaf","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}