{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/image-transformer","title":"Image Transformer","arxiv_id":"1802.05751","date":"2018-02-15","proceeding":null,"authors":["Niki Parmar","Ashish Vaswani","Jakob Uszkoreit","Łukasz Kaiser","Noam Shazeer","Alexander Ku","Dustin Tran"],"abstract":"Image generation has been successfully cast as an autoregressive sequence\ngeneration or transformation problem. Recent work has shown that self-attention\nis an effective way of modeling textual sequences. In this work, we generalize\na recently proposed model architecture based on self-attention, the\nTransformer, to a sequence modeling formulation of image generation with a\ntractable likelihood. By restricting the self-attention mechanism to attend to\nlocal neighborhoods we significantly increase the size of images the model can\nprocess in practice, despite maintaining significantly larger receptive fields\nper layer than typical convolutional neural networks. While conceptually\nsimple, our generative models significantly outperform the current state of the\nart in image generation on ImageNet, improving the best published negative\nlog-likelihood on ImageNet from 3.83 to 3.77. We also present results on image\nsuper-resolution with a large magnification ratio, applying an encoder-decoder\nconfiguration of our architecture. In a human evaluation study, we find that\nimages generated by our super-resolution model fool human observers three times\nmore often than the previous state of the art.","url_abs":"http://arxiv.org/abs/1802.05751v3","url_pdf":"http://arxiv.org/pdf/1802.05751v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"density-estimation","task_name":"Density Estimation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-super-resolution","task_name":"Image Super-Resolution"},{"task_slug":"super-resolution","task_name":"Super-Resolution"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/density-estimation-on-cifar-10","task":"Density Estimation","dataset":"CIFAR-10","model":"Image Transformer","rank_in_archive_order":6,"of":15,"metrics":{"NLL (bits/dim)":"2.90"},"uses_additional_data":false},{"leaderboard":"/sota/density-estimation-on-imagenet-32x32-1","task":"Density Estimation","dataset":"ImageNet 32x32","model":"Image Transformer","rank_in_archive_order":4,"of":5,"metrics":{"NLL (bits/dim)":"3.77"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-imagenet-32x32","task":"Image Generation","dataset":"ImageNet 32x32","model":"Image Transformer","rank_in_archive_order":21,"of":35,"metrics":{"bpd":"3.77"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1802.05751","atlas_url":"https://app.syntology.ai/?focus=1802.05751","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1802.05751"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1802.05751\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1802.05751"},"call":"get_concepts_for_paper(arxiv_id=\"1802.05751\")"}],"by_repo":[],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":"the archive holds no code link for this paper","arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"93fd1911a496a0624f325a4d1f3ea745a10e0e9a8f00d6c81e21863168431559","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}