{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/exploring-naive-approaches-to-tell-apart-llms","title":"Exploring Naive Approaches to Tell Apart LLMs Productions from Human-written Text","arxiv_id":null,"date":"2024-03-05","proceeding":"NLPIR '23: Proceedings of the 2023 7th International Conference on Natural Language Processing and Information Retrieval 2024 3","authors":["Oliver Giudice","Alessandro Maggi","Matteo Nardelli"],"abstract":"Powerful Large Language Models (large LMs or LLMs) such as BERT and GPT are making the task of detecting machine-generated text more and more prominent and crucial to minimize threats posed by text generation models misuse. Nonetheless, only a limited number of efforts exist so far, which can be classified into simple classifiers, zero-shot approaches, and fine-tuned LMs. These approaches usually rely on LMs whose discrimination accuracy decreases as the size difference in favor of the generator model increases (hence, a detector should always employ a LM with at least the same number of parameters of the source LM). Also, most of these approaches do not explicitly investigate whether the sentence syntactic structure can provide additional information that helps to build better detectors. All these considerations make the generalizing ability of detection methods into question. While generation techniques become more and more capable of producing human-like text, are the detection techniques capable of keeping up if not properly trained?\r\n\r\nIn this paper, we evaluate the most effective (and reproducible) detection method available in the state of the art in order to figure out the limits in its robustness. We complement this analysis by discussing results obtained using a novel naive approach that demonstrably achieves comparable results in terms of robustness with respect to much more advanced and sophisticated state-of-the-art methods. Code with details on experiments are available at: https://github.com/bancaditalia/gen-text-detect.","url_abs":"https://dl.acm.org/doi/abs/10.1145/3639233.3639354","url_pdf":"https://dl.acm.org/doi/abs/10.1145/3639233.3639354","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploring-naive-approaches-to-tell-apart-llms","repo_url":"https://github.com/bancaditalia/gen-text-detect","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"llm-generated-text-detection","task_name":"LLM-generated Text Detection"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"discriminative-fine-tuning","method_name":"Discriminative Fine-Tuning"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt","method_name":"GPT"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers","held":null,"reason":"no_arxiv_id","checked_against":null,"by_repo":[{"repo":"bancaditalia/gen-text-detect","kind":"named_in_paper","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"f3154449f4fb1b513a19336c8122659ff734d8b2a8ee54da9457a4a628e80e7a","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}