Artificial intelligence detection software aims to determine whether some content (text, image, video or audio) was generated using

artificial intelligence Artificial intelligence (AI) is the capability of computer, computational systems to perform tasks typically associated with human intelligence, such as learning, reasoning, problem-solving, perception, and decision-making. It is a field of re ...

(AI). However, this software is often unreliable.

Accuracy issues

Many AI detection tools have been shown to be unreliable when generating AI-generated text. In a 2023 study conducted by Weber-Wulff et al., researchers evaluated 14 detection tools including

Turnitin Turnitin (stylized as turnitin) is an Internet-based similarity detection service run by the American company Turnitin, LLC, a subsidiary of Advance Publications. Founded in 1998, it sells its licenses to universities and high schools who the ...

and GPTZero and found that "all scored below 80% of accuracy and only 5 over 70%." They also found that these tools tend to have a bias for classifying texts more as human than as AI, and that accuracy of these tools worsens upon paraphrasing.

False positives

In AI content detection, a

false positive A false positive is an error in binary classification in which a test result incorrectly indicates the presence of a condition (such as a disease when the disease is not present), while a false negative is the opposite error, where the test resu ...

is when human-written work is incorrectly flagged as AI-written. Many AI detection platforms claim to have a minimal level of false positives, with Turnitin claiming a less than 1% false positive rate. However, later research by

The Washington Post ''The Washington Post'', locally known as ''The'' ''Post'' and, informally, ''WaPo'' or ''WP'', is an American daily newspaper published in Washington, D.C., the national capital. It is the most widely circulated newspaper in the Washington m ...

produced much higher rates of 50%, though they used a smaller sample size. False positives in an academic setting frequently lead to accusations of academic misconduct, which can have serious consequences for a student's academic record. Additionally, studies have shown evidence that many AI detection models are prone to give false positives to work written by those whose

first language A first language (L1), native language, native tongue, or mother tongue is the first language a person has been exposed to from birth or within the critical period hypothesis, critical period. In some countries, the term ''native language'' ...

isn't English and neurodiverse people. In June 2023, Janelle Shane wrote that portions of her book ''You Look Like a Thing and I Love You'' were flagged as AI-generated.

False negatives

false negative A false positive is an error in binary classification in which a test result incorrectly indicates the presence of a condition (such as a disease when the disease is not present), while a false negative is the opposite error, where the test resu ...

is a failure to identify documents with AI-written text. False negatives often happen as a result of a detection software's sensitivity level or because evasive techniques were used when generating the work to make it sound more human. False negatives are less of a concern academically, since they aren't likely to lead to accusations and ramifications. Notably, Turnitin stated they have a 15% false negative rate.

Text detection

For text, this is usually done to prevent alleged

plagiarism Plagiarism is the representation of another person's language, thoughts, ideas, or expressions as one's own original work.From the 1995 ''Random House Dictionary of the English Language, Random House Compact Unabridged Dictionary'': use or close ...

, often by detecting repetition of words as telltale signs that a text was AI-generated (including

hallucinations A hallucination is a perception in the absence of an external stimulus that has the compelling sense of reality. They are distinguishable from several related phenomena, such as dreaming ( REM sleep), which does not involve wakefulness; pse ...

). They are often used by teachers marking their students, usually on an ''

ad hoc ''Ad hoc'' is a List of Latin phrases, Latin phrase meaning literally for this. In English language, English, it typically signifies a solution designed for a specific purpose, problem, or task rather than a Generalization, generalized solution ...

'' basis. Following the release of

ChatGPT ChatGPT is a generative artificial intelligence chatbot developed by OpenAI and released on November 30, 2022. It uses large language models (LLMs) such as GPT-4o as well as other Multimodal learning, multimodal models to create human-like re ...

and similar AI text generative software, many educational establishments have issued policies against the use of AI by students. AI text detection software is also used by those assessing job applicants, as well as online

search engines Search engines, including web search engines, selection-based search engines, metasearch engines, desktop search tools, and web portals and vertical market websites have a search facility for online databases. By content/topic Gene ...

. Current detectors may sometimes be unreliable and have incorrectly marked work by humans as originating from AI while failing to detect AI-generated work in other instances. ''

MIT Technology Review ''MIT Technology Review'' is a bimonthly magazine wholly owned by the Massachusetts Institute of Technology. It was founded in 1899 as ''The Technology Review'', and was re-launched without "''The''" in its name on April 23, 1998, under then pu ...

'' said that the technology "struggled to pick up ChatGPT-generated text that had been slightly rearranged by humans and obfuscated by a paraphrasing tool". AI text detection software has also been shown to discriminate against non-native speakers of English. Two students from the

University of California, Davis The University of California, Davis (UC Davis, UCD, or Davis) is a Public university, public Land-grant university, land-grant research university in Davis, California, United States. It is the northernmost of the ten campuses of the University ...

, were referred to the university's Office of Student Success and Judicial Affairs (OSSJA) after their professors scanned their essays with positive results; the first with an AI detector called GPTZero, and the second with an AI detector integration in

. However, following media coverage, and a thorough investigation, the students were cleared of any wrongdoing. In April 2023,

Cambridge University The University of Cambridge is a Public university, public collegiate university, collegiate research university in Cambridge, England. Founded in 1209, the University of Cambridge is the List of oldest universities in continuous operation, wo ...

and other members of the

Russell Group The Russell Group is a self-selected association of twenty-four public research universities in the United Kingdom. The group is headquartered in Cambridge and was established in 1994 to represent its members' interests, principally to governme ...

of universities in the United Kingdom opted out of Turnitin's AI text detection tool, after expressing concerns it was unreliable. The

University of Texas at Austin The University of Texas at Austin (UT Austin, UT, or Texas) is a public university, public research university in Austin, Texas, United States. Founded in 1883, it is the flagship institution of the University of Texas System. With 53,082 stud ...

opted out of the system six months later. In May 2023, a professor at

Texas A&M University–Commerce East Texas A&M University (ETAMU; formerly Texas A&M University–Commerce) is a public university in Commerce, Texas, United States. With an enrollment of over 12,000 students as of fall 2017, the university is the third-largest institution in ...

used ChatGPT to detect whether his students' content was written by it, which ChatGPT said was the case. As such, he threatened to fail the class despite ChatGPT not being able to detect AI-generated writing. No students were prevented from graduating because of the issue, and all but one student (who admitted to using the software) were exonerated from accusations of having used ChatGPT in their content. In July 2023, a paper titled "GPT detectors are biased against non-native English writers" was released, reporting that GPTs discriminate against non-native English authors. The paper compared seven GPT detectors against essays from both non-native English speakers and essays from United States students. The essays from non-native English speakers had an average false positive rate of 61.3%. An article by Thomas Germain, published on

Gizmodo ''Gizmodo'' () is a design, technology, science, and science fiction website. It was originally launched as part of the Gawker Media network run by Nick Denton. ''Gizmodo'' also includes the sub-blogs ''io9'' and ''Earther'', which focus on pop ...

in June 2024, reported job losses among freelance writers and journalists due to AI text detection software mistakenly classifying their work as AI-generated. In September 2024,

Common Sense Media Common Sense Media (CSM) is an American nonprofit organization that reviews and provides ratings for media and technology with the goal of providing information on their suitability for children.

reported that generative AI detectors had a 20% false positive rate for Black students, compared to 10% of Latino students and 7% of White students. To improve the reliability of AI text detection, researchers have explored

digital watermarking A digital watermark is a kind of marker covertly embedded in a noise-tolerant signal such as audio, video or image data.H.T. Sencar, M. Ramkumar and A.N. Akansu: ''Data Hiding Fundamentals and Applications: Content Security in Digital Multimedia'' ...

techniques. A 2023 paper titled "A Watermark for Large Language Models" presents a method to embed imperceptible watermarks into text generated by

large language models A large language model (LLM) is a language model trained with Self-supervised learning, self-supervised machine learning on a vast amount of text, designed for natural language processing tasks, especially Natural language generation, language g ...

(LLMs). This watermarking approach allows content to be flagged as AI-generated with a high level of accuracy, even when text is slightly paraphrased or modified. The technique is designed to be subtle and hard to detect for casual readers, thereby preserving readability, while providing a detectable signal for those employing specialized tools. However, while promising, watermarking faces challenges in remaining robust under adversarial transformations and ensuring compatibility across different LLMs.

Anti text detection

There is software available designed to bypass AI text detection. A study published in August 2023 analyzed 20 abstracts from papers published in the Eye Journal, which were then paraphrased using

GPT-4 Generative Pre-trained Transformer 4 (GPT-4) is a multimodal large language model trained and created by OpenAI and the fourth in its series of GPT foundation models. It was launched on March 14, 2023, and made publicly available via the p ...

.0. The AI-paraphrased abstracts were examined for plagiarism using QueText and for AI-generated content using Originality.AI. The texts were then re-processed through an adversarial software called Undetectable.ai in order to reduce the AI-detection scores. The study found that the AI detection tool, Originality.AI, identified text generated by GPT-4 with a mean accuracy of 91.3%. However, after reprocessing by Undetectable.ai, the detection accuracy of Originality.ai dropped to a mean accuracy of 27.8%. Some experts also believe that techniques like

are ineffective because they can be removed or added to trigger false positives. "A Watermark for Large Language Models" paper by Kirchenbauer et al. also addresses potential vulnerabilities of watermarking techniques. The authors outline a range of adversarial tactics, including text insertion, deletion, and substitution attacks, that could be used to bypass watermark detection. These attacks vary in complexity, from simple paraphrasing to more sophisticated approaches involving tokenization and homoglyph alterations. The study highlights the challenge of maintaining watermark robustness against attackers who may employ automated

paraphrasing A paraphrase () or rephrase is the rendering of the same text in different words without losing the meaning of the text itself. More often than not, a paraphrased text can convey its meaning better than the original words. In other words, it is a ...

tools or even specific language model replacements to alter text spans iteratively while retaining semantic similarity. Experimental results show that although such attacks can degrade watermark strength, they also come at the cost of text quality and increased computational resources.

Multilingual text detection

One shortcoming of most AI content detection software is their inability to identify AI-generated text in any language. Large language models (LLMs) like ChatGPT, Claude, and Gemini can write in different languages, but traditional AI text detection tools have primarily been trained in English and a few other widely spoken languages, such as French and Spanish. Fewer AI detection solutions can detect AI-generated text in languages like Farsi, Arabic, or Hindi.

Image, video, and audio detection

Several purported AI image detection software exist, to detect AI-generated images (for example, those originating from

Midjourney Midjourney is a generative artificial intelligence program and service created and hosted by the San Francisco-based independent research lab Midjourney, Inc. Midjourney generates images from natural language descriptions, called '' prompts'', ...

DALL-E DALL-E, DALL-E 2, and DALL-E 3 (stylised DALL·E) are text-to-image models developed by OpenAI using deep learning methodologies to generate digital images from natural language descriptions known as Prompt engineering, ''prompts''. The first ...

). They are not completely reliable. Others claim to identify video and audio

deepfake ''Deepfakes'' (a portmanteau of and ) are images, videos, or audio that have been edited or generated using artificial intelligence, AI-based tools or AV editing software. They may depict real or fictional people and are considered a form of ...

s, but this technology is also not fully reliable yet either. Despite debate around the efficacy of watermarking,

Google DeepMind DeepMind Technologies Limited, trading as Google DeepMind or simply DeepMind, is a British–American artificial intelligence research laboratory which serves as a subsidiary of Alphabet Inc. Founded in the UK in 2010, it was acquired by Goo ...

is actively developing a detection software called SynthID, which works by inserting a digital watermark that is invisible to the human eye into the

pixel In digital imaging, a pixel (abbreviated px), pel, or picture element is the smallest addressable element in a Raster graphics, raster image, or the smallest addressable element in a dot matrix display device. In most digital display devices, p ...

s of an image.

References

{{reflist Cybernetics Computational linguistics Computational neuroscience Computational fields of study Data science Educational assessment and evaluation Formal sciences Natural language processing Speech recognition Unsolved problems in computer science Plagiarism detectors Large language models ChatGPT Applications of artificial intelligence