Research report
Does Turnitin Detect Gemini 3.5 Flash-Generated Undergraduate Essays?
A Benchmark Experiment on Turnitin's Word-Level Accuracy for Gemini 3.5 Flash-Generated Undergraduate Essays
Overview
This report presents a benchmark experiment evaluating Turnitin's AI-detection accuracy on a dataset of 90 undergraduate essays generated by the Gemini 3.5 Flash language model. The essays span 30 majors across five domains (Business, Education, Humanities, Psychology, STEM) and three word-count buckets (400, 800, 1200 words). Word-level accuracy is measured as the percentage of words correctly flagged as AI-generated.
- Overall word accuracy is 98.55% (73,426 correct words out of 74,505 total words).
- By domain: Psychology shows the highest accuracy (99.58%), followed by Business (99.30%), Education (99.13%), Humanities (98.64%), and STEM (98.16%).
- By major: Accounting, Business Administration, and Political Science achieve 100.00% accuracy. Information Technology has the lowest accuracy at 90.64% (2,364 correct out of 2,608 words).
- By word count: 800-word essays have the highest accuracy (98.99%), followed by 1200-word (98.40%) and 400-word (98.20%).
- Dataset: 90 essays generated by Gemini 3.5 Flash, all at undergraduate level, non-ESL, no humanizers applied.
- Domains and majors: 5 domains covering 30 majors (e.g., Accounting, Biology, Computer Science, Psychology).
- Word counts: Three buckets of 30 essays each: 400, 800, and 1200 words.
- Evaluation: Turnitin AI detection was run on each essay. Word-level accuracy is computed as (correctly flagged words / total words) × 100.
- Scoring unit: accuracy is measured in words. First identify correct paragraphs under the expectation below, then sum the words in those paragraphs for the correct-word count; total words are summed over all counted paragraphs.
- Correct paragraph: a counted paragraph that was flagged as AI. Correct words are the words in those paragraphs.
- References excluded: from a References/Bibliography-style heading onward, those paragraphs are left out of word counts.
- Limited sample: 90 essays per dataset; larger samples may yield different accuracy estimates.
Overall Performance
- Total articles: 90
- Correct words: 73,426 out of 74,505
- Word accuracy: 98.55%
Accuracy by Domain
- Business: 99.30% (12,326 correct / 12,413 words)
- Education: 99.13% (2,402 / 2,423)
- Humanities: 98.64% (19,198 / 19,462)
- Psychology: 99.58% (2,356 / 2,366)
- STEM: 98.16% (37,144 / 37,841)
Accuracy by Major
- Top majors (100% accuracy): Accounting, Business Administration, Political Science
- Other high accuracy: International Relations (99.65%), Mathematics (99.62%), Mechanical Engineering (99.61%)
- Lowest accuracy: Information Technology (90.64%), Sociology (93.30%), Health Sciences (97.03%)
Accuracy by Word Count
- 400 words: 98.20% (11,329 correct / 11,537 words)
- 800 words: 98.99% (23,032 / 23,267)
- 1200 words: 98.40% (39,065 / 39,701)