Deep Learning Approaches For AI-generated Content Detection: From Benchmark Models To Masked Convolutional Self-Attention (MCSA)

Song, Zhiben

Deep Learning Approaches For AI-generated Content Detection: From Benchmark Models To Masked Convolutional Self-Attention (MCSA)

Song, Zhiben

Permalink

Publication Type:: Thesis
Issue Date:: 2025

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download thesisAdobe PDF (1.15 MB)

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Song, Zhiben
dc.date.accessioned	2026-10-26T23:48:01Z
dc.date.available	2026-10-26T23:48:01Z
dc.date.issued	2025
dc.identifier.uri	http://hdl.handle.net/10453/197255
dc.description	University of Technology Sydney. Faculty of Engineering and Information Technology.	en_US.UTF-8
dc.description.abstract	With the widespread adoption of large language models, distinguishing between AI-generated and human-written text has become a critical task across fields such as academic integrity, cybersecurity, digital forensics, and content moderation. This thesis investigates and compares various deep learning-based architectures for the detection of AI-generated content, aiming to evaluate their effectiveness and limitations in classifying textual data based on its origin. The research begins with a comprehensive review of prior studies in AI-generated content (AIGC) detection, identifying the key challenges, methodological trends, and evaluation metrics. Building upon these insights, this work implements and systematically evaluates a set of neural network models, including Recurrent Neural Networks (RNN), Long Short-Term Memory networks (LSTM), LSTM with Attention, Self-Attention mechanisms, Convolutional Neural Networks (CNN), and Recurrent Convolutional Neural Networks (RCNN). Motivated by the comparative findings, we further introduce a lightweight hybrid detector, Masked Convolutional Self-Attention (MCSA), which fuses multi-scale convolutional features with a self-attention encoder via position-wise gated fusion. Each model is trained and tested on benchmark datasets containing both human-written and AI-generated text, and performance is measured using multiple classification metrics such as accuracy, precision, recall, and F1-score, under a unified protocol that also includes mild robustness probes. The evaluation reveals how different architectures handle the nuanced linguistic features and statistical patterns of AI-generated text. Attention-based models, particularly those incorporating self-attention or hybrid convolutional-recurrent structures, demonstrate improved capability in capturing subtle contextual cues, outperforming baseline models on various metrics. Within this landscape, MCSA remains competitive on standard accuracy while narrowing robustness gaps-e.g., under word-order perturbations and train-test mismatch-benefiting from local n-gram/subword cues (via convolutions) and global dependencies (via self-attention). The analysis also uncovers specific limitations, such as reduced performance on shorter texts and sensitivity to adversarial perturbations. By presenting a comparative study of state-of-the-art models for AIGC detection and introducing MCSA as a balanced hybrid, this thesis contributes valuable empirical evidence and methodological insights for advancing detection robustness and reliability. The findings not only inform future development of AI content detectors but also support broader efforts in safeguarding digital spaces against manipulation, deception, and academic dishonesty in the era of generative AI.	en_US.UTF-8
dc.format	Thesis (MSc(CompSc))
dc.language.iso	en_US	en_US.UTF-8
dc.relation	https://opus.lib.uts.edu.au/bitstream/10453/197255/1/thesis.pdf
dc.rights	info:eu-repo/semantics/openAccess
dc.rights	The author owns the copyright in this thesis including all reproduction and reuse rights for the work. The work may not be altered without the permission of the copyright owner. Attribution is essential when quoting or paraphrasing from this thesis.
dc.rights	© 2025 Zhiben Song
dc.rights	au.edu.uts.lib/cph
dc.title	Deep Learning Approaches For AI-generated Content Detection: From Benchmark Models To Masked Convolutional Self-Attention (MCSA)	en_US.UTF-8
dc.type	Thesis
utslib.copyright.status	open_access	*

Abstract:

With the widespread adoption of large language models, distinguishing between AI-generated and human-written text has become a critical task across fields such as academic integrity, cybersecurity, digital forensics, and content moderation. This thesis investigates and compares various deep learning-based architectures for the detection of AI-generated content, aiming to evaluate their effectiveness and limitations in classifying textual data based on its origin. The research begins with a comprehensive review of prior studies in AI-generated content (AIGC) detection, identifying the key challenges, methodological trends, and evaluation metrics. Building upon these insights, this work implements and systematically evaluates a set of neural network models, including Recurrent Neural Networks (RNN), Long Short-Term Memory networks (LSTM), LSTM with Attention, Self-Attention mechanisms, Convolutional Neural Networks (CNN), and Recurrent Convolutional Neural Networks (RCNN). Motivated by the comparative findings, we further introduce a lightweight hybrid detector, Masked Convolutional Self-Attention (MCSA), which fuses multi-scale convolutional features with a self-attention encoder via position-wise gated fusion. Each model is trained and tested on benchmark datasets containing both human-written and AI-generated text, and performance is measured using multiple classification metrics such as accuracy, precision, recall, and F1-score, under a unified protocol that also includes mild robustness probes. The evaluation reveals how different architectures handle the nuanced linguistic features and statistical patterns of AI-generated text. Attention-based models, particularly those incorporating self-attention or hybrid convolutional-recurrent structures, demonstrate improved capability in capturing subtle contextual cues, outperforming baseline models on various metrics. Within this landscape, MCSA remains competitive on standard accuracy while narrowing robustness gaps-e.g., under word-order perturbations and train-test mismatch-benefiting from local n-gram/subword cues (via convolutions) and global dependencies (via self-attention). The analysis also uncovers specific limitations, such as reduced performance on shorter texts and sensitivity to adversarial perturbations. By presenting a comparative study of state-of-the-art models for AIGC detection and introducing MCSA as a balanced hybrid, this thesis contributes valuable empirical evidence and methodological insights for advancing detection robustness and reliability. The findings not only inform future development of AI content detectors but also support broader efforts in safeguarding digital spaces against manipulation, deception, and academic dishonesty in the era of generative AI.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/197255