International Journal of Drug Delivery Technology
Volume 16, Issue 10s, 2026

An Explainable Transformer-Based Framework for Software Defect Prediction Using QLoRA and Semantic Evaluation

1* Vijya Tulsani, 2 Kinjal Patni, 3 Dr. Prashant Sahatiya

1Parul Institute of Computer Applications, Faculty of IT & Computer Science, Parul University. Email: vijya.tulsani42087@paruluniversity.ac.in

2Centre for Distance and Online Education, Parul University. Email: kinjal.patni29421@paruluniversity.ac.in

3Centre for Distance and Online Education, Parul University. Email: prashant.sahatiya30784@paruluniversity.ac.in

* Correspondence: vijya.tulsani42087@paruluniversity.ac.in
Academic Editor: Firstname Lastname
Received: date | Revised: date | Accepted: date | Published: date
Citation: To be added by editorial staff during production.
Copyright: © 2025 by the authors. Submitted for possible open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Abstract

Software Defect Prediction (SDP) is critical to improving software reliability by identifying fault-prone components during development. Despite progress in deep learning, current models often lack interpretability and require substantial computational resources. This study proposes an explainable, Transformer-based framework fine-tuned using Quantized Low-Rank Adaptation (QLoRA) to enhance both efficiency and transparency in defect prediction. The model leverages a custom dataset comprising over 2.3 million labeled code lines—including 292,064 faulty lines—collected from real-world software projects. Byte Pair Encoding (BPE) tokenization and static metric extraction were applied to build a unified feature representation, enabling line-level classification. The Transformer backbone, adapted from the LLaMA model, was fine-tuned using 4-bit quantization for parameter-efficient training. Semantic evaluation was performed using BERTScore to quantify alignment between predicted and actual defect explanations. Experimental results demonstrate that the proposed model achieves a BERTScore of 0.82 and outperforms baseline BiLSTM and vanilla Transformer architectures in both Recall and F1-score, while operating efficiently on consumer-grade hardware. The findings confirm that integrating QLoRA and semantic interpretability mechanisms enables scalable, explainable defect prediction suitable for modern software engineering pipelines.

Keywords: Software Defect Prediction; Transformer Architecture; QLoRA Fine-Tuning; Explainable Artificial Intelligence; BERTScore; Tokenization; LLaMA Model; Semantic Evaluation; Static Code Metrics

How to cite this article: Tulsani V, Patni K, Sahatiya P. An Explainable Transformer-Based Framework for Software Defect Prediction Using QLoRA and Semantic Evaluation. Int J Drug Deliv Technol. 2026;16(10s): 20-29; DOI: 10.25258/ijddt.16.10s.4

Source of support: Nil.

Conflict of interest: None