Lung cancer remains one of the most critical global public health challenges due to its high incidence, aggressive progression, and poor prognosis when diagnosed at advanced stages. It is consistently ranked among the leading causes of cancer-related morbidity and mortality worldwide, accounting for a substantial proportion of all cancer incidences (Wu et al., 2024). According to estimates from the World Health Organization (WHO), more than 1.8 million deaths occur annually due to lung cancer, making it one of the deadliest malignancies globally (Tu et al., 2026; Li et al., 2023; Thandra et al., 2021). The primary reason for this high mortality rate is delayed diagnosis, as the disease is often detected only after it has reached an advanced or metastatic stage. Early-stage detection significantly improves survival rates; however, identifying lung cancer at its earliest stage remains a major clinical challenge (Habbab et al., 2025). This challenge is further compounded by the subtle morphological differences between benign and malignant tissues in histopathological images, which require expert interpretation and are often difficult to distinguish even for experienced pathologists.

The conventional diagnostic process for lung cancer heavily relies on histopathological examination, where tissue samples are observed under a microscope by trained pathologists (Davri et al., 2023). Although this method is considered the diagnostic gold standard, it is inherently subjective, time-consuming, and prone to inter-observer and intra-observer variability. The increasing workload in pathology laboratories further compounds these limitations, often leading to diagnostic delays and potential inconsistencies in interpretation (Faria et al., 2025). As the number of biopsy samples continues to rise globally, there is an increasing need for automated systems that can assist pathologists in improving diagnostic efficiency and accuracy.

The burden of lung cancer is particularly concerning in developing countries such as Bangladesh, where environmental and infrastructural challenges significantly impact public health outcomes. Rapid urbanization and industrialization have led to severe air pollution in major cities, with Dhaka frequently ranking among the most polluted cities in the world according to global air quality indices (Rahman et al., 2025). Prolonged exposure to harmful pollutants such as particulate matter (PM_2.5), carbon monoxide, and other toxic gases has been strongly associated with respiratory diseases and an increased risk of lung cancer. As a result, lung-related illnesses are becoming increasingly common, and many cases are diagnosed at advanced stages due to limited access to early screening facilities and insufficient awareness (Wang et al., 2025). This highlights a critical gap in healthcare systems and underscores the urgent need for reliable and scalable diagnostic tools.

In recent years, artificial intelligence (AI) and machine learning (ML) have emerged as transformative technologies in medical image analysis. In particular, deep learning techniques, especially convolutional neural networks (CNNs), have demonstrated exceptional capability in automatically learning complex hierarchical features from images without requiring manual feature engineering (Smrity et al., 2026). These models have shown strong performance in various medical imaging tasks, including tumour detection, organ segmentation, and disease classification. Their ability to analyse large volumes of data efficiently makes them particularly suitable for histopathological image analysis, where subtle morphological differences between benign and malignant tissues must be identified with high precision (Ahmad et al., 2025; Rayed et al., 2024).

Despite strong progress in deep learning for lung cancer diagnosis, several key limitations still restrict clinical translation. The most important issue is poor generalizability, as many high-performing models reported in different studies that are trained and tested on relatively clean datasets like LC15000 or LC25000 (Shweikeh et al., 2026; Durgam et al., 2025). In real clinical environments, however, histopathological images vary widely due to differences in staining protocols, imaging quality, and the presence of rare tumour subtypes. Without multi-centre or heterogeneous dataset validation, the reliability of these models in practical deployment remains uncertain (Raza et al., 2026). Another major limitation is the lack of interpretability in many deep learning systems. Although some works, such as Garg and Garg, use techniques like Grad-CAM to highlight important image regions, most high-accuracy models still function as black-box systems (Ennab and Mcheick, 2025; Henna et al., 2025). In medical applications, this lack of transparency is a serious barrier because clinicians need to understand and trust model decisions before integrating AI tools into diagnostic workflows.

In addition, the complexity of many proposed deep learning pipelines presents a barrier to real-world clinical deployment. Several studies rely on multi-stage architectures combining deep feature extraction with ensemble classifiers or hybrid models (Ochoa-Ornelas et al., 2024). While these approaches often yield high performance, they may not be practical in hospital environments where computational resources, processing time, and system simplicity are critical considerations. A clinically viable model must balance accuracy with efficiency and ease of integration into existing workflows (Ali et al., 2025).

There is still limited progress in extending deep learning applications beyond simple binary or categorical classification tasks. More advanced clinical applications such as prognosis prediction, survival analysis, and drug response estimation remain relatively underexplored. Although some recent studies, such as Zhang et al. and Pan et al., have made progress in areas like contrastive clustering and mutation prediction, these approaches are still in early stages of development (Pan et al., 2025; Großerueschkamp and Gerwert, 2020). This represents a significant opportunity for future research to move toward more clinically meaningful outcomes that support personalized cancer treatment and precision oncology.

This study addresses several research questions: How do pre-trained CNN architectures (VGG16, MobileNetV3-Large, and EfficientNetB5) perform on the LC15000 dataset with and without data augmentation? Can a hybrid CNN-BiLSTM model improve lung cancer classification performance compared to standard CNN models? Does data augmentation enhance the generalization ability of deep learning models in histopathological image classification? Which model provides the most robust and clinically reliable performance for this task? Based on these questions, it is hypothesized that data augmentation improves model generalization, and that the hybrid CNN-BiLSTM model achieves superior performance over traditional CNN architectures in terms of accuracy and overall classification metrics.

This study addresses these challenges by systematically evaluating multiple CNN architectures under both augmented and non-augmented conditions using the LC15000 dataset. The primary aim is to identify robust, generalizable, and efficient models for lung cancer classification. By comparing classical, lightweight, and advanced architectures, as well as introducing a hybrid CNN-BiLSTM model, this research seeks to contribute to the development of more reliable and clinically applicable AI-based diagnostic systems. The findings are expected to provide insights into model performance, generalization behaviour, and the role of data augmentation in improving robustness, ultimately supporting the advancement of automated histopathological analysis in lung cancer diagnosis.

2. Literature review

The rapid growth of AI in medical imaging has significantly transformed the field of histopathological analysis, particularly in cancer diagnosis. Lung cancer detection, in particular, has benefited from deep learning approaches due to the increasing demand for fast, accurate, and reproducible diagnostic systems. Traditional histopathological examination relies on manual microscopic evaluation by expert pathologists, which is inherently subjective, time-consuming, and susceptible to inter-observer and intra-observer variability. These limitations have motivated the development of automated systems capable of assisting in diagnosis and improving consistency and accuracy in clinical decision-making (Arshad et al., 2025; Zhu et al., 2025).

Deep learning, especially CNNs, has emerged as the dominant approach for medical image analysis due to its ability to automatically extract hierarchical feature representations without manual feature engineering. Models such as VGGNet, ResNet, Inception, and EfficientNet have demonstrated strong performance across a variety of medical imaging tasks, including tumor classification and organ segmentation (Gayap and Akhloufi, 2024). Among these, EfficientNet has gained significant attention due to its compound scaling strategy, which efficiently balances network depth, width, and resolution, achieving state-of-the-art performance in many image classification benchmarks (Lin et al., 2023). Transfer learning has also played a crucial role in medical imaging applications, where models pre-trained on large-scale natural image datasets such as ImageNet are fine-tuned on domain-specific medical datasets, enabling improved feature learning even with limited labelled data (Wen et al., 2021).

In addition to model architecture advancements, data augmentation techniques have been widely used to improve generalization and reduce overfitting in deep learning models. Common augmentation strategies such as rotation, flipping, scaling, shifting, and brightness variation artificially increase dataset diversity and simulate real-world variability in medical images (Goceri, 2023). Although augmentation generally improves model robustness, its impact is not always consistent across different architectures and datasets, particularly in histopathological imaging where subtle texture differences are clinically significant (Lerch et al., 2024).

More recently, hybrid deep learning approaches have been explored to enhance classification performance by combining CNNs with sequential learning models such as Long Short-Term Memory (LSTM) networks. These hybrid architectures aim to capture both spatial feature representations and contextual dependencies within feature maps, offering improved discriminatory capability in complex medical imaging tasks (Hayat et al., 2025). Despite these advancements, most existing studies still rely on standalone CNN architectures or handcrafted feature-based pipelines, with limited comparative analysis across multiple models under standardized experimental conditions (Raza et al., 2026). This creates a clear research gap in evaluating how different architectures perform under consistent datasets and training settings, particularly when considering augmentation effects.

Several prior studies have contributed significantly to lung cancer classification using histopathological images. For example, previous studies developed a custom CNN model trained on the LC15000 dataset, achieving very high classification accuracy by leveraging deep convolutional layers to automatically learn discriminative tissue features (Attallah, 2025; Uddin et al., 2024). However, the study lacked interpretability mechanisms such as Grad-CAM, limiting clinical transparency. Similarly, Talukder et al. (2022) combined pretrained CNN feature extractors with ensemble classifiers, demonstrating high performance on the LC25000 dataset, though the complexity of the multi-stage pipeline raised concerns regarding clinical deployability (Ochoa-Ornelas et al., 2025).

Garg and Garg (2020) investigated multiple pre-trained CNN models including ResNet50, DenseNet169, and Xception, and incorporated Grad-CAM visualization to enhance interpretability. Their study highlighted the importance of explainability in clinical AI systems, although performance varied across architectures depending on transfer learning compatibility (Abraham et al., 2025). In a broader context, Parra-Medina et al. (2025) conducted a systematic review on deep learning models for predicting oncogenic drivers in non-small cell lung cancer, reporting moderate to high predictive performance but emphasizing the need for multi-center validation to ensure generalizability (Fuster-Matanzo et al., 2025).

Further advancements have extended deep learning applications beyond classification tasks. Großerueschkamp and Gerwert (2020) proposed a contrastive clustering framework for prognosis and drug response prediction, demonstrating the potential of deep learning in precision oncology beyond diagnostic classification. Similarly, Abd El-Aziz et al. (2024) introduced a multi-CNN fusion model combining ResNet101V2, NASNetMobile, and EfficientNet-B0, achieving near-perfect classification performance on histopathological datasets, although concerns remain regarding external validation and real-world robustness. Recent developments also include dataset-driven and multi-instance learning approaches. Pan et al. (2026) introduced PathGene, a multicentre dataset designed for mutation prediction, and demonstrated that deep learning models can predict genetic alterations such as EGFR, KRAS, and TP53 directly from histopathology images. Another study by Pan et al. (2025) proposed SMILE, a scale-aware multiple-instance learning framework for detecting spread-through-air-space (STAS) patterns, achieving strong AUC performance across multicentre datasets.

Lightweight and efficient architectures have also been explored to address computational constraints in clinical deployment. For instance, Pang et al. (2022) proposed DCSNet, a lightweight model based on knowledge distillation that achieved high classification accuracy with reduced computational cost and improved interpretability through attention-based visualization. Such approaches are particularly relevant for real-world deployment in resource-limited healthcare settings.

Despite these advancements, several research gaps remain evident. First, most existing models demonstrate high accuracy on curated datasets such as LC15000 and LC25000, but their generalizability to real-world clinical environments remains uncertain due to variations in staining protocols, imaging quality, and biological heterogeneity. Second, interpretability remains limited in many high-performing models, reducing clinical trust and adoption. Third, many studies rely on complex multi-stage or ensemble-based pipelines that are difficult to deploy in real-time clinical workflows. Finally, there is limited comparative analysis of multiple CNN architectures under consistent experimental settings, particularly with systematic evaluation of data augmentation effects.

To address these gaps, this study proposes a comprehensive comparative analysis of multiple deep learning architectures, including VGG16, MobileNetV3-Large, EfficientNetB5, and a hybrid CNN-BiLSTM model, using the LC15000 dataset. The study also systematically evaluates the impact of data augmentation on model performance and generalization (Table 1). The proposed hybrid architecture integrates spatial feature extraction from CNNs with sequential learning from BiLSTM networks to improve classification robustness. This dual experimental design enables a fair comparison of classical, lightweight, and advanced architectures under standardized conditions, contributing to the development of more reliable and clinically applicable AI-based diagnostic systems for lung cancer classification. A summary of related works on deep learning applications in lung cancer histopathol¬ogy is presented in following Table 1.

Table 1. Summary of deep learning-based studies for lung cancer diagnosis using histopathological images.

3. Methodology

3.1 Workflow

The research follows a structured and reproducible experimental pipeline designed to evaluate multiple deep learning architectures for lung cancer histopathological image classification. The workflow spans dataset acquisition, preprocessing, augmentation, model development, training, evaluation, and comparative analysis, ensuring consistency across all experimental settings (Figure 1).

Figure 1. Workflow diagram of methodology.

3.2 Dataset description and acquisition

The LC15000 histopathological image dataset is used in this study. It contains 15,000 high-resolution lung tissue images, equally distributed into three categories: 5000 adenocarcinoma of lung, 5000 squamous-cell carcinoma of lung, and 5000 normal lung tissue images. The dataset is publicly available and widely used in lung cancer image analysis research due to its balance and quality. For the purpose of this study, the dataset is reformulated into a binary classification task where adenocarcinoma and squamous-cell carcinoma are combined into a single malignant class, while normal tissue represents the benign class. This transformation aligns the dataset with clinically relevant diagnostic requirements for malignancy screening.

3.3 Data preprocessing

A standardized preprocessing pipeline is applied to ensure compatibility with pre-trained CNN architectures and to improve numerical stability during training. All images are resized to 224 × 224 pixels, which matches the input requirements of VGG16, MobileNetV3-Large, and EfficientNetB5. Pixel values are normalized from the original range of 0–255 to a scaled range of 0–1 to stabilize gradient updates and accelerate convergence. Label encoding is performed by converting categorical class labels into binary format, where benign is represented as 0 and malignant as 1. The dataset is shuffled to eliminate ordering bias and then split into training (70%), validation (20%), and testing (10%) subsets. This stratified split ensures balanced class distribution across all subsets and prevents sampling bias during evaluation (Table 2).

Table 2. Dataset distribution across training, validation, and testing sets.

3.4 Data augmentation strategy

To evaluate the effect of data variability on model generalization, two parallel experimental branches are created. In the augmentation branch, real-time transformations are applied only to the training dataset, while validation and test sets remain unchanged to ensure unbiased evaluation. The augmentation techniques include random rotation within ±20 degrees, horizontal flipping, zooming between 0.8 and 1.2, width and height shifts up to ±10%, shear transformation with a range of 0.2, and nearest-neighbor fill mode to preserve image structure.

3.5 Model architectures and implementation

Four deep learning architectures are implemented and evaluated: VGG16, MobileNetV3-Large, EfficientNetB5, and a custom hybrid CNN-BiLSTM model. All models are initialized with ImageNet pretrained weights and fine-tuned for binary classification on the LC15000 dataset. VGG16 consists of 13 convolutional layers and 3 fully connected layers with 3×3 convolutional filters and max pooling operations, making it a deep but computationally expensive baseline model. MobileNetV3-Large incorporates depthwise separable convolutions, squeeze-and-excitation blocks, and the Hard-Swish activation function, making it lightweight and suitable for efficient computation. EfficientNetB5 employs compound scaling of depth, width, and resolution and integrates MBConv blocks with squeeze-and-excitation mechanisms for high representational capacity. The hybrid model integrates MobileNetV3-Small with a Bi-directional Long Short-Term Memory (BiLSTM) network to combine spatial feature extraction with sequential dependency modeling. For all models, the original classification layers are removed and replaced with a global average pooling layer, followed by fully connected dense layers, dropout regularization, and a sigmoid activation layer for binary classification (Figure 2).

Figure 2. Architecture of the proposed hybrid CNN-BiLSTM model.

3.6 Training procedure

All models are trained using a standardized deep learning pipeline implemented in TensorFlow with a Keras backend. Training is conducted under identical conditions to ensure fair comparison. The Adam optimizer is used with an initial learning rate of 1 × 10⁻⁴, and binary cross-entropy is used as the loss function due to the binary classification setup. A batch size of 32 is used across all experiments. Models are trained for a maximum of 50 epochs; however, early stopping is applied with a patience of 5 epochs based on validation loss to prevent overfitting. Model checkpoints are used to retain the best-performing weights during training. Real-time augmentation is applied only in the augmented branch using Keras ImageDataGenerator. In the non-augmented branch, only rescaling is applied (1/255 normalization). A fixed random seed is used to ensure reproducibility, and identical data splits are maintained across all experiments.

3.7 Validation and testing strategy

The dataset is partitioned into training (70%), validation (20%), and testing (10%) subsets, ensuring no overlap between sets. The validation set is used during training to monitor model performance, guide early stopping, and prevent overfitting. The test set is completely isolated and used only for final performance evaluation. The validation set consists of 20% of the dataset (3,000 images), while the test set consists of 10% (1,500 images).

3.8 Regularization techniques

Multiple regularization strategies are incorporated to improve generalization and reduce overfitting. Dropout is applied in fully connected layers across all models, with dropout rates ranging from 0.3 in lightweight architectures such as MobileNetV3-Large to 0.5 in deeper models such as VGG16, EfficientNetB5, and the hybrid model. Data augmentation acts as an additional regularization mechanism by introducing controlled variability in the training data. Early stopping is applied based on validation loss with a patience of 5 epochs, ensuring training halts at optimal convergence. Light L2 weight regularization is selectively applied in the hybrid model to penalize overly complex weight distributions and improve stability.

3.9 Evaluation methodology

Model evaluation is conducted exclusively on the test set, which contains 10% of the dataset (1,500 images) and remains unseen during training and validation. Each model is evaluated under both experimental conditions: with augmentation and without augmentation, resulting in eight total experimental configurations. Performance is measured using accuracy, precision, recall, F1-score, binary cross-entropy loss, confusion matrix, and ROC-AUC curves. Accuracy measures overall correctness, while precision and recall assess class-specific predictive performance. The F1-score provides a balanced measure of precision and recall, particularly important in medical classification tasks where false negatives are critical. The confusion matrix is used to analyze misclassification patterns, while ROC-AUC evaluates the trade-off between sensitivity and specificity. All metrics are computed using TensorFlow/Keras evaluation functions and systematically recorded for comparative analysis across models and experimental conditions.

4. Results

4.1 Performance evaluation and model performance comparison

The evaluation was conducted using a test dataset comprising 2,250 histopathological images, representing 15% of the total LC15000 dataset. All samples were strictly excluded from the training and validation phases to ensure unbiased performance assessment. Each model was evaluated under two experimental conditions, with and without data augmentation, using identical preprocessing pipelines and training configurations to maintain comparability.

The performance was assessed using test accuracy, precision, recall, F1-score, and test loss computed via binary cross-entropy or categorical cross-entropy depending on the model structure. In addition, validation accuracy and validation loss were monitored during training to assess convergence behavior and detect overfitting. Confusion matrices and ROC curves were used to evaluate class-wise performance and discriminative capability. Without augmentation, EfficientNetB5 achieved the highest performance with an accuracy of 0.996, followed by VGG16 and the Hybrid model (0.9913), while MobileNetV3-Large recorded 0.9907 (Table 3).

Table 3. Performance of CNN models without data augmentation on LC15000 dataset (accuracy, precision, recall, F1-score).

Under augmentation, the Hybrid CNN-BiLSTM model achieved the best performance with an accuracy of 0.9967, followed by VGG16 (0.988), EfficientNetB5 (0.9773), and MobileNetV3-Large (0.9727) (Table 4).

Table 4. Performance of CNN models with data augmentation on LC15000 dataset (accuracy, precision, recall, F1-score).

4.2 VGG16 performance analysis

The VGG16 model was evaluated using ImageNet-pretrained weights with a modified binary classification head. Training was conducted for up to 20 epochs with early stopping based on validation loss. VGG16 demonstrated fast convergence in the non-augmented setting, with rapid reduction in training and validation loss during early epochs followed by stabilization. In the augmented setting, convergence was slower but smoother, indicating improved generalization and reduced overfitting due to increased data variability. In the non-augmented setting, VGG16 achieved an accuracy of 0.9913 (Figure 3), while in the augmented setting it achieved 0.988 (Figure 4).

Figure 3. Confusion matrix of VGG16 without data augmentation.

Although there was a slight reduction in accuracy with augmentation, the model exhibited improved robustness to unseen variations in histopathological images. In the non-augmented case, the model correctly classified all normal lung tissue samples with zero error, while minor misclassifications occurred between adenocarcinoma and squamous cell carcinoma. In the augmented case, the normal class remained perfectly classified; however, slightly increased confusion was observed between cancer subtypes, indicating a minor reduction in class boundary sharpness due to augmentation-induced variability.

Figure 4. Confusion matrix of VGG16 with data augmentation.

4.3 MobileNetV3-large performance analysis

MobileNetV3-Large, a lightweight CNN architecture optimized for efficiency, was evaluated under both experimental conditions. The model showed stable convergence with steadily increasing training and validation accuracy. Minimal divergence between training and validation curves indicated low overfitting and stable learning behavior across both settings. Without augmentation, MobileNetV3-Large achieved an accuracy of 0.9907, while with augmentation the accuracy dropped to 0.9727 (Figures 5 and 6).

Figure 5. Confusion matrix of MobileNetV3-Large with data augmentation.

Precision, recall, and F1-score followed a similar trend, indicating that augmentation slightly reduced discriminative performance in this architecture. The augmented model showed increased misclassification between adenocarcinoma and squamous cell carcinoma, while the non-augmented model demonstrated better class separation with fewer errors.

Figure 6. Confusion matrix of MobileNetV3-Large without data augmentation.

4.4 EfficientNetB5 performance analysis

EfficientNetB5, based on compound scaling, demonstrated strong feature extraction capability and stable learning behavior. The model exhibited smooth convergence with consistently decreasing loss and stable validation performance. It maintained strong generalization in both augmented and non-augmented settings. EfficientNetB5 achieved 0.996 accuracy without augmentation and 0.9773 with augmentation (Figures 7 and 8).

Figure 7. Confusion matrix of EfficientNetB5 with data augmentation.

Despite the reduction under augmentation, it remained one of the strongest standalone models in this study. In the non-augmented setting, only a very small number of misclassifications occurred. In the augmented setting, limited confusion was observed mainly between adenocarcinoma and squamous cell carcinoma classes, reflecting high but slightly reduced class separability.

Figure 8. Confusion matrix of EfficientNetB5 without data augmentation.

4.5 Hybrid CNN–BiLSTM model performance analysis

The hybrid model combines MobileNetV3Small as a feature extractor with a Bi-directional LSTM layer for sequential feature modeling. The model demonstrated rapid convergence, with validation accuracy stabilizing early in training. The inclusion of BiLSTM layers improved feature interaction learning, resulting in strong class separability. ROC analysis showed an AUC value of 1.00 across all classes, indicating near-perfect discrimination. The hybrid model achieved an accuracy of 0.9913 without augmentation and 0.9967 with augmentation, making it the best-performing model under augmented conditions (Figures 9 and 10).

Figure 9. Confusion matrix of the hybrid model without data augmentation.

Precision, recall, and F1-score followed similar trends, indicating balanced and stable classification performance. The augmented hybrid model showed minimal misclassification between adenocarcinoma and squamous cell carcinoma, while maintaining strong performance across all classes.

Figure 10. Confusion matrix of the hybrid model with data augmentation.

ROC analysis confirmed perfect separability with AUC = 1.00, indicating highly reliable classification performance (Figure 11).

Figure 11. Training and validation accuracy/loss curves of the hybrid model with augmentation.

4.6 Impact of data augmentation

Data augmentation had a measurable influence on both model generalization and convergence behavior. Although augmentation slightly reduced peak accuracy in some models, it consistently improved robustness and reduced overfitting. In non-augmented settings, models such as VGG16 and EfficientNetB5 achieved faster convergence and lower training loss but showed higher sensitivity to overfitting. In contrast, augmented models demonstrated smoother training curves and closer alignment between training and validation performance, particularly in EfficientNetB5 and the hybrid model.

5. Discussion

The results demonstrate that deep learning-based architectures can achieve very high performance in histopathological lung cancer classification using the LC15000 dataset. Across all experiments, both classical and modern CNN architectures consistently achieved accuracy above 97%, confirming the strong suitability of convolutional feature extraction for histopathological image analysis. The consistently high performance aligns with previous studies reporting strong results of CNN-based models in lung cancer classification tasks (Abd El-Aziz et al., 2024; Mohalder et al., 2023).

However, a key observation from this study is that model ranking is not fixed but varies depending on whether data augmentation is applied. In non-augmented settings, EfficientNetB5 performed best, indicating that its compound scaling strategy is highly effective in learning discriminative histopathological features from relatively clean and well-structured datasets. In contrast, under augmented conditions, the hybrid CNN–BiLSTM model achieved the best performance, suggesting that feature fusion and sequential modeling improve robustness when the data distribution becomes more diverse (Shafay et al., 2025). This shift in performance ranking highlights an important methodological insight: model superiority in medical imaging is context-dependent and strongly influenced by preprocessing strategy and data variability rather than architecture alone.

VGG16 showed strong baseline performance in both experimental conditions, confirming its reliability as a classical deep learning architecture for medical image classification. The slight reduction in accuracy after augmentation suggests that while VGG16 benefits from increased data diversity, it may also be sensitive to excessive transformation of fine-grained histopathological features (Hussain et al., 2024). The confusion matrix patterns indicate that VGG16 consistently classified normal lung tissue with perfect or near-perfect accuracy, which reflects strong feature separability between normal and diseased tissues. However, misclassifications between adenocarcinoma and squamous cell carcinoma remained present in both settings. This suggests that these two classes share overlapping morphological characteristics that are difficult to distinguish even with deep convolutional filters (Nayeem et al., 2025; Shahadat et al., 2024). The observed trade-off between accuracy and robustness is consistent with earlier findings that deeper CNNs can overfit to subtle texture-level differences when trained on limited or moderately diverse datasets (Martins et al., 2026).

MobileNetV3-Large demonstrated stable learning behavior and efficient convergence across both experimental settings, reflecting its design goal of computational efficiency. Despite its lightweight structure, it achieved competitive accuracy close to deeper models in the non-augmented scenario, confirming that efficient architectures can still capture meaningful histopathological features (Rahaman et al., 2026). However, the noticeable drop in performance under augmentation indicates that MobileNetV3-Large is more sensitive to aggressive data transformations. The increased misclassification between adenocarcinoma and squamous cell carcinoma suggests that lightweight architectures may have limited capacity to retain fine-grained morphological distinctions when feature distributions are heavily altered (Ochoa-Ornelas et al., 2024). This observation supports prior studies indicating that lightweight CNNs may trade representational capacity for computational efficiency, which can affect their robustness under highly variable training conditions (Raza et al., 2026).

EfficientNetB5 consistently demonstrated strong performance in both settings, particularly in the non-augmented scenario where it achieved the highest overall accuracy. This confirms the effectiveness of compound scaling in balancing network depth, width, and resolution for histopathological feature learning (Abd El-Aziz et al., 2025). The relatively small performance drop under augmentation indicates that EfficientNetB5 maintains strong generalization capability even when exposed to artificially increased data variability. However, the slight increase in misclassification between cancer subtypes suggests that augmentation may introduce subtle feature ambiguity that affects highly sensitive models (Ishaq et al., 2025). These findings are consistent with existing literature where EfficientNet variants have been shown to outperform traditional CNNs in medical imaging due to their optimized scaling strategy (Rahman et al., 2026).

The hybrid CNN–BiLSTM model demonstrated the most interesting behavior among all evaluated architectures. Its superior performance under augmented conditions suggests that combining spatial feature extraction (CNN) with sequential dependency modeling (BiLSTM) enhances the model’s ability to generalize in complex feature spaces. The near-perfect AUC value (1.00) indicates that the hybrid model achieves excellent class separability, which is particularly important in medical diagnostics where minimizing false negatives is critical. The early stabilization of validation accuracy further indicates strong learning efficiency and effective feature fusion. This improvement aligns with recent studies suggesting that hybrid architectures can enhance representation learning by capturing complementary feature dimensions beyond traditional CNNs (Hayat et al., 2025). The slight improvement in performance under augmentation also suggests that the hybrid architecture benefits from increased data diversity more than individual CNN models, likely due to its enhanced capacity to integrate heterogeneous feature representations.

Data augmentation played a crucial role in shaping model generalization behavior across all architectures. While it slightly reduced peak accuracy in some cases, particularly for EfficientNetB5 and MobileNetV3-Large, it consistently improved robustness and reduced overfitting. In non-augmented settings, models exhibited faster convergence and lower training loss, but this was often accompanied by higher sensitivity to training data distribution. In contrast, augmented models showed smoother learning curves and closer alignment between training and validation performance, indicating improved generalization (Wanyonyi et al., 2025). This trade-off between accuracy and generalization is well-documented in deep learning literature, particularly in medical imaging contexts where dataset size and variability are often limited (Goceri, 2023).

From a clinical perspective, the results suggest that deep learning models can provide highly accurate decision-support tools for lung cancer histopathology classification. However, the variability in model performance across augmentation settings highlights the importance of selecting architectures not only based on accuracy but also based on robustness and deployment context. The strong performance of the hybrid model under augmented conditions suggests its potential suitability for real-world clinical environments where data variability is high. Conversely, EfficientNetB5 may be more suitable for controlled or high-quality imaging environments due to its superior performance on clean datasets (Hassan et al., 2026).

6. Conclusions

This study evaluated VGG16, MobileNetV3-Large, EfficientNetB5, and a MobileNetV3Small–BiLSTM hybrid model for lung cancer classification using the LC15000 histopathological dataset under with and without data augmentation settings. All models achieved high diagnostic performance, with EfficientNetB5 performing best without augmentation (accuracy 0.996), while the hybrid model achieved the highest performance under augmentation (accuracy 0.9967, AUC 1.00), indicating strong discriminative capability when spatial and sequential feature learning are combined. Overall, augmentation improved model generalization and reduced overfitting but slightly lowered peak accuracy in some architectures due to increased intra-class variability. EfficientNetB5 proved most effective for high-precision standalone classification, whereas the hybrid model showed superior robustness and generalization in augmented conditions; therefore, a hybrid CNN-based framework with optimized augmentation is recommended for practical histopathological lung cancer diagnosis.