Explainable End-to-End Autonomous Driving Using Vision-Based Deep Learning in Safety-Critical Scenarios
DOI:
https://doi.org/10.70062/globalscience.v1i4.185Keywords:
End-to-End Driving, Explainable AI, Vision-Based Control, Safety-Critical Systems, Grad-CAM AnalysisAbstract
End-to-end autonomous driving has emerged as a promising paradigm in which deep neural networks directly map raw visual inputs to continuous control actions. Despite its effectiveness, this approach suffers from limited transparency, posing significant challenges for deployment in safety-critical driving scenarios. This study addresses the lack of interpretability in vision-based end-to-end autonomous driving systems and aims to analyze model decision-making behavior under critical conditions such as sharp steering maneuvers and abrupt control transitions. To this end, an explainable end-to-end autonomous driving framework is proposed, combining a convolutional neural network trained via imitation learning with gradient-based visual attribution techniques, including Grad-CAM. The model predicts continuous steering, throttle, and braking commands directly from front-facing camera images, while explainability mechanisms are applied to reveal input regions influencing each control decision. Model performance is evaluated using both prediction accuracy and safety-oriented behavioral metrics. Experimental results show that the proposed explainable model achieves lower control prediction errors compared to a baseline end-to-end CNN, reducing steering mean squared error from 0.034 to 0.031, throttle error from 0.021 to 0.019, and brake error from 0.018 to 0.016. Moreover, safety-oriented analysis indicates improved driving stability, with steering variance reduced from 0.087 to 0.072 and abrupt control changes decreased from 14.6 to 10.3 events. Visual explanations consistently highlight road surfaces and lane-related structures during complex maneuvers, indicating reliance on semantically meaningful cues. In conclusion, the results demonstrate that integrating explainability into end-to-end autonomous driving not only preserves predictive performance but also correlates with smoother and more stable driving behavior. This framework contributes to the development of transparent and trustworthy autonomous driving systems suitable for safety-critical applications
References
Azam, S., Munir, F., Kyrki, V., Kucner, T. P., Jeon, M., & Pedrycz, W. (2024). Exploring Contextual Representation and Multi-modality for End-to-end Autonomous Driving. Engineering Applications of Artificial Intelligence, 135, 108767. https://doi.org/https://doi.org/10.1016/j.engappai.2024.108767
Ben Elallid, B., Benamar, N., Bagaa, M., & Mrani, N. (2025). Secure and efficient vehicle control of autonomous vehicles using federated deep reinforcement learning. Applied Soft Computing, 185, 113924. https://doi.org/https://doi.org/10.1016/j.asoc.2025.113924
Candela, E., Doustaly, O., Parada, L., Feng, F., Demiris, Y., & Angeloudis, P. (2023). Risk-aware controller for autonomous vehicles using model-based collision prediction and reinforcement learning. Artificial Intelligence, 320, 103923. https://doi.org/https://doi.org/10.1016/j.artint.2023.103923
Chaurasia, V., Tiwari, A. N., & Tripathi, S. M. (2024). Optimal hybrid strategy in adaptive cruise control system for enhanced autonomous vehicle stability and safety. Computers and Electrical Engineering, 118, 109377. https://doi.org/https://doi.org/10.1016/j.compeleceng.2024.109377
Chen, G., Gao, Z., Hua, M., Shuai, B., & Gao, Z. (2024). Lane change trajectory prediction considering driving style uncertainty for autonomous vehicles. Mechanical Systems and Signal Processing, 206, 110854. https://doi.org/https://doi.org/10.1016/j.ymssp.2023.110854
Du, D., Bi, M., Xie, Y., Liu, Y., Qi, G., & Guo, Y. (2026). MLE-YOLO: A lightweight and robust vehicle and pedestrian detector for adverse weather in autonomous driving. Digital Signal Processing, 168, 105628. https://doi.org/https://doi.org/10.1016/j.dsp.2025.105628
ELallid, B. Ben, Benamar, N., Bagaa, M., & Mrani, N. (2025). Enhancing Autonomous Vehicle control in complex scenarios with Deep Transformer Reinforcement Learning. Engineering Applications of Artificial Intelligence, 158, 111483. https://doi.org/https://doi.org/10.1016/j.engappai.2025.111483
Evans, B. D., Jordaan, H. W., & Engelbrecht, H. A. (2023). Comparing deep reinforcement learning architectures for autonomous racing. Machine Learning with Applications, 14, 100496. https://doi.org/https://doi.org/10.1016/j.mlwa.2023.100496
Fan, J., Zhang, X., Zou, Y., Li, Y., Liu, Y., & Sun, W. (2024). Improving policy training for autonomous driving through randomized ensembled double Q-learning with Transformer encoder feature evaluation. Applied Soft Computing, 167, 112386. https://doi.org/https://doi.org/10.1016/j.asoc.2024.112386
Freire, M., Maia, A., Figueiredo, G., Prazeres, C., Lobato, W., Villas, L., Sommer, C., & Peixoto, M. (2025). Clear data, clear roads: Imputing missing data for enhanced intersection flow of connected autonomous vehicles. Journal of Network and Computer Applications, 242, 104233. https://doi.org/https://doi.org/10.1016/j.jnca.2025.104233
Görçün, Ö. F., Özçalıcı, M., Gurler, H. E., Pamucar, D., & Simic, V. (2025). Promoting sustainable urban mobility: An integrated fuzzy decision-making model for assessing autonomous bus alternatives. Engineering Applications of Artificial Intelligence, 156, 111034. https://doi.org/https://doi.org/10.1016/j.engappai.2025.111034
Guo, X.-Y., Zhang, G., & Jia, A.-F. (2023). Study on mixed traffic of autonomous vehicles and human-driven vehicles with different cyber interaction approaches. Vehicular Communications, 39, 100550. https://doi.org/https://doi.org/10.1016/j.vehcom.2022.100550
Gutiérrez-Zaballa, J., Basterretxea, K., Echanobe, J., Martínez, M. V., Martinez-Corral, U., Mata-Carballeira, Ó., & del Campo, I. (2023). On-chip hyperspectral image segmentation with fully convolutional networks for scene understanding in autonomous driving. Journal of Systems Architecture, 139, 102878. https://doi.org/https://doi.org/10.1016/j.sysarc.2023.102878
Huang, H., Cheng, H., Zhou, Z., Wang, Z., Wang, H., Liu, Q., & Li, X. (2026). REACT: Runtime-Enabled active collision-avoidance technique for autonomous driving. Advanced Engineering Informatics, 71, 104248. https://doi.org/https://doi.org/10.1016/j.aei.2025.104248
Huang, J., He, Z., Liu, T., & Hu, L. (2026). A multimodal trajectory prediction framework with multi-scale spatio-temporal interaction and multi-stage adaptive decoding for autonomous vehicles. Advanced Engineering Informatics, 70, 104178. https://doi.org/https://doi.org/10.1016/j.aei.2025.104178
Huang, L., & Wu, Y. (2025). Ethically Responsible Decision Making for Anomaly Detection in Complex Driving Scenes. Information Fusion, 122, 103226. https://doi.org/https://doi.org/10.1016/j.inffus.2025.103226
Huang, Z., Sun, S., Zhao, J., & Mao, L. (2023). Multi-modal policy fusion for end-to-end autonomous driving. Information Fusion, 98, 101834. https://doi.org/https://doi.org/10.1016/j.inffus.2023.101834
Jiang, H., Lu, Y., Zhang, D., Shi, Y., & Wang, J. (2024). Deep learning-based fusion networks with high-order attention mechanism for 3D object detection in autonomous driving scenarios. Applied Soft Computing, 152, 111253. https://doi.org/https://doi.org/10.1016/j.asoc.2024.111253
Jiang, J.-A., Wang, J.-C., Lin, H.-J., Chou, C.-Y., Chien, C.-C., Huang, W.-H., Liu, A.-C., Lee, C.-C., Guo, M.-C., & Peng, J.-C. (2025). Pilot study on integrating multidisciplinary technologies to implement an autonomous and smart harvester for asparagus. Computers and Electronics in Agriculture, 239, 111137. https://doi.org/https://doi.org/10.1016/j.compag.2025.111137
Khosravian, A., Masih-Tehrani, M., Amirkhani, A., & Ebrahimi-Nejad, S. (2024). Robust autonomous vehicle control by leveraging multi-stage MPC and quantized CNN in HIL Framework. Applied Soft Computing, 162, 111802. https://doi.org/https://doi.org/10.1016/j.asoc.2024.111802
Li, Y., Zhao, Q., Hou, M., Bai, Q., Zou, X., Xie, C., Shu, C., Ma, B., & Li, Z. (2025). D2: An LLM agent driving end-to-end visual AI modeling in energy platforms. Energy and AI, 21, 100582. https://doi.org/https://doi.org/10.1016/j.egyai.2025.100582
Liao, L., Xiao, H., Xing, P., Gan, Z., He, Y., & Wang, J. (2024). Perception Enhanced Deep Deterministic Policy Gradient for Autonomous Driving in Complex Scenarios. CMES - Computer Modeling in Engineering and Sciences, 140(1), 557–576. https://doi.org/https://doi.org/10.32604/cmes.2024.047452
Liu, J., Wang, L., Bie, Y., & Guo, M. (2026). Evaluation of road environment complexity in autonomous driving: a quantitative method. Advanced Engineering Informatics, 69, 104064. https://doi.org/https://doi.org/10.1016/j.aei.2025.104064
Liu, Y., Ziatdinov, M. A., Vasudevan, R. K., & Kalinin, S. V. (2023). Explainability and human intervention in autonomous scanning probe microscopy. Patterns, 4(11), 100858. https://doi.org/https://doi.org/10.1016/j.patter.2023.100858
Martínez, F. S., Parada, R., & Casas-Roma, J. (2023). CO2 impact on convolutional network model training for autonomous driving through behavioral cloning. Advanced Engineering Informatics, 56, 101968. https://doi.org/https://doi.org/10.1016/j.aei.2023.101968
Memon, M. A., Filippone, G., Scoccia, G. L., Autili, M., & Inverardi, P. (2026). RobEthiChor: Automated context-aware ethics-based negotiation for autonomous robots. Journal of Systems and Software, 232, 112692. https://doi.org/https://doi.org/10.1016/j.jss.2025.112692
Mohamed, A. A., & Aslan, H. K. (2025). Toward Safe AI in Autonomous Vehicles: Challenges, Standards, and an Integrated Safety Engineering Framework. Procedia Computer Science, 272, 706–711. https://doi.org/https://doi.org/10.1016/j.procs.2025.10.271
Nguyen, D.-N., Tong, R. K.-Y., & Dinh, N.-D. (2025). DropMicroFluidAgents (DMFAs): autonomous droplet microfluidic research framework through large language model agents. Digital Discovery, 4(10), 2827–2851. https://doi.org/https://doi.org/10.1039/d5dd00306g
Paniego, S., Calvo-Palomino, R., & Cañas, J. (2024). Enhancing end-to-end control in autonomous driving through kinematic-infused and visual memory imitation learning. Neurocomputing, 600, 128161. https://doi.org/https://doi.org/10.1016/j.neucom.2024.128161
Prasanth, T., Padhy, R. P., Sivaselvan, B., & Bakshi, S. (2026). LinkNet3D: An efficient end-to-end network for 3D object detection. Computers and Electrical Engineering, 129, 110821. https://doi.org/https://doi.org/10.1016/j.compeleceng.2025.110821
Punia, D., & Kumar, R. (2025). A YOLOv3-powered edge computing technique for real-time rear-end collision prediction in autonomous vehicles. Applied Soft Computing, 185, 113981. https://doi.org/https://doi.org/10.1016/j.asoc.2025.113981
Rottmann, L., Johannsen, A., Niedling, M., & Vollrath, M. (2026). Influence of seating orientation on motion sickness in autonomous vehicles. Applied Ergonomics, 130, 104643. https://doi.org/https://doi.org/10.1016/j.apergo.2025.104643
Shin, J., Han, J., Park, K., Doh, M., Luong, T., Pico, N., & Moon, H. (2025). Practical implementation of obstacle avoidance strategies in the truly multi-vehicle Autonomous Robot Racing Competition. Engineering Science and Technology, an International Journal, 70, 102164. https://doi.org/https://doi.org/10.1016/j.jestch.2025.102164
Singh, A., Murzello, Y., Lee, H., Abdalla, S., & Samuel, S. (2024). Moral decision making: Explainable insights into the role of working memory in autonomous driving. Machine Learning with Applications, 18, 100599. https://doi.org/https://doi.org/10.1016/j.mlwa.2024.100599
Su, Y., Shi, X., & Song, T. (2025). Research on Vehicle Safety Based on Multi-Sensor Feature Fusion for Autonomous Driving Task. Computers, Materials and Continua, 83(3), 5831–5848. https://doi.org/https://doi.org/10.32604/cmc.2025.064036
Sun, H., Li, M., Cui, Z., Huang, Y., & Chen, H. (2025). Semantic Shapley-based counterfactual explanations for end-to-end autonomous driving. Engineering Applications of Artificial Intelligence, 159, 111638. https://doi.org/https://doi.org/10.1016/j.engappai.2025.111638
V, D. R., J, B. P., & Leung, M.-F. (2025). Visual Perception and Adaptive Scene Analysis with Autonomous Panoptic Segmentation. Computers, Materials and Continua, 85(1), 827–853. https://doi.org/https://doi.org/10.32604/cmc.2025.064924
Wang, S., & Huang, Z. (2025). Trust transfer in the adoption of autonomous last mile services toward digital urban system. Digital Engineering, 5, 100030. https://doi.org/https://doi.org/10.1016/j.dte.2024.100030
Wang, Y., Zhou, Y., Zhong, M., Mei, Y., Fujita, H., & Aljuaid, H. (2025). A multimodal traffic scene understanding model integrated with optical flow maps. Neurocomputing, 649, 130838. https://doi.org/https://doi.org/10.1016/j.neucom.2025.130838
Xu, D., Qiao, E., Gu, T., Fu, H., Sun, C., Guo, H., & Liu, Y. (2025). LEAD: LLM-enhanced deep reinforcement learning for stable decision-making in critical autonomous driving scenarios. Neurocomputing, 657, 131619. https://doi.org/https://doi.org/10.1016/j.neucom.2025.131619
Xu, S., & Rai, R. (2024). Vision-based autonomous navigation stack for tractors operating in peach orchards. Computers and Electronics in Agriculture, 217, 108558. https://doi.org/https://doi.org/10.1016/j.compag.2023.108558
Xu, T., Tian, A., Gao, J., Yan, H., & Liu, C. (2024). Analysis of the spatial heterogeneity of glacier melting in Tibet Autonomous Region and its influential factors using the K-means and XGBoost-SHAP algorithms. Environmental Modelling & Software, 182, 106194. https://doi.org/https://doi.org/10.1016/j.envsoft.2024.106194
Yang, Y., & Yuan, G. (2024). High precision DSRC and LiDAR data integration positioning method for autonomous vehicles based on CNN. Computers and Electrical Engineering, 120, 109741. https://doi.org/https://doi.org/10.1016/j.compeleceng.2024.109741
Yang, Z., Huang, S., Wang, X., Bai, T., & Wang, Y. (2025). MT-Nod: Metamorphic testing for detecting non-optimal decisions of autonomous driving systems in interactive scenarios. Information and Software Technology, 180, 107659. https://doi.org/https://doi.org/10.1016/j.infsof.2024.107659
Yildirim, M., Dagda, B., Asodia, V., & Fallah, S. (2025). HighwayLLM: Decision-making and navigation in highway driving with RL-informed language model. Robotics and Autonomous Systems, 193, 105114. https://doi.org/https://doi.org/10.1016/j.robot.2025.105114
Zhang, Y., Yin, J., Ma, Y., Niu, Y., & Zhang, X. (2026). Physical and virtual safety evaluation in autonomous driving systems using three-dimensional adversarial implementations. Engineering Applications of Artificial Intelligence, 163, 112898. https://doi.org/https://doi.org/10.1016/j.engappai.2025.112898
Zhou, R., Huang, J., Li, M., Li, H., Cao, H., & Song, X. (2025). Knowledge transfer from simple to complex: A safe and efficient reinforcement learning framework for autonomous driving decision-making. Advanced Engineering Informatics, 65, 103188. https://doi.org/https://doi.org/10.1016/j.aei.2025.103188
Zhu, L., Zhang, Z., Zhang, Y., Yu, J., & Wang, H. (2025). Autonomous vehicle crash risk modeling by integrating data augmentation and two-layer stacking. Computers in Industry, 171, 104320. https://doi.org/https://doi.org/10.1016/j.compind.2025.104320
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Global Science: Journal of Information Technology and Computer Science
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
