Повышение устойчивости математических моделей в состязательных сценариях с использованием подходов обобщения тема диссертации и автореферата по ВАК РФ 00.00.00, кандидат наук Рашид Бадер

• Рашид Бадер
• кандидат наук
• 2023

Оглавление диссертации кандидат наук Рашид Бадер

Contents

1 Introduction

1.1 Relevance and Importance

1.1.1 Research Motivation

1.1.2 Research Challenges and Existing Approaches

1.1.3 Contributions and Scientific Novelty

1.2 Presentations and Validation of Research Results

1.3 Publications

1.4 Thesis Structure Overview

2 Robustness in the Adversarial domain: Related work

2.0.1 Adversarial Perturbations Model

2.0.2 Perturbations Generation

2.0.3 White Box scenarios

2.0.4 Black Box scenarios

2.0.5 Transferability of Adversarial Samples

2.1 Current methods for Enhancing Robustness

2.1.1 Defensive Distillation

2.1.2 Robust Optimization

2.1.3 Regularization Approach

2.1.4 Adversarial Training (AT)

2.2 Chapter Conclusion

3 Robust Generalization under Adversarial Perturbations: Problem formulation

3.1 Statistical Learning

3.2 Definition of Robustness in Statistical Learning

3.2.1 Accounting for Uncertainties in Explanatory Variables

3.2.2 Flaws in Model Optimization

3.3 Robustness to Noise 25 3.3.1 Problem Statement: Robustness to Noise in Deep Neural Networks

3.4 Adversarial Robustness: The Worst Case

3.4.1 Adversarial Perturbations and Their Impact

3.4.2 Mathematical Formulation of Adversarial Robustness

3.4.3 Adversarial Training

3.5 Enhancing Robustness through Out-of-Distribution Generalization: Methods and Insights

3.5.1 The Adversarial-Clean Accuracy Trade-off

3.6 Chapter conclusion

4 Multiple Adversarial Domains Adaptation Approach for Robustness Enhancement

4.1 Motivation

4.2 Background

4.2.1 Adversarial Domain Adaptation (ADA)

4.2.2 Wasserstein distance

4.3 Multiple Adversarial Domain Adaptation

4.3.1 Formulation

4.3.2 Global Domain Alignment

4.3.3 Class-level Alignment

4.4 Experiments

4.4.1 Experiment Setup

4.4.2 Experimental results

4.5 Chapter conclusion

5 Boosting adversarial training using robust selective data augmentation

5.1 Related work

5.1.1 Automatic data augmentation

5.1.2 Influential samples

5.2 The proposed approach

5.3 Model

5.4 Experiments

5.4.1 Experiment Setup

5.4.2 Experimental results

5.5 Ablation Study

5.6 Chapter conclusion

6 Structure Estimation of Adversarial Distributions for Enhancing Model Robustness: A Clustering-based Approach

6.1 Introduction

6.2 Data Clustering 61 6.2.1 Proposed Novel Clustering Algorithm

6.3 Dimensionality reduction

6.4 Clustering and Information Content

6.4.1 Defining Cluster Variation and Entropy

6.4.2 Connection between Cluster Variation and Probability Distribution

6.4.3 Connecting Cluster Variation to Entropy

6.5 Experiments

6.5.1 Overall comparision

6.5.2 The effect of dimensionality reduction

6.6 Conclusion

Conclusion

6.7 Overall summary

6.8 Limitations, recommendations, and future research

References

List of abbreviations

List of Figures

List of Tables

Введение диссертации (часть автореферата) на тему «Повышение устойчивости математических моделей в состязательных сценариях с использованием подходов обобщения»

Introduction

The grand narrative of machine learning has evolved dramatically over the past few decades, with deep learning now taking center stage. Amidst this expansive landscape, certain complexities and curiosities have piqued the interest of researchers worldwide. This thesis steps into the territory of deep learning with a unique lens, focusing on the robustness under adversarial perturbations from a generalization perspective.

Our aim is to explore adversarial perturbations not as an isolated concern, but as a challenging scenario of distribution shifts - an approach not widely established in the existing literature but that forms the basis of our exploration.

This introductory chapter sets the stage for the subsequent detailed discussions on this research and presents an overview of our research motivation, challenges, contributions, and the structure of the rest of the thesis.

1.1 Relevance and Importance

Machine learning, a branch of artificial intelligence, has witnessed phenomenal growth and success over the past few decades. It works mainly by learning patterns and make predictions from data, effectively automating decision-making processes in various real-world applications. In the domain of machine learning, deep learning stands as a specialized subset that utilizes multi-layered artificial neural networks to automatically uncover complex relationships within extensive data sets. This particular capability has revolutionized various industries by obviating the necessity for manually designed features, thus enabling the model to learn hierarchical structures directly from the data.

However, although these models perform exceptionally well when the training and testing data come from the same distribution, their performance can degrade significantly when the data distribution shifts. Current approaches to deep learning often aim for maximum generalization, seeking to minimize training error and achieve optimal performance on seen data. Generalization typically refers to a model's ability to adapt effectively to new, unseen data drawn from the same distribution as the training set. However, this classical interpretation of generalization does not fully account for the complexities observed in real-world applications and tends to

1

overlook uncertainties and perturbations, which are inherent in real-world data. Indeed, it is often the case that the test instances, or the data the model interacts with during deployment, may not adhere strictly to the same distribution as the training data. This discrepancy between the training and deployment data distributions, often referred to as distribution shift, poses a serious challenge to the generalization capabilities of the model. This issue of generalization, particularly on out-of-distribution data, is a critical challenge in machine learning and deep learning. The test or target distribution often differs from the training distribution due to various factors. One of these factors includes input perturbations that exist either because of naturally occurring noise or intentional adversarial perturbations. The latter is particularly concerning as adversarial samples have been shown to be highly effective at fooling deep learning models, making them a significant threat in real-world applications.

In the realm of machine learning, robustness is traditionally defined as the ability of a model to maintain its performance on variations or disturbances in the input data. These variations could arise from deviations naturally present within the data, or adversarial perturbations. Particularly, adversarial samples are of a big concern in the machine learning research community. Adversarial samples intentionally introduce distribution shifts that deviate from the training data. These perturbations push the network to make errors on samples that are considered out-of-distribution, despite being visually similar to the training data. These samples exploit vulnerabilities in the model's decision boundaries, and can serve as worst-case scenarios for generalization. Therefore, effectively addressing generalization on out-of-distribution samples necessitates developing solutions that improve the network's generalization on adversarial distributions.

Understanding this, we position our investigation into robustness as an extension of the exploration into generalization. It is important to note that, while these two concepts have often been treated separately in the literature, our work approaches robustness as a sub-category within the broader generalization framework. We consider adversarial perturbations as an extreme case of distribution shift, where the adversary intentionally manipulates the input data to induce misclassification. Although the literature does not commonly classify adversarial perturbations as a form of distribution shift, we argue that this interpretation is not only plausible but useful in expanding our understanding of robust generalization. Viewing adversarial robustness through this lens allows for the transfer of insights from generalization literature to adversarial robustness challenges.

The study of adversarial robustness not only enhances model security but also assists in debugging deep learning models and understanding their behavior. Adversarial perturbations exposes vulnerabilities, facilitating model refinement and the development of robust architectures. Therefore, adversarial robustness not only safeguards against threats but also provides insights into model learning processes, turning adversarial samples into opportunities for model improvement.

This thesis addresses these crucial aspects of deep learning: the concept of generalization and its more specific manifestation in the face of adversarial perturbations. Adversarial perturbations are small, often imperceptible changes to input data that can cause a trained model to misclassify

it. This leads us to explore the subfield of robustness, which deals with the model's resilience to these perturbations. Within robustness, we further narrow our focus on adversarial training techniques, which incorporate adversarial samples into the training process to improve model robustness. We consider the existing problems with adversarial training methods, including overfitting and the Adversarial-Clean Accuracy Trade-off, and propose solutions based on OOD generalization techniques. In doing so, we hope to shed light on the importance of understanding and enhancing robust generalization under adversarial perturbations in deep learning, an area of significant relevance in our increasingly data-driven world. A high-level overview of the problems addressed in this thesis is provided in Figure 1.1

Заключение диссертации по теме «Другие cпециальности», Рашид Бадер

6.7 Overall summary

This thesis focuses on developing mathematical models and algorithms to enhance the adversarial robustness and generalization of intelligent systems. It addresses adversarial perturbations as instances of data distribution shifts, emphasizing their characteristics and methods to enhance the adversarial robustness of deep neural networks.

The primary findings include:

1. A robust neural network architecture was designed, implemented and investigated by developing model-based protections using multi-domain adversarial domain adaptation. The Multiple Adversarial Domain Adaptation (MADA) method was proposed, which uses adversarial domain adaptation for learning robust domain-invariant features. By seeking optimal alignment of the adversarial and clean domains, the sample space for adversarial examples is decreased. The overall architecture consists of three components: a feature generator, a domain critic, and a classifier. Our experiments showed that MADA surpasses AT on adversarial samples by about 4% on average and on clean samples by about 1% on average.

2. A data-based approach to robustness improvement was developed. This method explores the roots of adversarial samples, leading to more robust model construction. The robust selective data augmentation (RSDA) approach was proposed to enhance the performance of AT. RSDA performs data transformation operations on specific neighboring samples of each adversarial sample in the latent space. Our method reduces the decrement of clean accuracy significantly. The results showed that RSDA surpasses AT on adversarial samples by about 3% on average and on clean samples by about 0.5% on average.

3. A novel approach to enhancing model robustness against adversarial input perturbations was developed by estimating the structure of adversarial distributions through clustering algorithms. The Adversarial Structural Clustering (ASC) method, which is a density and boundary-aware clustering algorithm, captures the inherent structure of adversarial distributions, creating denser and more robust clusters. ASC leverages adversarial per-

turbations to enhance the boundary delineation between clusters and prioritizes samples near the decision boundary for clustering. The proposed ASC algorithm exhibited the highest robustness against adversarial samples, validating its effectiveness in capturing the structure of adversarial distributions. The combination of ASC with t-SNE preprocessing achieved the highest clean and adversarial accuracy.

The scientific novelty of this work lies in its development of mathematical modeling methods for robust training, enhancing the performance of deep learning models under diverse adversarial settings. The approaches presented here are comprehensive, treating adversarial perturbations as challenging scenarios of distribution shifts.

The theoretical significance of this work stems from the exploration of adversarial robustness in DNNs, and the introduction of an innovative understanding of adversarial sample nature. The developed methods advance our comprehension of adversarial robustness in deep learning, paving the way for further research and development in this critical area.

6.8 Limitations, recommendations, and future research

The main focus of this thesis was on how to increase the adversarial robustness of deep neural networks against adversarial perturbations. For this, we used different model-based and data-based techniques to boost adversarial training. Despite experimentally proved effectiveness of the proposed methods, there are certain limitations.

The proposed methods do not take into account the adaptive ability of an adversary against the proposed methods. An adversary might try to overcome the proposed methods by using new techniques not considered before. This is a game between the adversary and the defender.

Apart from that, although the data-based method gives good results, it depends on the effectiveness of the influence functions we use, and faster functions should be considered in order to use the proposed method in real life.

The future directions of our research include:

1. Include the proposed methods in popular open-source deep learning libraries as a new way for training accurate and robust models;

2. Study in more details the connection between the robustness and interpretability of deep neural networks.

3. Study in more details the geometry of adversarial samples and how the decision boundary

changes in adversarial training in order to understand deep neural networks more. Recommendations for the use of scientific findings and research perspectives:

1. This work's findings extend beyond improving adversarial robustness, shedding light on neural networks' operations. The adversarial robustness study, as detailed in this thesis,

can serve as an effective debugging tool for models, allowing us to comprehend where and why a model fails. In this sense, our research offers a lens through which one can critically evaluate a model's performance and identify its weak spots.

2. The theoretical and practical possibilities of increasing the robustness of deep neural networks are possible. However, developers should be careful when applying our methods in real-life scenarios not to hurt the clean accuracy. Choosing the hyperparameters that control the tradeoff between accuracy and robustness should be considered seriously when training robust models.

3. Abundant real-world evidence suggests that actual adversaries use simple tactics to subvert ML-driven systems, most of them not related to adversarial samples but to other bugs in the deployed system. So the robustness of the model should be studied inside the whole system.

Список литературы диссертационного исследования кандидат наук Рашид Бадер, 2023 год

References

[1] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214-223. PMLR, 2017.

[2] Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information fusion, 58:82-115, 2020.

[3] Samuel A Barnett. Convergence problems with generative adversarial networks (gans). arXiv e-prints, pages arXiv-1806, 2018.

[4] Imad Eddine Ibrahim Bekkouch, Dragos Constantin Nicolae, Adil Khan, S. M.Ahsan Kazmi, Asad Masood Khattak, and Bulat Ibragimov. Adversarial Reconstruction Loss for Domain Generalization. IEEE Access, 9:42424-42437, 2021.

[5] Sam Bond-Taylor, Adam Leach, Yang Long, and Chris George Willcocks. Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models. IEEE Transactions on Pattern Analysis and Machine Intelligence,

mar 2021.

[6] Nicholas Carlini and David Wagner. Towards Evaluating the Robustness of Neural Networks. In Proceedings - IEEE Symposium on Security and Privacy, pages 39-57, CW, 2017.

[7] Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay. Adversarial attacks and defences: A survey. arXiv e-prints, pages arXiv-1810, 2018.

[8] Ambra Demontis, Marco Melis, Maura Pintor, Matthew Jagielski, Battista Biggio, Alina Oprea, Cristina Nita-Rotaru, and Fabio Roli. Why do adversarial attacks transfer? Explaining transferability of evasion and poisoning attacks. In Proceedings of the 28th USENIX Security Symposium, pages 321-338. USENIX Association, sep 2019.

[9] Li Deng. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-142, 2012.

[10] Garima, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. In Advances in Neural Information Processing

Systems, volume 2020-Decem, 2020.

[11] Ian Goodfellow, Nicolas Papernot, Sandy Huang, Yan Duan, Pieter Abbeel, and Jack Clark. Attacking machine learning with adversarial examples. Www.Openai.Com, pages 1-9, 2017.

[12] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets. In Z Ghahramani, M Welling, C Cortes, N Lawrence, and K Q Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.

[13] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.

[14] Juha Heinonen. Lectures on lipschitz analysis. Lecture Notes, (100):1-77, 2005.

[15] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. stat, 1050:9, 2015.

[16] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.

[17] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In 34th International Conference on Machine Learning, ICML 2017, volume 4, pages 29762987, mar 2017.

[18] Alex Krizhevsky and G Hinton. Learning multiple layers of features from tiny images.(2009). Cs.Toronto.Edu, pages 1-58, 2009.

[19] Dong C. Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(1-3):503-528, aug 1989.

[20] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In ASIA CCS 2017 - Proceedings of the 2017 ACM Asia Conference on Computer and Communications Security, pages 506-519. Association for Computing Machinery, Inc, apr 2017.

[21] Nicolas Papernot, Patrick Mcdaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In Proceedings - 2016 IEEE European Symposium on Security and Privacy, EURO S and P 2016, pages 372-387, 2016.

[22] Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks. In Proceedings - 2016 IEEE Symposium on Security and Privacy, SP 2016, pages 582-597, 2016.

[23] Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training. Advances in Neural Information Processing Systems, 34:20596-20607, 2021.

[24] B Rasheed, A Khan, SMA Kazmi, R Hussain, MJ Piran, and DY Suh. Adversarial attacks on featureless deep learning malicious urls detection. Computers, Materials and Continua, 68(1):921-939, 2021.

[25] Bader Rasheed, Adil Khan, Muhammad Ahmad, Manuel Mazzara, SM Kazmi, et al. Multiple adversarial domains adaptation approach for mitigating adversarial attacks effects. International Transactions on Electrical Energy Systems, 2022, 2022.

[26] Bader Rasheed, Adil Khan, and Asad Masood Khattak. Structure estimation of adversarial distributions for enhancing model robustness: A clustering-based approach. Applied Sciences, 13(19), 2023.

[27] Bader Rasheed, Asad Masood Khattak, Adil Khan, Stanislav Protasov, and Muhammad Ahmad. Boosting adversarial training using robust selective data augmentation. International Journal of Computational Intelligence Systems, 16(1):89, 2023.

[28] Kui Ren, Tianhang Zheng, Zhan Qin, and Xue Liu. Adversarial Attacks and Defenses in Deep Learning. Engineering, 6(3):346-360, mar 2020.

[29] Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In International conference on machine learning, pages 4334-4343.

PMLR, 2018.

[30] Rui Shao, Xiangyuan Lan, Jiawei Li, and Pong C Yuen. Multi-adversarial discriminative deep domain generalization for face presentation attack detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2019-June, pages 10015-10023, 2019.

[31] Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of big data, 6(1):1-48, 2019.

[32] Lucas FA Silva, Daniel CG Pedronette, Fabio A Faria, Joao P Papa, and Jurandy Almeida. Improving transferability of domain adaptation networks through domain alignment layers. In 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pages 168-175. IEEE, 2021.

[33] Aman Sinha, Hongseok Namkoong, Riccardo Volpi, and John Duchi. Certifying some distributional robustness with principled adversarial training. arXiv preprint arXiv:1710.10571, 2017.

[34] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. arXiv preprint arXiv:1805.12152, 2018.

[35] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, volume 2017-Janua, pages 2962-2971, 2017.

[36] Tianyang Wang, Jun Huan, and Bo Li. Data dropout: Optimizing training data for convolutional neural networks. In Proceedings - International Conference on Tools with Artificial Intelligence, ICTAI, volume 2018-Novem, pages 39-46, 2018.

[37] Xiaodong Wang and Feng Liu. Triplet loss guided adversarial domain adaptation for bearing fault diagnosis. Sensors (Switzerland), 20(1):320, jan 2020.

[38] Zeya Wang, Baoyu Jing, Yang Ni, Nanqing Dong, Pengtao Xie, and Eric Xing. Adversarial

domain adaptation being aware of class relationships. In arXiv, volume 325, pages 1579-1586, 2019.

[39] Rui Xu and Donald Wunsch. Survey of clustering algorithms. IEEE Transactions on neural networks, 16(3):645-678, 2005.

[40] Chih-Kuan Yeh, Joon Kim, Ian En-Hsu Yen, and Pradeep K Ravikumar. Representer point selection for explaining deep neural networks. Advances in neural information processing systems, 31, 2018.

[41] Zhilu Zhang and Mert R. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In Advances in Neural Information Processing Systems, volume 2018-Decem, pages 8778-8788, 2018.