CAREER: Towards Trustworthy Machine Learning via Learning Trustworthy Representations: An Information-Theoretic Framework

The objective of this project is to enable machine learning (ML) to be trustworthy. ML, especially deep learning that uses deep neural networks, has made remarkable breakthroughs in various research domains and disciplines including computer vision, natural language processing, biology, and math, to name a few. However, in the past decade, extensive work has shown ML models are vulnerable to privacy and security attacks.

For example, email spam filters can be compromised by data poisoning attacks, where attackers confuse ML models by feeding them bogus data, allowing adversaries to send malicious emails containing malware or other security threats without being noticed. Attackers can also make repeated requests to models, looking at the results in order to reconstruct the data used to build ML models; in health domains, for instance, successful data reconstruction attacks might expose private medical details about patients.

Many defense methods have been proposed to mitigate these attacks, but they face several limitations: they often aren’t effective in real-world applications with strict confidentiality requirements, or unacceptably degrade the performance of the models. Further, most defenses are aimed at particular learning methods or attack types, making it hard to deal with multiple concurrent attacks, and generalizing poorly to different types of models and data.

This project’s goal is to address these limitations by designing a trustworthy learning framework based on information theory. The outcomes of the project will advance the state-of-the-art trustworthy ML and information-theoretic approaches to privacy, while contributing to the growing national need for professionals in ML and cybersecurity.

To do this, the team will design a practical, accurate, flexible, and generalizable information-theoretic trustworthy representation learning framework with robustness and privacy guarantees. The work will be structured around three thrusts. Thrust 1 will design novel information-theoretic representation learning methods against common privacy attacks, including membership inference, property inference, and data reconstruction attacks.

Thrust 2 will design novel information-theoretic representation learning methods against common security attacks, including test-time evasion attacks, training-time poisoning attacks, and training- and test-time backdoor attacks. Thrust 3 will generalize Thrust 1 and Thrust 2 to handle diverse attack types (e.g., multiple privacy/security attacks or their combination), data types (e.g., spatial-temporal data, multimodal data), and learning types (e.g., federated learning, graph learning, self-supervised learning).

The proposed framework will be evaluated on datasets and learning tasks from several domains, including computer vision, natural language processing, multimedia, and networking. The team will develop an open-source toolkit to make the techniques widely available to other researchers in academia, industry, and government. Outreach and educational activities, including summer camps, talks, lectures, tutorials, and workshops, will promote the participation of K-12, undergraduate, and graduate students, with a focus on providing opportunities for people from groups underrepresented in STEM.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.