Abstract: Machine learning (ML) is the driving force of multitude of modern applications, e.g., object detection in self-driving cars and captions generation in YouTube videos.
To achieve high performances, ML models are trained on large amounts of data of multiple users. User's data can be of sensitive nature and recent research has demonstrated the possibility of inferring sensitive information even from just the outputs of ML models. Privacy preserving ML strives to learn the distribution of data without learning sensitive information about individual samples in the data. Human beings immensely value their privacy, and therefore, a truly privacy preserving ML is the key to successful deployment of numerous ML application.
Thus, the development of privacy preserving ML technology has seen significant advances in recent years. For instance, different from centralized learning, federated learning is an emerging distributed learning paradigm that allows users to never share their private data with any other party. For both centralized and federated learning, membership inference has emerged as a severe risk to users' privacy as well as a promising tool to assess the privacy risks of ML models. In spite of the aforementioned advances, many more challenges related to privacy and robustness of privacy preserving ML technology remain unsolved.
To this end, in this thesis, we first present novel defenses against membership inference attacks that, unlike defenses such as differential privacy, provide acceptable trade-offs between membership privacy and model utility. Such acceptable trade-offs are instrumental towards real-world deployment of privacy preserving ML technology. Next, we turn to the robustness analysis of federated learning (FL). FL is a distributed learning protocol whose performance can be compromised by the presence of malicious clients. Previous works have proposed multiple provably robust FL algorithms to address this issue. We present a general framework to design poisoning attacks and demonstrate that, unfortunately, these robust FL algorithms are highly vulnerable as compared to their theoretical robustness guarantees. Our framework can also be used to audit the future robust FL algorithms. Finally, we present a critical analysis of existing poisoning attacks on FL and robust FL algorithms from the lens of production FL systems and draw significant conclusions about the state of robustness of FL technology.