Faculty Recruiting Support CICS

PL/SE Seminar - Improving the Reliability of Deep Learning Software Systems

06 Apr
Wednesday, 04/06/2022 2:30pm to 3:30pm
Virtual via Zoom
Seminar

Abstract: For the last decade, Deep learning (DL) has emerged as a new effective machine learning approach that is capable of solving difficult challenges. Due to their increasing effectiveness, DL approaches have been applied widely in commercial products such as social media platforms and self-driving cars. Such widespread application in critical areas means that mistakes caused by bugs in such DL systems would lead to serious consequences. Our research focuses on improving the reliability of such DL systems.

On a high level, the DL systems development process starts with labeled data. This data is then used to train the DL model with some training methods. Once the model is trained, it can be used to create predictions for some unlabeled data in the inference stage. We developed testing and analysis techniques that help improve the DL system reliability for all stages.

In this talk, I will first present our work in improving the reliability of the DL system inference by applying differential testing to find bugs in DL libraries. We leverage different equivalence concepts such as equivalence DL libraries or equivalence DL models to detect bugs in DL software systems.

In the second part of the talk, I will present our analysis of how the software implementation of DL libraries alone causes a large variance in training in terms of both model accuracy and fairness. Our large DL developers survey also shows that a majority of DL developers are not aware of or underestimate such training variance caused by software implementation.

I will also present our work in detecting blindspots in training data. Our approach leverages N models to detect weak spot samples that can be added to improve the models with retraining.

Finally, I will present my research vision for applying software engineering techniques to improve the reliability of DL software systems. My future research would focus on the synergy between the DL model and its implementation to improve the reliability of the DL software system from both the software engineering and artificial intelligence perspectives.

Bio: Hung Pham is a Ph.D. candidate at the University of Waterloo, Canada working under the supervision of Prof. Lin Tan and Prof. Yaoliang Yu. His research interests include software testing, program repair, software engineering, and machine learning. He received the M.Sc. degree in Evolutionary and Adaptive System (EASy) from the University of Sussex, Brighton, UK, and the B.Eng degree in Computing from the Imperial College, London, UK.

Join the Seminar

This seminar will be streaming via Zoom at the link above but requires a passcode. To obtain the passcode for this event, please see the announcements on the college email lists or contact us.