This thesis addresses the problem of representation learning for novel domains with limited data from three different perspectives. First, we address the case when transfer learning is potentially useful but the data structure restricts the use of pre-trained networks in color images, by designing lightweight domain adapters that can be plugged in before the pre-trained network to make it compatible with the new domain. This is demonstrated in hyperspectral image classification tasks and detecting roosting birds in radar imagery. Second, we address the case when transfer learning is ineffective as the target domain is too different but there is a large amount of unlabeled data available. We focus on the astronomy domain and develop self-supervised learning approaches to construct star cluster catalogs from high-resolution images of galaxies taken by space telescopes. Third, we address the case when we have unreliable models due to out-of-distribution deployment or the difficulty of the task, and human verification of the predictions is imperative to extract accurate measurements. Here we focus on counting different objects from massive image collections with reduced human labeling effort. We propose a framework based on importance sampling, an imperfect detector, and human-in-the-loop. We apply our methodology to estimate unbiased counts of birds from radar data over large spatial regions and temporal contexts and to count damaged buildings from satellite images for natural disaster response.
We will end with some work on how AI with human-in-the-loop can be used to solve other measurement tasks and some future challenges in deploying AI for scientific research.
Advisor: Subhransu Maji