Abstract: Although data visualizations have been around for centuries and are encountered frequently by the general public, existing evidence suggests that a significant portion of people have difficulty understanding and interpreting them. It might not seem like a big problem when a reader misreads a weather map and finds themselves without an umbrella in a rainstorm, but for those who lack the means, experience, or ability to make sense of data, misreading a data visualization concerning public health and safety could be a matter of life and death. However, figuring out how to make visualizations truly usable for a diverse audience remains difficult. Therefore, in my dissertation, I will examine what makes current data visualizations understandable and what techniques data visualization practitioners might be able to employ in the future to increase their impact.
First, I will present the results of a survey critically analyzing the ways that data visualizations intended for use by non-expert audiences are evaluated. In this study, I critically analyzed the research methodologies of 92 existing research papers containing human-subjects experiments through examining high-level features of each paper (e.g., how the authors describe the paper's main contribution) and low-level features of individual experiments (e.g., the measures operationalized by researchers). My analysis revealed several mismatches between (1) the intended audience and the people recruited to participate in evaluation studies and (2) the evaluation metrics used in experiments and those that are meaningful for the audience and the context of use. These mismatches can severely impact the external validity of evaluation and impede our understanding of what visualization techniques work well for those who are novices in visualization. Further, they emphasize the need for novel research methodologies that more holistically capture readers' experiences and thought processes.
Next, I will examine the impact of design techniques on understanding and experience using a novel method inspired by the field of Education. One kind of data visualization often encountered by the general public is the infographic. In existing research, infographics show positive empirical findings in terms of memory, engagement, and assessment of risk -- particularly when they contain pictographs (simple, iconic pictures that represent a word or topic). However, there is little exploration of how pictographs affect and afford the general public's understanding of the underlying data or how the choice to use pictographs affects readers' personal experiences. Therefore, I will present the results of an experiment that utilized a novel method of producing questions that probe different aspects of a reader's understanding of 6 pairs of real-world visualizations which are identical except for their use of pictograph arrays. My results indicate that the use of pictograph arrays does not directly impact understanding but can allow readers to more easily envision real-world connections.
Finally, I will explore how including information that is not currently present in visualizations could impact understanding and experience. While visualizations are designed to present a multitude of data, they often are not accompanied by key metadata which provides background on the source of the data, the transformations applied to the data, the visualization elements, its purpose, the people involved in its creation, or its intended audience. However, providing these metadata to a reader might have significant impacts on their understanding and experience of the visualization. Therefore, in the final component of my thesis, I will describe some of the ethical and practical benefits (and risks) to providing readers with access to metadata. Further, I will propose a study to investigate what kinds of metadata that members of the general public want access to and the impact that having it has on factors like understanding and perceived trustworthiness.
Advisor: Narges Mahyar