I had the pleasure to work with colleagues from Microsoft on a CHI 2019 paper, Guidelines for Human-AI Interaction. This post provides links to the paper, related resources, and blog posts:
Problem: The National Science Foundation (NSF) needed a way to help them understand and evaluate their funding portfolio.
Context: NSF is a federal agency that provides, on a peer-reviewed, competitive basis, funding for research, in order to advance the mission of science.
My role: I was a co-Principal Investigator on this $3 million, 4-year project. I lead the UX research and design.
Project goals:
Phase 1: NSF employees
In the first phase of the project, we focused on serving the internal NSF audience. My contribution to the project was as follows:
I planned & conducted formative research in the field in order to identify user groups inside the NSF and understand their needs. I focused the data collection around questions such as:
I chose the most vulnerable persona as the primary one. I lead the interdisciplinary research team through a design exercise where we created a context scenario for our primary persona and extracted design requirements – what information would Matt need in order to begin understanding his funding portfolio and be productive at his new job? I generated early sketches based on our brainstorming. The general approach we took to knowledge mining and visualization is explained here and here.
I directed students as they started working on wireframes based on my sketches and coordinated the communication between the UX and technical teams. In the days before Slack, I used an internal team blog to track and communicate work.
I conducted early testing on the alpha version. My goal was to assess ease of learning. Could our users figure out how to use our Web application? Would they understand the interactive data visualizations and interpret them correctly? User feedback, documented in this report, included comments such as:
I feel this was designed for me!
and
This thing reads my mind!
We delivered DIA2 to the NSF and proceeded to focus on the external audience:
Phase 2: NSF external audience
The team identified STEM faculty members as the largest external audience. These are researchers who need to understand the funding portfolio in order to better target their proposals to the NSF.
I designed an interview protocol for intercept interviews we conducted at the annual meeting of the Association for Engineering Education, where we were most likely to encounter STEM researchers from various fields. I trained a number of graduate students and we all conducted interviews and collected the data we needed in 3 days.
I lead a cross-disciplinary team of graduate students from both the UX and technical teams through a 2-day affinity diagramming process, which resulted in one persona, documented in this report and this conference paper.
With an understanding of the second user group’s needs, I wanted to ensure DIA2 served them well. I lead the team through cognitive walkthrough exercises where we asked whether Dr. Anderson, our persona, would know what to do, and if he performed an action, whether he would know he was making progress towards his goals. I supervised one of my graduate students as she conducted usability testing with this new user group. This work resulted in a conference paper and her M.S. thesis.
DIA2 served about 2 million visitors in a 2 year period. About 2,000 users had created accounts. The project is over and the data is no longer updated, as is common with a lot of academic projects.
The research & design process, as well as technical aspects of DIA2 are presented in a paper we published in IEEE Transactions on Visualization and Computer Graphics. More research related to DIA2 is indexed on the project’s research page.
Dr. V 