Category: Research

Guidelines for Human-AI Interaction

I had the pleasure to work with colleagues from Microsoft on a CHI 2019 paper, Guidelines for Human-AI Interaction. This post provides links to the paper, related resources, and blog posts:

  • Publication page on Microsoft Research (MSR) – contains the paper and printable cards, poster, and presentation
  • MSR blog post about the work
  • Medium post explaining the guidelines
  • … check back for more!

ASEE 2015: Paper about the new HCDD major

It feels like I just returned from the annual ASEE meeting. I presented a paper about a topic near and dear to my heart: the new undergraduate major in Human-Centered Design and Development (HCDD) I spearheaded at Purdue.

The paper tells the design story (birth story) of the new program. I took a user-centered approach to curriculum design, since that’s what I know best. I think one of the most valuable tools that came out of it was the vision persona. And, of course, the program itself. 🙂

The paper is available online (you can read it here) and the slides I used are below.

Research project recipe

I came across this article in HuffPo about a new app some students created that can help you identify your most toxic friends. They call it an art project, but I seem to recognize here a common structure for research projects in HCI. So, if you’re my student looking for thesis ideas, read on. 🙂

The recipe goes like this:

  1. Take a problem or issue from the social world (e.g. toxic friendships, collaboration, long-distance family relationships, etc.)
  2. Create a technology that mediates how people deal with that issue – ideally, the technology should improve the human condition or raise critical questions.
  3. Evaluate the technology.
  4. As a result/consequence of evaluating the technology, illuminate some aspect of and contribute knowledge to #1. And/Or, at the very least, derive design implications for this type of technology.

Some examples of papers following this structure:

How to write a Discussion chapter for your thesis or dissertation

WhatDoesItAllMean-300x115I noticed that the Discussion chapter is one of the hardest to write, especially when you are so close to the results and your head is wrapped up in all the data. Writing the Discussion chapter requires taking a few big steps back and seeing the big picture. For that reason, I often write it with my eyes closed, without looking at the results. Or I ask students to imagine they ran into a friend or colleague at a coffee shop. They don’t have the manuscript or slides on them. They just need to explain to the colleague, without using numbers, or tables, or figures – just narrative – the following:

  • what they did (briefly)
  • what they found – what were the significant, memorable findings?
  • what do the findings mean? – what does it mean that X was rated as 4.61 and Y was rated as 3.93?
  • do the best of your knowledge, why do you think that is? what accounts for these results?
  • why are the findings significant/important/useful? how can they be used, and who can use them?

This is the part where you sell your research. But then, a word of caution:

  • what went wrong?
  • what should we keep in mind as we buy into your findings? how do the limitations of your study affect the results? (this is, indeed, the Limitations section)

Think of the Discussion chapter as an executive summary. If it is the only thing I read, I should get a good understanding of what you found and why it matters. You should explain it to me clearly, in a narrative, without restating your results.

And now that we are so close, I might as well address the Conclusion chapter. It should accomplish 2 things:

  1. Summary of the entire project – this can be an extended abstract. What you set out to do (purpose of research), what you did (methods) and what you found out (main results).
  2. Directions for future research. I learned something great about this in a thesis defense yesterday. Think beyond replicating your study and overcoming your limitations. Think beyond better ways of addressing the same research questions. Now that we know what your research results are, what are other interesting questions we should address? What other issues and questions arise?

I’ve said this so many times in the past few weeks that I felt writing a blog post I can refer students to might be helpful. Please feel free to add your advice or questions in the comments below.

Happy discussing,

Dr. V

Information and Contemplation course at UW

The Chronicle of Higher Education published today an article about a course on Information and Contemplation taught by David Levy at UW. Interesting to see that Levy’s previous work on effects of meditation on multitasking was actually funded by the National Science Foundation. Interesting to see that ACM CHI and Graphics Interface publish this kind of work.

Website Experience Analysis

This post explains an alternative research protocol, website experience analysis (WEA).

Website experience analysis is a research protocol (set of procedures) that can help researchers identify what specific interface elements users associate with particular interpretations.

WEA focuses on the messages that users take-away from their experience with the interface.

All interfaces try to communicate something, such as:

  • you should trust this application with your credit card data
  • you should come study for a MS degree in CGT at Purdue
  • etc.

WEA allows you to find out:

  1. whether the interface actually communicates this message – do people actually take away the message that you intended, and to what extent?
  2. what specific elements of the interface users associate with those particular messages (trust, CGT is a good program, etc.)

The WEA questionnaire is based on prominence-interpretation theory. It works with pairs of items that ask:

  1. Ratings of user perceptions (e.g. trust – on a scale of 1-10)
  2. Open-ended: what about the interface makes the user feel this way?

WEA is based on a much more complex theoretical framework of the website experience. The framework breaks the website experience down into two major dimensions: time and space. WEA then explains the phases of the experience as they unfold across time, and the elements of the website space (elements are categorized according to element functions). The theoretical framework is likely only valid for websites, because the experience with another type of interface, even though it may have the same three main temporal phases (first impression, engagement, exit) will likely differ in terms of the steps within those phases and the nature of the spatial elements and their functions.

WEA is different from a regular questionnaire because it connects perceptions with specific interface elements. Questionnaires will tell you whether the user trusts the product, but they won’t provide specific feedback as to what particular elements may account for that perception.

WEA is modular, which means that a different battery of items can be used, depending on the focus of the research. I used WEA in 2 contexts:

  1. To evaluate the experience of visiting organizational websites. Here, I used the 5 dimensions of good relationships between organizations and their publics: trust, commitment, investment, dialog, etc.
  2. To evaluate whether emergency preparedness websites persuade users to take emergency preparedness actions. Here I used a battery of items derived from a theory of fear appeals (EPPM) and assessed whether users perceived there is a threat, believe they can do something about it, believe the recommended actions would be effective, etc.

I think WEA would provide excellent feedback about how prospective students perceive the CGT department, based on their experience with the website. It would be very valuable to find out exactly what about the website makes them feel that:

  • they would benefit from a CGT MS
  • they would fit in
  • they would have a good educational experience
  • etc. – we have to determine the relevant set of items. Ideally, we would have a theory to guide item development.

WEA can be used with other research questions, such as: How do HR managers look at job candidates’ online information? (hello, Jack!)

WEA can be improved upon to better tap into emotional aspects of the user experience. It can be modified to be a more inductive approach, that elicits emotions and interpretations from users rather than asking about specific interpretations (such as trust, etc.)  – thank you, Emma, for these suggestions!

If you would like to read more about WEA, you can find the relevant citations in Google Scholar. I can provide copies of the papers if you don’t have access to them.

Online identity management article in JOEE

We (myself and my research group) have been working for a while on projects related to the management of online identity – that is, how people do, and should, present themselves online, across several social media sites.

One of the first papers in what will hopefully be a series was published this past Friday in the Journal of Online Engineering Education – Online Identity Management Literacy for Engineering and Technology Students (pdf).

This video explains why you should read the paper. 🙂

The paper discusses the current employment climate in the U.S., where many employers check students’ Google resume in addition or even instead their paper resumes. It presents data about online identity management of undergraduate students, collected through in-depth interviewing. It then presents a program for students to manage their online identities:

The program begins with social media literacy – understanding the dynamics that make identity management trickier online than off. It then suggests tools, ideas, and direction for 4 steps:

  1. Creating professional online content
  2. Optimizing online content for social media
  3. Developing and maintaining an online professional network
  4. Monitoring and maintaining online presence

Coincidentally, there is an article in today’s university newspaper about this very topic. The article quotes yours truly (I forgot to mention that my name is Dr. V!) and summarizes informally some of the points that are developed formally in the journal article.

The journal article is intended for:

  • college professors, who are asked to consider teaching a bit of social media literacy in their courses;
  • college students, who benefit from awareness of how to manage their online information (at least that’s what the students who read the paper tell me).

And no, you don’t have to be in engineering or technology to find this article useful. 🙂

Graduate students who are interested in this and other social media topics may find the research seminar I am teaching this Fall interesting. TECH 621: Research Focus: The Social Internet is a course that reviews current research in core social media topics. In this course, students are immersed in social media and work on an original research project of their own interest. More info about the course coming soon, but feel free to contact me if you’re interested and are a Purdue student. And, if you are interested but are not a Purdue graduate student… well, why not become one?