This post explains an alternative research protocol, website experience analysis (WEA).
Website experience analysis is a research protocol (set of procedures) that can help researchers identify what specific interface elements users associate with particular interpretations.
WEA focuses on the messages that users take-away from their experience with the interface.
All interfaces try to communicate something, such as:
- you should trust this application with your credit card data
- you should come study for a MS degree in CGT at Purdue
WEA allows you to find out:
- whether the interface actually communicates this message – do people actually take away the message that you intended, and to what extent?
- what specific elements of the interface users associate with those particular messages (trust, CGT is a good program, etc.)
The WEA questionnaire is based on prominence-interpretation theory. It works with pairs of items that ask:
- Ratings of user perceptions (e.g. trust – on a scale of 1-10)
- Open-ended: what about the interface makes the user feel this way?
WEA is based on a much more complex theoretical framework of the website experience. The framework breaks the website experience down into two major dimensions: time and space. WEA then explains the phases of the experience as they unfold across time, and the elements of the website space (elements are categorized according to element functions). The theoretical framework is likely only valid for websites, because the experience with another type of interface, even though it may have the same three main temporal phases (first impression, engagement, exit) will likely differ in terms of the steps within those phases and the nature of the spatial elements and their functions.
WEA is different from a regular questionnaire because it connects perceptions with specific interface elements. Questionnaires will tell you whether the user trusts the product, but they won’t provide specific feedback as to what particular elements may account for that perception.
WEA is modular, which means that a different battery of items can be used, depending on the focus of the research. I used WEA in 2 contexts:
- To evaluate the experience of visiting organizational websites. Here, I used the 5 dimensions of good relationships between organizations and their publics: trust, commitment, investment, dialog, etc.
- To evaluate whether emergency preparedness websites persuade users to take emergency preparedness actions. Here I used a battery of items derived from a theory of fear appeals (EPPM) and assessed whether users perceived there is a threat, believe they can do something about it, believe the recommended actions would be effective, etc.
I think WEA would provide excellent feedback about how prospective students perceive the CGT department, based on their experience with the website. It would be very valuable to find out exactly what about the website makes them feel that:
- they would benefit from a CGT MS
- they would fit in
- they would have a good educational experience
- etc. – we have to determine the relevant set of items. Ideally, we would have a theory to guide item development.
WEA can be used with other research questions, such as: How do HR managers look at job candidates’ online information? (hello, Jack!)
WEA can be improved upon to better tap into emotional aspects of the user experience. It can be modified to be a more inductive approach, that elicits emotions and interpretations from users rather than asking about specific interpretations (such as trust, etc.) – thank you, Emma, for these suggestions!
If you would like to read more about WEA, you can find the relevant citations in Google Scholar. I can provide copies of the papers if you don’t have access to them.
Most research articles you find in academic journal follow a similar recipe. If you understand how the article is structured and what to look for in each section, you can read articles much faster. I can get what I want from a research article in 5 minutes or less. When I started grad. school it took me 45-60 minutes to get through a research article and I still didn’t get much out of it. I wish someone had taught me how to read them.
Here are my lessons, based on my experiences. They work for me. I hope they work for you, too. If they don’t, use this as a starting point to figure out your own reading process.
Understanding the anatomy of a research article will also help you write easier.
Usually long and cryptic. Most titles are poorly written. I don’t pay much attention to the title.
I read it carefully and look for:
- purpose of study/research question
- a hint as to research methods
- key results
I read the introduction looking for the following information:
- explanation of the problem the study addresses
- explanation of the larger context of the problem
- argument about the importance/need/relevance of studying the problem
- purpose of the study
- an overview of how the article is structured, and how the next section is organized
It may be called something else, or the article may not even have headings – but it should be there somewhere. The literature review should accomplish 2 purposes:
- make an argument for the need to conduct this specific study (identify a gap, or a need in previous literature)
- present the previous theories, concepts, etc. that this study uses and builds upon
Usually, each paragraph or small section of the literature review covers a body of literature (the best lit. reviews are organized thematically, IMO). When reading the literature review it is important to identify these major themes. They give you a lay of the land.
Imagine the body of literature is a garden. The article you’re reading attempts to plant a new seed in this garden. Before doing so, the authors explain the layout of the garden (vegetables here, flowers there, weeds over there) and they explain why their plant is needed and where it fits in.
When reading the lit. review, you get a feel for this garden. If you are:
- very familiar with the literature, the lit. review confirms that the authors looked in all the right places and didn’t reinvent the wheel. OK to skim.
- completely unfamiliar with the literature, this section will be terribly confusing. Don’t worry. All you have to get out of it are the major themes (sections of the garden). You can come back later and examine each individual plant. OK to skim.
- are trying to learn the literature – read carefully, and mark on the list of references the sources you want to read.
The literature review ends with the research question(s). Find them and highlight them. They are promises that the article should deliver on.
This section explains the research methods and procedures used for the research study. Read them carefully, make sure they are valid. If the research methods are faulty, the data are not to be trusted. If the research methods are absurdly faulty, stop reading here. Go back to the literature review and the list of references and see if they can help you find better articles on the topic.
In this section, the authors present their data, along with their (statistical or interpretive, etc.) analysis. This is as close as you can get to the raw data. This section, in a quantitative article, should be as free as possible of interpretation. Try your best to understand the results for yourself, so you can create your own interpretation of what they mean. But, if the statistics baffle you AND if you trust the authors, skim this section and move on to:
This section explains what the results mean, in the context of the garden (literature review). You should see how the problem from the introduction is solved, how the research questions are answered, and whether the purpose of the study was accomplished. I usually read this section very carefully, because it tells me what the authors think they have accomplished.
Either here or at the end of the conclusion, you will find suggestions for future research. These can be very useful for your own literature review – you can cite the article, if it calls for exactly the research you’re doing. You can use this to support your own argument about the need for your research.
The first part of the conclusion should be a summary of the entire paper. I read it carefully, because the repetition helps me remember what I read. The last part of the conclusion is usually the most difficult part to write, very often fluff, and I don’t feel guilty about skimming or skipping it.
I used to teach this recipe to graduate students and they found it very helpful. I hope you do, too. Please share your own reading and writing tips, and ask me other questions you may have about graduate school.
There are several books that can help you, and the APA style manual has a chapter that explains the structure of APA research papers.
Someday, I will understand quantum physics. But since in the past few weeks I’ve been unpacking, unpacking, unpacking, unpacking, unpacking… (you get it)… OK, never mind. Here’s a video about quantum physics. It should be the beginning of any research methods class.
Thanks to Twitter user @c4chaos for pointing to a link that lead me to this video.