Problem: The National Science Foundation (NSF) needed a way to help them understand and evaluate their funding portfolio.
Context: NSF is a federal agency that provides, on a peer-reviewed, competitive basis, funding for research, in order to advance the mission of science.
My role: I was a co-Principal Investigator on this $3 million, 4-year project. I lead the UX research and design.
- Create a tool for monitoring and evaluating the NSF funding portfolio that would serve NSF’s internal and external users.
- Advance the mission of science by generating fundamental knowledge and research publications in the area of portfolio management.
Phase 1: NSF employees
In the first phase of the project, we focused on serving the internal NSF audience. My contribution to the project was as follows:
I planned & conducted formative research in the field in order to identify user groups inside the NSF and understand their needs. I focused the data collection around questions such as:
- What are decisions made by NSF employees on a regular basis?
- What information do they need to make those decisions? Where is that information located, in what formats, and how do they access that information?
- What are the larger goals of NSF employees? What makes them feel successful? What are their big motivators?
- I closely supervised a graduate research assistant who assembled the user modeling report containing 3 personas. We published a paper in the Proceedings of HCI International that presents the results to an academic audience.
I chose the most vulnerable persona as the primary one. I lead the interdisciplinary research team through a design exercise where we created a context scenario for our primary persona and extracted design requirements – what information would Matt need in order to begin understanding his funding portfolio and be productive at his new job? I generated early sketches based on our brainstorming. The general approach we took to knowledge mining and visualization is explained here and here.
I directed students as they started working on wireframes based on my sketches and coordinated the communication between the UX and technical teams. In the days before Slack, I used an internal team blog to track and communicate work.
I conducted early testing on the alpha version. My goal was to assess ease of learning. Could our users figure out how to use our Web application? Would they understand the interactive data visualizations and interpret them correctly? User feedback, documented in this report, included comments such as:
I feel this was designed for me!
This thing reads my mind!
We delivered DIA2 to the NSF and proceeded to focus on the external audience:
Phase 2: NSF external audience
The team identified STEM faculty members as the largest external audience. These are researchers who need to understand the funding portfolio in order to better target their proposals to the NSF.
I designed an interview protocol for intercept interviews we conducted at the annual meeting of the Association for Engineering Education, where we were most likely to encounter STEM researchers from various fields. I trained a number of graduate students and we all conducted interviews and collected the data we needed in 3 days.
I lead a cross-disciplinary team of graduate students from both the UX and technical teams through a 2-day affinity diagramming process, which resulted in one persona, documented in this report and this conference paper.
With an understanding of the second user group’s needs, I wanted to ensure DIA2 served them well. I lead the team through cognitive walkthrough exercises where we asked whether Dr. Anderson, our persona, would know what to do, and if he performed an action, whether he would know he was making progress towards his goals. I supervised one of my graduate students as she conducted usability testing with this new user group. This work resulted in a conference paper and her M.S. thesis.
DIA2 served about 2 million visitors in a 2 year period. About 2,000 users had created accounts. The project is over and the data is no longer updated, as is common with a lot of academic projects.
The research & design process, as well as technical aspects of DIA2 are presented in a paper we published in IEEE Transactions on Visualization and Computer Graphics. More research related to DIA2 is indexed on the project’s research page.
I’m sitting in a meeting with colleagues from industry and showing them the persona of a mid-career academic. They ask, “But where’s the EGO? We heard faculty members have big egos.”
So, I’ve been thinking about that… actually, for the past 20 years or so, since I was a graduate student. I think I’m only now beginning to see it – or to have the courage to articulate it:
Academia is bad for the soul. It is bad for mental health, for psychological well-being. The individual reward system of academia, the secrecy and ambiguity of evaluation criteria, the lack of control over the products academics are evaluated on – they all create a culture of fear. Fear invites different coping and protection mechanisms, one of which is the ego. Fed on a diet of fear and occasional success that carries the author’s name in bold letters, the ego inflates.
Or, at least that’s my experience of academia, from my point of view. (See what I did there? I am qualifying, minimizing, making my opinion smaller, more precise. Good academic thinking. Powered by years of learned fear of merciless critics.)
In my earlier years [of writing in academia] I tried the opposite approach – filling my mind with critics and naysayers. I would sit at my desk and picture the faces of my least favorite professors, my harshest and most cynical colleagues, and my most unforgiving online critics. If I keep them happy, I thought, or at the very least quiet, I’ll be good to go. The outcome was the worst-case scenario for a researcher or a social scientist: findings that were gently folded into a preexisting way of seeing the world; findings that carefully nudged existing ideas but did so without upsetting anyone; findings that were safe, filtered, and comfortable. But none of that was authentic. — Brene Brown, Braving the Wilderness, p. 4
If you’re an academic – I bet you hear those voices in your head. I bet your advisor, reviewer #2, or that scared, mean kid back in grad school – they are in your head. You write for them. They are in my head right now. I write in spite of them, pushing, fearfully, letter by letter, through the thick fog of fear.
I bet that most of the successful research you do is just that – careful, comfortable, safe. How else would it get published? You can’t take the risk now. You’ll take it after you get tenure. Then, you will shake that tree.
The truth is, it is awfully hard to be your authentic self in the publish or perish environment of scholarly research. And, ironically, it is hard to do your best work.
At the end of the day, success in academia rewards the individual – the sole hero, HIS name (and yes, it’s mostly his name, still, unfortunately). It is HE who won the prize, who got the grants, who created The Theory. There’s no account of the team – of the nameless, faceless graduate students, for example, who were instrumental to the work. No, it’s just HIM. The Professor. The Researcher. He is worshipped, adored, and dreaded at conferences. A cloud of timid, hopeful, terrified graduate students surrounds him. He hides, he tries to protect himself from the annoyance of people without whom THERE WOULD NOT BE A UNIVERSITY. Oh, wait. Those are undergraduate students. He doesn’t know much about those. He hasn’t taught an undergraduate course since his first years as an assistant professor. Undergraduates are a complete waste of time. They distract from The Research.
In academia, mentorship is a joke. Mentors fear mentees – the young, eager faculty, with so much more energy, enthusiasm, and hope. They are on the fast track, and they publish more, accomplish more. That’s threatening. As a mentor, I look bad by comparison. Old. Tired. Blase. I don’t publish as many papers each year. I don’t have the same kind of pressure, so I don’t. I’m worn out. I’ve got tenure. Counting down to retirement. In my own life, the mentors I’ve met who are not threatened by mentees can be counted on the fingers of one hand. The pattern is, tenured faculty bully talented, untenured ones.
Back to the ego. It is fostered, encouraged, demanded, by the nature of the system of evaluation and rewards. But, is it necessary? The most successful people I’ve met (by academic standards) are kind and humble. That is, I think, because they are free of fear. They have nothing to prove. The rest of us, are drenched in fear. Fear that our publications are not good enough. And it’s true, they are not good enough. Any research paper can be criticized and ripped apart. Any research study can be done better. Fear that our publication number is not high enough. How many publications does it take to be “good”? To be productive? The target is moving and ambiguous. The evaluators are anonymous, cranky, sometimes uninformed. We call them blind peer reviewers. Their assessment is unreliable. The same paper that is an embarrassing disaster in reviewer 1’s eyes is brilliant in reviewer 2’s eyes. The same paper that got painfully rejected from conference A got an award at conference B. Hilarious, isn’t it? Not when those are the standards that define your success.
You cannot know whether your work is ever good enough, not by the gold standard of peer review. You cannot know if you are good enough.
A healthy mind requires we divorce ourselves from our behaviors. Just because an act, or a behavior, is not good enough, this does not mean the person is not good enough. Bullshit. In academia, it does not work that way. As a researcher, as a writer, I put my heart and soul into those papers. My mind works on them day and night, during a movie, during vacation, who knows, maybe even during sex (do academics even do that anymore?). It’s non stop. You can’t turn it off. You can’t stop thinking about it. You see the solution in a dream, and then panic when you wake up having forgotten it. The work consumes so much of your life. Most of us have a hard time turning it off. I remember working on my dissertation, and how I could not take a break, even when I did take a break. I worked on it non-stop for 2 years. It was exhausting. But, you ask, don’t you have something else in your life, something else to make you feel worthy, accomplished, outside your work? I don’t know, maybe children. I don’t have children, I don’t know. I don’t have a life. I don’t have a hobby. Academics who have hobbies are losers. Who has time for that? Or for taking the weekend off? I remember the advice of a well meaning mentor, or maybe it was a workshop on academic life: You should be sure to take half day off during the weekend (oh, no – it was a talk from a university provost, at a celebration of accomplishments, telling us we can relax a bit now). Half a day. How generous! (Note for the assistant professors out there: this does not apply to you. Please work all weekend.) So, no, there’s not much else. If there is, you are lucky, and you must have worked really hard at it. Because, if you’re an academic, as long as you are awake, you are working. And your brain continues working on problems when you are asleep. So, then, you bet your smartest body part that that paper is not divorced from the sense of self. When anonymous reviewer #2 writes a snarky comment about your research methodology, you read it as a comment about your own self, your worth. Many a therapist would say that is not healthy.
As you can gather, this entire experience is rather unpleasant. It is full of fear. Fear of the snarky comments. Fear that my work is not enough in quantity or quality. Fear that I am not enough. So then, what does the poor psyche do, to protect itself? It builds a big bubble around it, a big bubble I’ll call ego, and it fills it with hot air and it feeds it with acts of bullying that reassure it that, after all, I do have some power – power over something, someone, that starry-eyed assistant professor, that hapless student who dared write me an email.
I’ve been noticing a lot lately, in various writings (Tara Brach, Brene Brown, Gary Zukov) – that fear is related to feelings of powerlessness. I plan to explore that in the near future.
Academia operates on powerlessness. You are trained to be fiercely independent. Yet you have no power over whether you even get to do the work you want. You depend on the unpredictable game or research grant funding (oh, what a circus that is!). You have no power over your publications. You write them, but you don’t know if and when they will see the light of day. And yet, your success – your job, your salary, depend on a certain number of high quality publications seeing the light of day in a given time frame that you have NO CONTROL over. You try to please anonymous reviewers, and to work to standards that are not only secret – they are infinitely ambiguous and debatable. It’s not that you’re being judged, like in a Kafkian novel, in secret, by criteria unknown to you. You are being judged, in secret, by criteria that evolve and shift as the evaluation is being conducted. What would Kafka say about that?!
It is no surprise then that this environment of powerlessness breeds fear. Fear breeds ego – and we create a world where we are too scared to be kind, vulnerable, authentic. So we become assholes. Yes, yes, there are exceptions. Fortunately, a lot of us academics are decent enough human beings, and introspective enough, that we are able, to varying degrees, to keep these fears under control, and to not let them rule our behavior. But it’s not easy. And it is not healthy.
Let me get it straight. I loved my job in academia. I loved what I was able to do, and that I was able to do it. A lot of great work gets done, a lot of lives are changed. But this does not make the environment healthy or positive for the people who call it home.
I’ve been thinking about the authentic self a lot. For me, the moments when I was able to be my authentic self in academia happened only in the classroom. A lot of magic happened in my classes, I think. Enough to keep me alive, happy, and in love with my job.
If it’s all so bad/sad, what is the solution? I don’t know. Perhaps we need to rethink “Publish or Perish.” Many much more informed minds have already thought about that. I believe it takes a systemic solution, but I can’t even begin to wrap my head around it.
On an individual level, I can tell you what helped me: yoga, meditation, therapy, friends who could see and love my authentic self, antidepressants, books, my love for students, my husband, my cats.
That being said, I love academia. I love the way it teaches you to think, the freedom it provides (tenure is not all bad!), the privilege of living a life of the mind. I know many people who thrive there, and some say I might be one of those. I was even happy there. I was very much in love with many aspects of my work. Yet I’ve always known that parts of it (well, the Publish or Perish part, specifically), is not healthy. I have a lot of fear, and it’s deep in my bones, and I know it is still keeping me from being my authentic self. But I think I’m beginning to feel ready to look it in the eye, thank it very much for trying to protect me, and work my way to authenticity.
I leave you with another quote from Brene Brown, something she said during her interview at Microsoft the other day:
I had so much fear about my career that I engineered smallness.
And the first sentence of her latest book, Braving the Wilderness:
When I start writing, I inevitably feel myself swallowed by fear.
What do we do about that fear, babies? How’s your soul doing in academia? Are you able to find a home there for your authentic self? Teach me.
Like any good academic, I leave you with a reading list. Here are some of my favorite books about academia:
#1 – Jane Tompkins, A Life if School: What the Teacher Learned (nonfiction)
#2 – David Lodge’s trilogy of satirical novels about academia
My friend, my sister, the loving witness to the darkest patches of my soul. How I’ve missed you. In the past few years, the monster illness that ravaged your body and psyche only allowed me brief glimpses into your beautiful mind. You’d wake me up with a call on weekend mornings, and wait for me so we could have coffee together. And we’d talk. I’d hear of doctors, and medicines, and struggles, of nightmares and hospitalizations, and of hope. Maybe this time it will be better. Maybe this time the good times will last.
We’d always call each other “iubita” – my love. Even though you didn’t know enough happiness in this brief life of yours, you knew love. You gave it with grace and generosity and joy. The monster didn’t touch your soul.
You’re brave, my love. You fought the monster out of love for those who loved you and you did not give up on us. I know death was an appealing relief but you didn’t seek it, because you did not want to hurt those who loved you. You endured, until your body didn’t.
It was a mild winter evening – the first snow of the season, that made Bucharest quiet, clean, and magical. We had both decided to walk after high school rather than catch public transport nearby. I walked up to you, standing in the falling snow in front of the Bucharest Opera house, snow piling on the brims of your black fedora, waiting for your bus. I barely knew you. I was sad. I opened up. You said, “Today, I’m strong. Today, I’m happy. I can support you.” And we supported each other, inseparable, through this hard time called adolescence, and then through this hard thing called life.
You’d never admit it, but I gave you your first cigarette. We’d spend our high school days smoking and eating dark chocolate, skipping class and taking walks, going to coffee shops, figuring out life. At school, we’d stand controversially close to each other and enjoy the side glances. If the boys didn’t work out, we were going to end up together, taking care of each other, into old age and sisterhood.
You threw me the most epic birthday party for 18 – coming of age. Somehow, dunking me into a tub filled with champagne was part of the deal. Ever so thoughtful, you had clean clothes and new underwear ready for me to change into 🙂
We wrote letters. Long, handwritten letters. One day, you sent me the box of letters you had kept. You thought they were valuable, maybe there’s something there, maybe we’ll publish one day – but maybe, you didn’t trust yourself to preserve them. That box has moved with me from place to place. I caressed it just last month when I unpacked in my new house. Sometimes your letters puzzled me, your metaphors not always accessible to me – those metaphors that earned you a poetry prize within a year of moving to Germany. Your mind was sharp and switched from philosophy to math and physics, but to me, your gift with words was the most precious. I kept waiting, I thought, one day, that book you’ll write, it will be phenomenal. The monster took that book away from us.
You leave behind The Boy. I remember, as if it was yesterday, how you described, in a letter, meeting him. He was not the most handsome of them all, but so smart, and so kind, and you just talked and talked through the night. The Boy has been your partner for maybe 20 years now… he loved and cared for you more than a parent. He was with you until the end, and I am so grateful for him, because I know, your life would have been even shorter, even more tragic, if it weren’t for him. So many times I had sent thoughts of gratitude and relief that he was in your life. The Boy, he could still see you, find you in there, even when I couldn’t. I love him so much for that and I’ll do my best to take care of him, as you would want me to, I know.
I so wish, iubita, it had happened in the middle of a phone call. I’d have wanted to be there with you. It kills me that you were alone when your body collapsed. We were so close, you and I – whether we didn’t talk for a day or a year, we never lost closeness. I pray you are in a good place, free of suffering, and that our spirits remain close, connected in the bond of our sisterhood – a sisterhood like no other.
Te iubesc, iubita. Rest in peace.
It feels like I just returned from the annual ASEE meeting. I presented a paper about a topic near and dear to my heart: the new undergraduate major in Human-Centered Design and Development (HCDD) I spearheaded at Purdue.
The paper tells the design story (birth story) of the new program. I took a user-centered approach to curriculum design, since that’s what I know best. I think one of the most valuable tools that came out of it was the vision persona. And, of course, the program itself. 🙂
The paper is available online (you can read it here) and the slides I used are below.
I am so pleased that we launched the redesign of DIA2 and the new homepage this weekend! It’s been a long and fun journey!
DIA2 is a Web application for knowledge mining and visualization of the NSF funding portfolio. Anyone can use it to explore where NSF funding goes, how it’s distributed geographically, across NSF divisions, across topics, and institutions. You can explore collaboration networks of researchers who worked together on proposals, identify who’s well connected in a field, and figure out what NSF programs and program managers have funded research similar to yours.
I’m happy to have been involved with DIA2 since the very beginning, as a co-Principal Investigator (co-PI). I led the UX team for the project. We started with user research to understand user needs, and moved through ideation, wireframing, testing, the whole 9 yards. It’s been very rewarding to hear users say, “This thing reads my mind!” and “I feel it was designed for ME!” Perhaps best of all, DIA2 gave me the opportunity to work with and mentor many talented students. All DIA2 “employees” have been students working under a PI’s supervision. I am so proud of them!
If you’d like to, go check DIA2 out for yourself – it’s available for all at DIA2.org.
Or, read some research papers about it:
Using visualization to derive insights from funding portfolios. In IEEE Computer Graphics and Applications, 2015.
DIA2: Web-based cyberinfrastructure for visual analysis of funding portfolios. In IEEE Transactions on Visualization and Computer Graphics, 2014.
Portfolio mining. In IEEE Computer, 2012.
I came across this article in HuffPo about a new app some students created that can help you identify your most toxic friends. They call it an art project, but I seem to recognize here a common structure for research projects in HCI. So, if you’re my student looking for thesis ideas, read on. 🙂
The recipe goes like this:
- Take a problem or issue from the social world (e.g. toxic friendships, collaboration, long-distance family relationships, etc.)
- Create a technology that mediates how people deal with that issue – ideally, the technology should improve the human condition or raise critical questions.
- Evaluate the technology.
- As a result/consequence of evaluating the technology, illuminate some aspect of and contribute knowledge to #1. And/Or, at the very least, derive design implications for this type of technology.
Some examples of papers following this structure: