Blog

New Communications Measurement and Evaluation

[Notes from Track1-session1, New Comm Forum]

Blake Cahill, Visible Technologies overviewed a couple of case studies of Visible Technologies clients and their online conversation analysis efforts.

Janet Eden-Harris, Umbria. Umbria was created to tap into and aggregate “unstructured text” and mine conversations. Umbria analyzes patterns of conversation – different groups of people speak differently. Umbria focuses on 4 areas:

  1. Tell me about my brand – not by doing surveys, but by listening to spontaneous conversations taking place in social media
  2. Tell me about my industry – are there trends I need to be paying attention to?
  3. Tell me about my customers – many people blog, and they say a lot about things they care about. Blogs can help you understand what your customers care about. For example, Umbria analyzed blog posts by dog lovers. Interesting patterns emerged: Gen Y talk about pets as accessories “How do I look with my pet?” Gen X talks about how to integrate the pet in family life. Baby Boomers talk about pets as people/companions – they talk to them, dress them up, etc. 🙂
  4. Tell me about my product – online conversations produce ideas about new products. Many new products fail, but if you build a product based on existing conversations, it might be more likely to fill a need. Umbria grouped online conversations about pets into smaller topics. If the conversation is positive, this means that particular need is being met: e.g. organic food, pet daycare. Negative conversations revolved around traveling with pets. Umbria’s client, DelMonte, saw a product idea there. They created a line of products for traveling with pets and created online content and information about traveling with pets: tips, pet-friendly hotels, etc.

Q&A

Q: Is what you’re talking about monitoring or measurement?

A: If you track monitoring over time, it can become measurement. You don’t track (only) eyeballs anymore. You track the change of sentiment in online conversations, and ultimately, you need to see if the social media campaign ties back to sales.

Q: What is Umbria’s data universe?

A: Umbria has the tools to collect tens of thousands of blog conversations. You can never get them all, but for one client, you might analyze about 10,000 blog posts.

Q: Is traditional business segmentation falling apart? Can you trust computers to analyze the data and come up with categories?

A: You cannot trust computers. You have to oversee the data. But it’s not feasible for humans to analyze every single comment. Segmentation is not dead, but you have to understand that one person fits in different segments for different contexts. The same person might be a bargain-hunter when shopping for cleaning products, but think nothing of spending $5 for coffee at Starbucks. So segmentation makes sense in specific contexts. It can be a useful and powerful tool.

Q: How do you code data? Is “positive” the same for Dell, Ernst & Young and dog food?

A: Jane answers: At Umbria we use natural language processing algorithms to analyze comments and identify: age, gender, and sentiment. If you show comments to people and ask them to identify sentiment, inter-coder agreement will be only 65%. Algorithms can only get close to that, but they can’t get better than that. Sentiment is really hard to identify and code. So you have to keep working on teaching the software how to code and score. Also have to keep in mind that language and language patterns keep changing.

Q: What’s it going to take to move these technologies to analyzing more than text?

A: You don’t look at comments individually, you have to look at interaction and conversation threads. For audio and videos, we look to transcribing the audio to text and having it analyzed.

Q: Is there a way to assess if blogs are increasing or decreasing in importance?

A: K.D. Paine: Don’t ask me, ask your audience. There are all these tools out there, find out what your particular audience is using.

Q: If I’m a nonprofit and don’t have $10,000 to spend, what do I do?

A: Do a quick keyword search on technorati, etc. and look through the conversations. Even if you don’t do the detailed segmentation we do, you can get a very good idea of online conversations on topics you care about. If you don’t use any tool or technology, you just have to read the posts & comments. So, what do you look for when you read them? You can note sentiment, visibility, themes & trends that emerge from those conversations, etc. Richard (Dell) explains you shouldn’t be afraid of going through comments manually. If you have the right searches set up, it takes 60-90 minutes to go through about 1,000 post. Richard responds to about 15% of blog posts. It’s likely that a nonprofit doesn’t have the same volume of comments as Dell, so reading comments using a feed reader is entirely feasible.

Q: Have you thought about open-sourcing your algorithms so smaller companies can use them?

A: Blake: We’re still working on improving those algorithms. We’ve been focusing on perfecting our technology. We’re operating at the enterprise level, but there are many people out there who provide basic services at very low prices. Jane: Companies will probably not do this, but there are tools being developed in academia which will become public domain.

SNCR New Communications Forum opening keynote: Joseph Jaffe

[cross-posted from New Communications Review, I’m live-blogging the 2008 SNCR New Communications Forum]

Notes from Joseph Jaffe’s opening keynote:

There are millions – millions of conversations going on around us: powerful, engaged, influential conversations: Isn’t it time we join them?

The world has changed. The consumer has changed. Why hasn’t marketing?

Is the consumer in control? Are organizations in control? Nobody’s in control – this is total anarchy. We seek order and control, but that’s a fantasy world, a false sense of control. Some organizations, such as P&G, are trying to give that control to consumers – but consumers don’t want control. They want to be listened to, respected, engaged.

Marketing theory is very outdated – based on simple top-of-mind associations. Would you rather have 10 quality relationships or 5 million impressions? Most marketers want the impressions. See the Comcast example – the first thing that comes to mind (and in search rankings) is the video with the Comcast technician falling asleep. That gets a lot of impressions, but are they valuable? If you cultivate 10 quality relationships, those will branch out and in the long term will bring much more value than impressions.

Moving marketing from 4 P’s to 6 C’s: Content, Customization, Commerce, Conversation, Community, Content – all revolving around the Consumer.

The history and future of media: Moving from the one-to-many model to the one-to-one model (personalization), to one-from-me (search). But the model that characterizes social media is the many-to-many model. The model can work, or can suffocate you if you don’t know how to listen. In some ways, social media has gone to a reverse one-to-many model, where the previous targets (consumers) have become the broadcasters.

Communication vs. Conversation: Communication can only take you so far. Communication gets your foot in the door, but conversation gets the consumer to open the door and invite you in. The way you do it is not to spend lots of money: “Don’t outspend… outsmart.”

SNCR survey: What does conversational marketing mean to you? Some definitions include: less hype, faith & trust in your brand, partnering with consumers, etc. Going back to the Cluetrain Manifesto and the ancient bazaars, those people knew conversations: those markets were conversations. Commerce was a social experience. But we’ve moved from the bazaar/market model to impersonal malls. It’s time to put the social & sociability back in media – and in marketing. Marketing doesn’t have to remain a spectator sport. It’s time to entice consumers to come out of spectator/lurker mode.

In a recent SNCR survey, 41% of respondents anticipated spending 10% or more of marketing budget on conversational marketing. Many CMO’s right now get it, and are ready for conversational marketing. You want your customers to trust you – but do you trust them?

The answer is not to spend absurd amounts of money on fireworks displays (advertising) – how many Superbowl ads can you remember?

It’s time to blend and mesh the worlds of marketing, PR, and advertising, to achieve transformational change. The biggest risk is to spend $4 million on a campaign no one notices. 90% of advertising is wasted. It’s time to reconsider our strategy.

The problem is we don’t nurture the young, fresh smart ideas and we don’t invest enough time in them. The seeds of conversation are not magic beans. Conversation will not happen overnight. Companies need to invest long-term in conversation and maintain that commitment.

At the end of the day, I’m a storm chaser. If you want to understand change, you have to be in the heart of the storm. I stick my neck out. It’s been chopped off a couple of times – but it keeps growing back.

Predictions based on recent SNCR research data:

  • by 2012, companies will have conversation departments. They will listen to customers. They will allow employees to engage with customers. They will sponsor the sand box – and create spaces for customers to engage with each other
  • campaigns will change radically to include influence outreach, blogger activation, conversation monitoring & optimization, etc.

So, where’s the catch?

  • talent: not enough people in the business right now. Advertisers are part of the problem. They’re not the creators; customers are.
  • measurement: not ROI, but return on experimentation; or return on infinity. Right now, there are 4 categories: sales, good PR, improved brand health, more intimate knowledge of customers – the last 2 are the most interesting.
  • integration: 55% of survey respondents stated that conversation should be a combination of marketing, PR, & advertising.
  • organizational buy-in: right now, change happens at the individual level, not at the organizational culture level; but that often means that change happens at the tactical level, and it does not radically change the way the organization operates
  • control: you lose it. Recall T-mobile claiming they own trademark on the color magenta and issuing a cease & desist letter to engadget. In response, many bloggers displayed “T-mobile sucks” magenta badges. 3H: humanity, humility, & humor.

Interview questions

On the PR network PROpenMic (if you haven’t joined yet, I think you should!), Phil Gomes, VP Edelman Digital answers questions from PR students & faculty. Below is the most recent video, in which Phil and a series of other Edelman employees answer the question:

What questions do you always ask in a job interview?

 

http://static.ning.com/PROPENMIC/widgets/video/flvplayer/flvplayer.swf?v=3.0.1%3A3917 Find more videos like this on PROpenMicIf you are a PR practitioner reading this, please add your favorite interview questions in the comments. If you’re a student, feel free to add an interesting/difficult interview question you have been asked in the past.

Silence

Every day, I feel bad about not posting on this blog. If I could get two thoughts together, I would.

Those who follow me on twitter know that I traveled to Romania over Spring break to visit my grandmother, who’s going through a very slow and painful recovery after hip replacement surgery.

Fortunately, this is the first time in my life so far that I’ve had to deal with a loved one’s suffering. I’m not dealing with it very well. It’s consuming my peace of mind, affecting my clarity of thought, and pretty much pushing me towards depression. I’m not proud of not being mentally strong enough to deal with this better… and thus, more guilt and more distress.

So, yes, her ups and downs have been affecting my life & my work. When she has a good day, I’m full of energy and ready to take on the world. Then she has a relapse (like today, when she did not get out of bed even once because of debilitating back pain) and my mind becomes a messy foggy worried knot.

I’m trying to learn about equanimity and offering compassion without letting my mind be deeply troubled – a Buddhist concept that takes years of practice to achieve.

Until I work through these things and figure out a way to bring my mind back to work, and social media, and things that all of a sudden seem dangerously unimportant, I am retreating into silence.

If I have something good to offer you, I will. Like this: PR Open Mic, a shiny brand new and very promising social network for PR people (students, faculty, and practitioners). I hope you join.

If you have any advice for me, any tips & tricks for dealing with this situation compassionately but with only a healthy amount of empathy, please let me know.

Twitter in education: Practical solutions

We talked at EDB about using twitter with students: the benefits, the drawbacks, the logistics – including the 48 hours of twitter assignment that Karen Russell and Kaye Sweetser use. One logistical issue was to help the students find and follow each other among the professor’s long list of followers. Karen Russell solved this problem by getting a new account to use with her class. However, one thing I like about twitter is the help I sometimes get from other people who jump in the class discussion. Creating a new account would make it impossible for my students to meet my twitter community.

Here are the 2 solutions I found, and I document them here so other instructors can use them (and so I don’t forget them!).

1) – ask students to use a specific number (or other set of characters) as part of their twitter user names. For example, the course number, or any other random number. Then, they can scroll through the list of followers and follow all the other people whose user names contain that number. This works well, but sometimes students dislike interfering with their freedom to create a user name they like. The second solution solves this problem, too:

2) – e-mail students a distinct, clearly different image that your other followers are unlikely to use and ask all students to upload that image as their twitter icon. Instruct students to scroll through your list of followers and follow all the people identified by that particular image. After the students have identified and followed all the other class members, they can upload their own photo to their twitter profile.

Another challenge is to keep track of a conversation students carry on a particular topic. Their tweets might get lost among the tweets of others you’re following. Ask your student to use hashtags (#) followed by a specific code so that Twemes will collect all (in theory, at least) the posts on that topic. For example, we discussed symbolic interactionism in my Communication Theory class and marked all tweets about it with #si. You can see the resulting conversation indexed on twemes. I noticed Twemes didn’t pick up ALL students’ posts, so I wouldn’t rely on it to assign participation points, at least not yet.

Do you have other practical solutions or ideas for using twitter in higher education? Please share them in the comments!

Need more on twitter? Here are some of my favorite links:

Recommendation letters and references

I’ve had some conversations with students lately and I noticed they know very little about recommendation letters. In the spirit of transparency, I thought I’d provide some information that can help students make the best decision about asking for recommendations and references.

What goes into a recommendation letter?

When I write a rec. letter, I’m expected to mention how long I’ve known the student and in what capacity. Hint: If I don’t know you well enough (i.e. you haven’t taken a class with me, you haven’t worked closely with me on PRSSA or something similar), it’s best not to ask for a letter.

Next, I’m expected to explain why I recommend the student. What are the student’s demonstrated abilities that make her/him right for the job? By demonstrated abilities, this means that I have to support my claims with specific examples from your performance. This often involves talking about your assignments and class performance. In fact, Clemson now requires you fill out a FERPA waiver* stating you allow me to discuss this confidential information in the letter. Hint: If you haven’t performed very well in my class, it’s in your best interest not to ask me for a letter.

Often, I’m expected to rank you among my other students. For example, I can state that “this student was in the top 1% of her class” or “in the top 5% of students I’ve ever worked with.” Hint: Ask for a recommendation from a professor who can rank you (very) high.

What should you send me along with your request for a letter?

First, think about who can give you a great recommendation based on your great performance. If you ask me for a letter but you’ve not done great in my class, you put both of us in an awkward position. I usually avoid to write letters if I can’t say that I highly recommend you for a position, without any reservations.

Second, contact the person and ask if they could write you a letter by a certain date. Or, ask if they agree to be listed as a reference. Never, ever list someone as a reference without getting their approval first! If they do agree, then:

Third, send a formal request including the items in the FERPA waiver*. Attach a current resume and the information about the position you’re applying for. Include a firm deadline by which the letter needs to be received (no, that cannot be “tomorrow” – it should be at least one week).

If you’re an employer and you read this, can you help out? What do you look for in recommendation letters professors write for their students?

FERPA waiver*

Your request for a recommendation letter should include the following information (which you can type in the body of an email):

PERMISSION TO DISCLOSE STUDENT RECORDS UNDER THE FAMILY EDUCATIONAL RIGHTS AND PRIVACY ACT (FERPA)

I, _______, am currently or have been a student at Clemson University. I hereby give Clemson University permission to disclose the following student education records under the following conditions:

1. Student Education Records to be disclosed:
______________

2. Person or entity to which the above-referenced Student Education Records can be disclosed:
______________

3. Purpose for which the Student Education Records can be disclosed:

______________
4. This permission to disclose Student Education Records will remain in effect until _______________

Student Name

Student Signature

Date

Edelman Digital Bootcamp

I’m live-blogging EDB, look for several updates throughout the day.

Other EDB updates: twitter, EDB website & blog

Session 1 – Social Media 101

Erin Caldwell kicks off the day with her personal story. As an Auburn student, she became familiar with the PR blogosphere (see this blog’s blog roll for a start), she started the Forward Blog and built her reputation online. Edelman contacted her and by the time she interviewed, they pretty much knew they wanted to hire her. Being able to use social media & building your professional reputation online can help you get a job in public relations, whether you want to practice online or offline PR.

Team Edelman introductions and personal stories about social media a-ha moments. Different stories, different people, with technology backgrounds varying from tech guru to “barely able to turn on a computer” – but all share passion, curiosity, and love for their work [9:05 am].

Who’s here from Edelman: Chris Broomall, Erin Caldwell, Steven Field, Phil Gomes, Jena Kozel, Monte Lutz, Stephanie Wasilik. Bios here.

Educators’ track

Session 2: How PR practice uses social media

PR educators introduce themselves and talk about: helping students establish connections between social and professional uses of social media; motivating students to learn social media; disparity between expectations (students know all about social media) and reality (students are not familiar and even intimidated by new communication technologies – RSS what?!) [9:50].

Phil Gomes provides the big picture of current social media use in PR.

Phil saw blogging as the ultimate media relations tool – you can demonstrate journalists that you’ve read them, and have reacted to their work.

Don’t think of it as a technology problem; the technology in social media is easy to learn.

Phil describes his approach to teaching social media in the Chicago T4 lab. They spend one day immersed in online conversation analysis. Phil doesn’t believe in teaching products, so he teaches his students to use free tools to analyze existing conversations about a client. Tools you can use: bloglines, technorati, alexa, blogpulse, etc.

Job description for an entry PR job (assistant account executive):

  • administration
  • coverage + conversation tracking
  • list building + community & member-list generation
  • editorial/speaking calendar building + identifying client conversation-entry opportunities
  • list & opportunity qualifications + deep-dive analysis
  • team knowledge mgmt
  • AP + web style
  • etc. [10:20]

Phil loves the advanced search in technorati that identifies all links to a specific URL. For example see who links to this blog.

What Phil looks for in a job candidate:

  • intellectual curiosity
  • up-managing skills (free of CLM -career limiting moves-)
  • an examined, omnivorous media consumption life (facebook or myspace? why? WSJ or NYT? why?)
  • basic knowledge of social media tools

== Tired : Pitching :: Wired : Engagement ==

Don’t write self-referential posts (what I did/wore today) – be useful.

 

The ideal job candidate would have:

 

  • perspective; understanding of online communities – Phil loves this Wired article
  • good online writing skills: concise; interlinked
  • online law & public policy (DMCA)
  • communication, technology & society
  • critical consumption of media
  • an understanding of rules/culture of online engagement [10:40]

Session 3: Social Media Tools in the Classroom

Session 4: Social Media Assignments

Educators shared assignment ideas that make use of social media in various PR and communication courses. [4:15 pm]

Session 5: Wrap-Up – Best Practices

Students sum-up some of the lessons that stood out:

  • transparency
  • there are many free social media tools out there!
  • update your site/blog often
  • blogger relations require a different mindset

Edelman practitioners were impressed to see students taking time out of their weekend to learn these skills – there’s hope they’ll have new colleagues with the right skills set.

Phil Gomes wrapped-up the day.

Congratulations UGA students for organizing a great event! You’ve worked very hard and it definitely paid off, it was a very successful day!

[5:30 pm, signing off]