Defining measurement standards for ePR and eReputation

Lack of consensus on measurement methods has plagued the communications and public relations businesses for the past decade. Not only do measurement goals vary between practitioners, actual methods for a same goal also do, leading to sometimes unreliable and non-repeatable protocols and rendering accurate benchmarking very difficult. Excessive focus on ROI, rather than on understanding the effects of PR activities on their target relative to predefined goals, has notably led to the dangerous use of Advertising Value Equivalency (AVE) as a proxy for financial outcomes.

Compounding this problem is the fact that, by allowing what used to be the ultimate target of communications programs – the public – to become a publisher in its own right and a mighty source of influence through peer-to-peer recommendations, new interactive and social media have rendered some traditional tools on content analysis all the more obsolete.

Measurement Standards

Standardization in Public Relations Measurement and Evaluation

The Valid Metrics Matrix, discussed in a previous post, is a useful attempt to provide a framework in which all items of measurement in a media analysis campaign are arranged in a logical order based on their contribution to the measurement goal and their place in the communications cycle.

A couple of weeks ago, David Michaelson, father of the Valid Metrics Matrix, sent me a report entitled Standardisation in Public Relations Measurement and Evaluation. The report goes further down the reliability/repeatability road by actually recommending specific measures for PR activities at every stage of the communications cycle, such as:

  • Target audience measures (awareness, knowledge)
  • Interest and Relevance
  • Intent
  • and Advocacy
  • Specific direct research question sets are suggested for each and every stage of the process, each having been show to be valid and reliable by statistical analysis.

    But for me, the most interesting part of the report is the chapter entitled Third party/intermediary measures that explains how to measure the relay effect of third-party publishing such as social media recommendations, peer reviews and online conversations.

    Measuring the impact of word of mouth on your online reputation

    While popular social media metrics such as facebook fans and twitter followers abound and can help to draw a complete picture of your online presence, they do little to measure the impact of online communities on your business or evaluate the efficiency of your activities on social media.

    What the report suggests is not new, but a time-tested method of media coverage evaluation transposed to social media: analysis of message accuracy online. Whether a community is commenting on your products, your CSR engagement or any of your reputation drivers, your communications towards it will contain specific messages that need to be relayed accurately. Relayed messages that are incorrect will help raise awareness but will be detrimental to most of your goals. Three specific measures can be put in place for this:

  • The presence of basic facts in community messages
  • The presence of misstatements or erroneous information
  • The absence or omission of facts

Measuring this is very straightforward: by defining two lists, one for messages that should appear and another for incomplete or erroneous messages that you anticipate, you can give your evaluation team a very precise brief or even automate the identification using advanced semantic analysis, for a very repeatable process.

Creating online communities and actively engaging with them is the best way to ensure conversations about your organization are happening in a controllable environment. And the method described ensures a very efficient ePR/eReputation evaluation process.

How are you measuring your social engagement?

================
Creative Commons image by cstmweb

Breaking free from reputation measurement silos

Silver Bullet In the first part of this article, I explained that no silver bullet single metric can adequately describe corporate reputation and described how most existing reputation measures fall into three categories:

  • Fashionable social media focused measures such as fans and likes
  • Primary research studies, periodically targeting either the general public or more specialized stakeholder groups in their surveys or focus groups
  • Monitoring-based solutions that evaluate in real-time the impact of (traditional, online and social) media exposure on a company’s stakeholders

While all these solutions have great advantages and some drawbacks, the real problem they present is a traditional one of ownership and measurement silos. Since results are rarely comparable and all target different services within the organisation, they make a bird’s-eye view interpretation very difficult.

Consistent messaging and consistent measurement

Reputation management has turned into a very fragmented market place with a very long tail indeed and the very definition of reputation has seen so many variations in the mouths of the numerous actors as to seem lost in myth. And it’s true that a formal definition of corporate reputation is hard to come by. The Financial Times Lexicon proposes the following, which is probably consensual enough for my use:

The term refers to the observers’ collective judgments of a corporation based on assessments of financial, social and environmental impacts attributed to the corporation over time.

More important than a definition, is the impact of reputation. The difference in corporate reputation between two companies mean that an identical event in both will induce different reactions from their respective stakeholder groups. The organisation with better reputation will recruit and retain better employees, increase profitability, lower legal costs, receive greater interest from investors and partners, navigate crises more smoothly, benefit from greater customer loyalty …

So reputation must be measured accordingly, as a perceptual capital in the eyes of all the organisation’s important stakeholder groups. Any (good) measurement system or study that focuses exclusively on one group and/or one department can be useful for tactical purposes, but in creating a measurement silo, it cannot provide the C-level with the complete dataset essential for strategic reputation management.

Just as consistent messaging from all communication touch points is essential for creating reputation, integrated measurement is a prerequisite for stakeholder analysis. And the two should hinge around the same communication pillars and messages.

Working with an integrated measurement framework

In Post Advertising Value Equivalent – New PR Measurement Metrics, I presented David Michaelson’s Valid Metrics Matrix, a table for organizing your various metrics into a meaningful framework.

Measuring reputation with the Valid Metrics MatrixThe matrix has two axes:

  • Horizontally, the progress from mere creation of awareness to the actions you would like your stakeholders to take
  • Vertically, the progress from company activity (press releases, community management, events) to intermediary effect (media coverage, blog posts from industry experts …) and finally to target audience effect (buying products, voting for a candidate, participating in clinical tests …)

The grid on the left is dedicated to reputation building. Click on it to access the complete version on the AMEC website.

Using such a framework, and having defined the most relevant topics on which you must communicate and measure perceptions, you can set goals and decide what indicators best represent success and track your progress within a meaningful context.

Sentiment EvolutionUsing our own ComSuite and ComDecision, for instance, you can define corporate messages, create campaigns and projects around them, distribute information organise events, meet with important stakeholders then measure open rates, click rates, stakeholder sentiment in surveys or social media, analyse media coverage, segments along countries, media types, sentiment scores, stakeholder groups, topics … ensuring all messaging and measurement hinge around common concepts whatever the service or targeted stakeholder group. You may also want to organise focus-groups or general public surveys. Combining all these reputation management tools within the context of the Valid Metrics Matrix guarantees that all the indicator values you have collected contribute to a common goal and can be interpreted with their true use in mind.

As usual, your feedback is welcome on this topic.

======================
Silver Bullet train Creative Commons image by Stuhacking

Post Advertising Value Equivalent – New PR Measurement Metrics

In a previous post, I lamented over the impression that the (deserved) worldwide ban of the AVE may not have been accompanied by enough replacement recommendations. Well, it’s time to make amends because yesterday’s PRSA/AMEC webinar, presented by David Michaelson, structured the field brilliantly.

Redux

Let’s repeat it once more: banning the AVE was not a rhetorical fad. It’s very calculation makes you examine your public relations work purely from an advertising point of view (ad space), thereby loosing much of your engagement work’s true value on the way. And, as it shows very little correlation with any meaningful outcomes, the formula cannot be used to bridge the gap between the CCO’s reporting and the KPI expectations of the other members of the board. However, the Barcelona Principles (AMEC declaration of PR measurement standards) always struck me as intellectually interesting in a French Declaration of Independance sort of way, but of very little field value to practitioners. To be fair, real life solutions have been described, notably at the London Measurement Conference but this webinar provided a very synthetic high level view, not found in other discussions, that every one should read and relate to.

Paraphrasing Dr Michaelson here would be of little value and his slides are available on the AMEC website. I will update this link asap). Instead, here are a few of my reactions to that very interesting work:

First of all, the backbone is a framework for describing the contribution of PR to the traditional marketing funnel. Imagine a matrix with one axis representing PR actors and (traditional) engagement stages (the content producer/the intermediary/the content consumer) and on the other,some marketing funnel stages (awareness, knowledge, interest, preference, action). Each cell is defines a precise situation to which a given set metrics apply best. So, to me, the completed matrix defines a measurement process that unifies MarCom quite elegantly.

The valid metrics matrix

Secondly, I believe this is an open framework that supports Barcelona Principle #2: Focus on outcomes (as in The Financial Value of Corporate Reputation) in many industries and situations. In the webinar slide presented above, sales are the desired business outcome and the horizontal axis follows a typical B2C marketing funnel. But the public relations and public affairs teams in many industries have other practical goals, and corresponding intermediate milestones, that could replace the funnel in the matrix. For instance, a pharmaceutical may focus on the market access of a new medicine. A bank may be discussing micro credits with the government of an emerging country or trying to extinguish a Wikileak fire (yes, I believe this measurement process would be great for crisis management scenarios). An energy major may want to monitor the impact of their engagement with local NGOs on their upstream activity. Each of these scenarios has a corresponding set of intermediate stages to replace the funnel and the corresponding matrix would define the measurement process for the campaign.

Thirdly, a partial disagreement on the integration of Social Media, and Facebook fans in particular, in this process. And this for two reasons. Reason #1: it’s my personal belief (and pet peeve, as regular readers will recognise ;) ) that Facebook Fans is the new AVE. It’s an easy metric to obtain and it flatters the ego in the same way. But, as argued previously (and heralded by Jay Baer) Facebook Fans only indicate previous affinity with a brand, not advocacy or the result of engagement. And, reason #2, it is my belief most Fans are attracted through reward marketing campaigns (Win an iPad, 10% off for new fans …) that turn a Facebook page into paid media. Still, people will use these metrics and at least this framework places them in a very meaningful context.

Fourthly, what I agree very strongly with, in terms of Social Media integration, is the focus on measuring conversations. And again, for two reasons: #1, The unidirectional PR process on the Y axis of the metrics matrix is still valid in a world with intermediaries such as bloggers and the media. But in social media that model no longer holds true as the consumer can also be the starting point of the conversation, even on the semi-owned territory of a brand page. #2, due to EdgeRank, Facebook’s algorithmic response to Dunbar’s social relationships limits, only a very small percentage of page updates (0.2% to 0.5%) will be seen by fans. The image below illustrates the typically low levels of feedback and engagement found on a status update. Much as lowering the rebound rate on a heavily trafficked page can have a drastic effect on sales, stimulating conversations with existing fans will likely have far more impact on desired outcomes than adding a few more fans.

Typically low Facebook feedback rates

To conclude, the framework presented in the webinar may seem daunting to already super-busy teams. But the point is not to fill in all the cells. Rather, it serves as a great navigational tool to contextualize what measurement you already have going and understand its results. We are currently working on a white paper describing what forms of measurement can be helped by platforms such as those Augure produce (outputting engagement data so that is can be used in other C-Suite software for business correlation, using indicators that have been shown to correlate strongly with business outcomes, integrating new media with traditional media, plugging into web-analytics, message retrieval, use of sentiment analysis for research …) and it will be very interesting to map the results into this matrix to further help structure and simplify the landscape.

If you have comments on this measurement framework, I’d love to hear from you!

Follow

Get every new post delivered to your Inbox.

Join 2,501 other followers

%d bloggers like this: