Measurement

Measurement: how effective is your international communication?

If something is worth doing, it is worth doing properly. Unfortunately, it is difficult to tell if ‘it’ was done properly unless you invest the necessary resources into finding out. It’s important to try, because you can’t correct missteps, or focus more of your resources on the things you are doing right, if you have no reliable corroboration of what things are effective, and why.

Often, when I have asked the question over the years – how effective is your international communication? – I’ve received too many blank stares, complicated explanations, excuses (usually blaming lack of budget, difficulties caused by language/culture/decentralization), or voicing disbelief that PR can be measured). Essentially answering with a shrug and, “how long is a piece of string?”

The Barcelona Principles of communication measurement

For reasons I won’t bore you with, the communications industry (AKA public relations) as a whole has historically been poor at measuring what it achieves in a valid way. There have always been individuals that have argued for valid measures, and in more recent years a coalition of interested bodies led by the International Association for the Measurement and Evaluation of Communication (AMEC) has established a set of measurement principles, known as the Barcelona Principles, which were recently updated.

I whole-heartedly endorse these principles, which in the past I’ve found extremely useful in helping international technology companies to roll out the ball of string, and think of relevant, yet affordable ways to measure it.

I share the Principles below for those who are interested. Shout out to my friend Mike Daniels who played a valued role as previous AMEC Chairman.

Principle 1: Goal setting and measurement are fundamental to communication and public relations

  • Measurement and evaluation against defined goals and SMART (i.e. specific, measurable, attainable, relevant and time-bound) objectives are fundamental to good communication and PR programs. They are critical to any communication program, whether it be a single campaign or an on-going effort where the results are incremental over time.
  • Goals can be quantitative or qualitative yet should address target audience and what about them the communication program is intended to affect, how much of a change represents success, and by when this result should be achieved. This should be based on a clear understanding of the target audience including their current positions and views, as well as the context in which the program is intended.
  • Measurement, evaluation and goal-setting should take a holistic approach, including representative traditional and social media; changes in awareness among key stakeholders, comprehension, attitude, and behavior as applicable; and impact on organizational results. It should be integrated and aligned across paid, earned, shared and owned channels where possible.

Principle 2: Measuring communication outcomes is recommended versus only measuring outputs

  • Outcomes include shifts in awareness, comprehension, attitude, behavior and advocacy related to purchase, donations, brand equity, corporate reputation, employee engagement, public policy, investment decisions, and other shifts in stakeholders regarding a company, NGO, government or other type of organization.
  • Practices for measuring the effect on outcomes should be tailored to the objectives of the communication program. Both quantitative and qualitative methods should be used as appropriate.
  • Standard best practices in target audience research including sample design, question wording and order, and analysis should be applied.

Principle 3: The effect on organizational performance can and should be measured where possible

  • To measure results from communication for an organization, models that determine the effects of the quantity and quality of communication outputs on organizational metrics, while accounting for other variables, are a preferred choice. Related points are:
    • Demand for models to evaluate the impact on target audiences of PR and communication activities in an integrated environment is significant and increasing.
    • Practitioners need to understand the value and implications of integrated marketing and communication models for accurate evaluation of each channel including PR. There exists an ongoing need to develop PR measures that can provide reliable input into integrated marketing and communication models.
    • Survey research can also be used to isolate the change in purchasing, purchase preference, engagement, recommendation or attitude shift resulting from exposure to communication initiatives versus other channels.

Principle 4: Measurement and evaluation require both qualitative and quantitative methods

  • Tracking surveys can do a good job in measuring quantitative change in outcomes. Qualitative methods can often add a needed dimension to better explain the quantitative, and may in some cases be preferable to measuring overall results.
  • Specific to the measurement of media resulting from public relations programs, overall clip counts and general impressions are generally meaningless. Instead, media measurement, whether in traditional or online channels, should account for:
    • Impressions among the stakeholder or target audience
    • Quality of the media coverage including, but not limited to:
    • Tone
    • Credibility and Relevance of the Medium to the Stakeholder or Audience
    • Message Delivery
    • Inclusion of a third party or company spokesperson
    • Prominence as Relevant to the Medium
  • Quality can be negative, positive, or neutral; the assumption should never be made that the results of a communication or public relations campaign or program are always positive or successful. Good measurement and evaluation allows for the possibility of negative or poor results from a campaign or program.

Principle 5: AVEs are not the value of communication

  • Advertising Value Equivalents (AVEs) do not measure the value of PR and do not inform future activity; they measure the cost of media space or time and are rejected as a concept to value communication, media content, earned media, public relations, etc.
  • Where a comparison has to be made between the cost of space or time from earned versus paid media, validated metrics should be used, stated for what they are, and reflect:
    • Negotiated advertising rates relevant to the client, as available;
    • Quality of the coverage (see Principle 4), including negative results; and
    • Physical space or time of the coverage related to the portion of the coverage that is relevant.
  • Multipliers intended to reflect a greater media cost for earned versus paid media should never be applied unless proven to exist in the specific case. This also applies to ‘pass-along values.’

Principle 6: Social media can and should be measured consistently with other media channels

  • Organizations need clearly defined goals and outcomes for social media.
  • Media content analysis should be supplemented by web and search analytics, sales and CRM data, survey data and other methods.
  • Evaluating quality and quantity is critical, just as it is with conventional media.
  • Measurement must focus on engagement, ‘conversation’ and ‘communities’ not just ‘coverage’ or vanity metrics such as ‘likes’.

Principle 7: Measurement and evaluation should be transparent, consistent and valid

  • All measurement should use valid methods and be reliable and replicable in the case of quantitative methods and trustworthy in the case of qualitative methods.
  • The concepts of integrity, honesty, openness and ethics are critical to this Principle. There are a number of organizations that have set relevant standards. These include, but are not limited to nor intended to be a complete list:
    • Media Measurement:
      • Source of the content (print, broadcast, internet, consumer generated media) along with criteria used for collection.
      • Analysis methodology – for example, whether human or automated, tone scale, reach to target, content analysis parameters.
    • Primary Research:
      • Methodology –sampling frame and size, response rates margin of error, probability or non-probability, screening criteria.
      • Questions – all should be released as asked (wording and order).
      • Statistical methodology – how specific metrics are calculated.
      • Identification of any potential biasing effects in the research itself or taking place in the broader societal context.

Add a Comment

Your email address will not be published. Required fields are marked *