The Enterprise Social Software Measurement Pyramid

Measurement remains a bug bear for internal social business initiatives. Some enterprise social software tools are implemented with little or zero budget, with an expectation that even if the return isn’t measured it is expected to be positive. In part, this approach is driven by the difficultly of measuring the impact of social tools in the workplace.

Unfortunately this can create problems later down the track, such as:

• Competing social and non-social tools exist and a decision to endorse one over any others is seen as necessary to support better adoption – how do you determine which has the most impact rather than popularity?
• Social tools are perceived to be creating a negative impact on the business (e.g. information overload, fragmentation of corporate communication, risk, etc) – how do you prove business value outweighs these concerns?
• After success with early adopters, the adoption ceiling is reached and a decision to invest in further support or technology needs to be made – how much should be invested?

Businesses that do decide to measure the impact of the enterprise social software they have introduced can still run into problems if they do not have base line metrics to compare against. Without knowing where you started, you may not be able to prove that the technology has provided a direct, positive impact. But due to the emergent properties of social software, how can know what data will give you the base line you will need?

To deal with this measurement challenge, we need to first recognise that organisations are typically blind to the benefits of social software. Conventional management thinking (and financial management practices) prefer simple aggregate data, but enterprise social software provides a potential solution for a myriad of organisation problems. The true value of solving these problems can only be calculated by identifying the cumulative benefit of each after the fact – and even collecting that data has a cost. Clay Shirky calls this problem the Coasean Floor.

As a result, any measurement model for enterprise social software must deal with the following issues:

• The cost of measurement must not outweigh the benefits.
• Measurement must be based on visible data and evidence.
• Measures must be linkable to business outcome.

The following Enterprise Social Software Measurement Pyramid addresses this pragmatically by differentiating between high value, but hard to gather metrics and other supporting data and evidence.

Enterprise Social Software Measurement Pyramid

In order of importance:

• Group or process specific KPIs – Key performance indicators (KPIs) provide specific improvement measures for a workgroup or business activity (better performance, reduced effort, risk avoidance, etc). These are not likely to be global KPIs.
• Qualitative examples & network measures – Individually these measures provide evidence of specific examples of business value that the social business system provides. Network measures (of the user and information network the system supports) are used to demonstrate organisational cohesion and value flow between people.
• User activity, trends & demographic data – This basic data provides an overall picture of who is using the social business system, which can be used to support higher order measures.

This model can be used at three broad stages:

1. Anticipate – during the development of the business case and requirements, we can anticipate benefits, based on use cases (at Ripple Effect Group, we apply a user-centred design approach to this stage).
2. Base line – as the social software tools are introduced, we need a base line to measure against. This base line can also be used to further argue the case for change with the people who will be using the new social tools.
3. Evaluate – this is an ongoing process of measurement and reporting, however this data needs to be evaluated so that the KPIs can be adjusted in response to emergent outcomes.

Relatively speaking, KPIs will be far and few between but will provide the most important metric. Such KPIs can be anticipated in advance, then refined at the base line phase before being qualified during the evaluation phase.

Qualitative examples (e.g. success stories or anecdotes) & network measures (using social network analysis) provide the most versatile measurements, although it may still be difficult to use them to demonstrate value to the satisfaction of all stakeholders. However, this evidence can be be used during all phases:

• Identifying problems in the current environment, to support the case for change.
• Creating base line metrics that can be easily compared in the future.
• Demonstrating emergent value during evaluation – this may become the basis for KPIs for unanticipated benefits areas.