This article argues that there are five core measures that organisations should be tracking (and benchmarking if possible) over time to determine the effectiveness of their intranets:
-
Average time spent on the intranet per employee
-
# of page views per employee
-
% and frequency of employees that view content
-
% and frequency of employees that contribute or update content
-
Level of employee satisfaction with the intranet
Combined together and in conjunction with some kind of comparative data, these 5 measures can provide a valuable snapshot of intranet success.
However, not everyone necessarily agrees with this approach. There is a view that traffic, page views and the like are too simplistic and superficial as ‘real’ measures and don’t give an accurate or deep understanding of the underlying business value of an intranet.
Approaches to intranet measurement
For example, Nigel Williams in his article Judging your intranet by hits alone is really a miss, says that “if you are judging your intranet by hits, stop!” He then provides a list of more than 20 other variables that intranet managers should also consider tracking (check out this interesting LinkedIn discussion about the article).
Gerry McGovern says in his article How to measure the success of your intranet”, “intranet teams are measuring the wrong things, like traffic, hits and page views. Is more traffic a good thing? Why?”. He goes on to say “Focus on your employees’ time. Be relentless in seeking to save it. If you do you will create a great intranet.” (The logical conclusion being that measuring time-saved is a better measure of intranet success than traffic).
In It’s not about the outputs, it’s about the outcomes, Jonathan Phillips says “Outputs such as page views, or time per visit, are interesting numbers but don’t get to the heart of the matter. It’s the outcomes, or the results of intranet activity, that’s most important.” He then says, “These numbers are much harder to generate and report, but do provide better clarity of return on investment and of purpose of your intranet.“
And Anna Rydne says in Why Miserable Intranet Stats May Indicate Success “effective communication means keeping the employees informed in the shortest period of time. And that’s why high click rate and a lot of time spent on the intranet doesn’t necessarily mean success.”
Is it worth looking beyond the raw data?
So it seems to really measure true intranet success, you need to look beyond just the raw numbers. You also need to look at business performance and outcomes.
I don’t necessarily disagree with the general intent of these articles but it does seem like a lot of hard work (as Jonathan points out). And is it overcomplicating something to the extent that it discourages organisations from even attempting to measure intranet value? Why bother measuring page views, usage and traffic at all if this data can be interpreted in different ways.
In fact, many organisations don’t do any intranet measurement so asking them to commit time and effort in applying sophisticated techniques that measure business value of an intranet is a big expectation. Wouldn’t it be better to use some fundamental measures where the data is relatively easy to obtain, than nothing at all?
But does increased intranet activity necessarily = success?
From my research into effective intranets I have found that a good case can be made that tracking the 5 measures above - which is a relatively simple and straightforward task - provides a reliable indicator that the intranet is also delivering business value. More reliable than perhaps the articles above acknowledge.
In the article, If employees spend more time on the intranet, does this mean it's more valuable? I reveal some interesting statistics that show that the more time staff spend on the intranet - both as viewers and contributors - the greater it's value.
While the statistics can't show whether there is any correlation between intranet value and business outcomes (the ideal measure of success) it's fair to say that an intranet that is perceived as being valuable by employees will most likely result in positive business outcomes.
It's similar to determining the success of a book. Lets say there are two books about SharePoint 2013 - one sells 10,000 copies and the other sells 100 copies. Which one is judged to be more successful? Which one has provided better business outcomes to the SharePoint community? You could possibly argue that the 100 copy book could potentially add more value. Maybe a single company that followed the advice in the second book was able save more money than all the companies combined that read the first book (almost impossible to measure). But I would say that, on balance, the chances are that the more popular book would also provide the most value (I know which one I would prefer to author).
Or compare two coffee shops - one is always busy the other has only a few people. Again, it could be argued that the one with the fewer people is making a bigger profit and is more successful. Perhaps they charge $50 for a cappucino. Again, I would argue this is unlikely and that the chances are that the busy shop is more successful.
Wright’s Rule for Intranet Success
Based on this research, I’ve come up with a highly scientific formula that is a guide to intranet success. I’ve imaginatively called it: Wright’s Rule for Intranet Success.
The key word in this formula is ‘more’. What is more? More in comparison to what? For this formula to work, the measures need to be compared to something. A baseline or benchmark. This could be past measures within your organisation or measures against competitors or against industry standards. As Nigel rightly points out in his article, statements such as “I have a great intranet because we have 100,000 hits a month” are meaningless.
What is wrong is not so much the fact that it’s measuring hits but that the hits are not being compared to anything. A better measure might be something like number of page views per employee per month for the last 12 months. An even better measure would be average page views per month per employee against an industry benchmark. The ultimate measure would be to correlate these intranet measures with key organisational performance indicators such as profit/loss, revenue, market share, growth, research & development, and employee engagement (register for the Worldwide Intranet Challenge if you would like to benchmark the level of end user satisfaction with your intranet).
There may be certain situations where a case can be made that increased traffic or an increase in the number of users is a negative thing or a decrease is positive (read Anna’s article), but I would argue that these cases are not common and Wright’s Rule is a fairly solid guide.
Time spent as indicator of success
Time spent as an indicator of success is not a new idea. I was interested to see the following Tweet recently from Rupert Murdoch about one of the indicators the signalled the decline of MySpace.
Could the same 'hours spent' indicator be used for intranets?
And then there is this interesting interview with Jeff Weiner, the CEO of LinkedIn:
Interviewer: Are you succeeding in getting people to stay on LinkedIn longer during the course of the day, and is that a goal?
Jeff Weiner: Time spent has never been the primary objective. As we like to say, LinkedIn is not a service that enables you to pass the time, it's a service that enables you to save time. And that goes back to our mission, which is to connect the world's professionals to make them more productive and successful.
So, ultimately would we like our membership to become more engaged [ie. spend more time, click more things], yes, as a proxy for the value that we're delivering to them on a daily basis, and that's why we're introducing products like LinkedIn Today, that's why we have over a million groups now servicing LinkedIn members within a professional context.
What does it all mean?
While it’s a noble and worthwhile goal to look for more meaningful ways to measure and improve the effectiveness of your intranet, it’s important not to overlook or undersell the above 5 core measures as important, valid and legitimate. This is even more important for organisations that don’t measure anything.
So the big challenge for intranet professionals is to provide content and tools that employees need to be more productive and do their work more effectively. As Weiner notes, ‘time spent’ should not be a goal in itself but as “a proxy for the value that we're delivering to them on a daily basis”.
You may also like
Would you like to compare the level of staff satisfaction with your intranet?
If you are interested in benchmarking your intranet against 150 other organisations, why not register for participation in the Worldwide Intranet Challenge (WIC) free online benchmarking service. Read what participating organisations say about the WIC or find out more on the WIC FAQ page.
These are valid arguments for the intranet indicators you listed. I appreciate that you included opposing views as well. I think we need to look at different metrics to get a good picture of how the intranet is performing. One question: what do you think is the best way to measure employee satisfaction with the intranet?
Posted by: Alexis Rodrigo | August 22, 2013 at 12:34 AM
Thanks for your comment Lexi - we don't get too many on this site :-( To answer your question about employee satisfaction, I'd think a simple internal survey conducted regularly would be the best way measure employee satisfaction with the intranet. I know this is something IBM do
http://www.digitalworkplacegroup.com/2013/05/14/screenshot-ibm-intranet-simplificatio/
Or even just adding a few intranet related questions to an annual employee engagement survey. I know many organisations run these each year.
There is also the free Worldwide Intranet Challenge benchmarking service that we offer which allows organisations to compare their end user feedback with against other organisations.
Posted by: Andrew Wright | August 22, 2013 at 01:04 AM
The most powerful stats are often the simple ones. One caveat about "time spent" is that you need to compare this with people's actual use. One real life example: In one of my previous companies the average time spent on the intranet was around 19 minutes per visit! Awesome by the looks of it, but it turned out that because of the fact that the intranet opened automatically in the morning roughly 40% of the employees just let it sit there and kept it running for the entire day. They certainly boosted the "time spent" variable but it was also useless as a metric. Actually, we ended up using it as a benchmark that we needed that number to go DOWN and not up - not something we talked to management about as it would be a hard sell.
I guess what I'm trying to say is that simple stats are great but they must be corelated closely with your intranet objective.
Posted by: Martin Risgaard | August 24, 2013 at 06:28 PM
Great blog!
I have a benchmark for intranet stats: If I can personally influence it (read: corrupt, manipulate), then it has no place on my dashboard.
The best example I can give is [#page views per month] which I often see on dashboards. I can easily manipulate this number to get it to display anything you'd like by running an autorefresh plugin or code. You need a hit a target of 1m pages? No problem. Care then should be applied with the measure [average pages per employee]. Is this [#total page views / #total employees]? If so, I've just smashed your target...!
Time spent goes up? "Hey, congrats, your content is sticky!" or "Hey buddy, #fail, no one can find what they want..."
Intranet mechanical performance and intranet effectiveness are tricky beasts to balance. Be under no illusion that page views = page reads. They don't. Further, page views do not equal page reads which do not equal calls to action. And isn't that the point? You don't publish stuff for the page views, you publish for the subsequent change in behaviour surely?
As Alexis said, I'd like us to look at measures of employee satisfaction and intranet effectiveness, not just intranet base performance. I'm centering now on measures related to employees: % of employees that visited this month/quarter/year; indications of effectiveness such as click-throughs to links from news articles; #comments (although that's slightly flawed) ...
Posted by: Jonathan | August 27, 2013 at 05:39 PM
Measuring intranet success is indeed an interesting topic. Yes, there is the possibility of gaming the stats, but to what end? If it's to impress the boss, it seems like a risky strategy - what if you get caught? Plus, as I mentioned in the article, for measures to be meaningful, they need to be compared against something. You could find yourself in an endless loop of artificially bumping up the stats to impress, making it riskier and riskier...
I guess I'm assuming an ideal world where the purpose of measuring an intranet's performance is to identify areas that need improvement and determine whether improvements have been effective or not.
I believe the 5 measures listed in the article in combination can help you do this. They also provide tangible and meaningful goals for intranet teams to aim for. More traffic, participants, and time on site are GOOD things to aspire towards (as a proxy for delivering value).
This idea that less views or less participants is somehow a good thing seems like a bit of a cop-out to me for not delivering an intranet that people need.
Posted by: Andrew Wright | August 27, 2013 at 09:09 PM
Interesting post, Andrew. As an (academic) social scientist, my first thought is to consider statistical analysis. Correlations between satisfaction and the basic measurements that you list would be helpful, rather than just using "more" compared to benchmarks. Then again, a CEO or chief communications officer likely isn't going to care much about significance levels and our magic "P" levels.
What we scholars need to do-- and practitioners should also consider-- is develop measure(s) for the degree or degrees to which channel use (intranet) facilitates task completion. My own dissertation research has shown a weak link between intranet use and overall job satisfaction. There's something more in between, and that's how much employees believe intranets can help them. You might see some employees spend less time on intranets (and thus have lower frequencies on some of your measures) and higher levels of task satisfaction if employees believe they only need to marginally use intranets in their work day.
Thanks again for the post since measurement is always a hot-button issue regardless of whether it's in academia or the actual professional world.
Posted by: Jwal | August 27, 2013 at 11:23 PM
Thanks Jwal - your comment about 'developing measure(s) for the degree or degrees to which the intranet can facilitate task completion' is an interesting one.
For a long time I believed the primary purpose of an intranet was all about helping employees complete tasks more effectively and efficiently.
But through my intranet research and interviews with many intranet managers, I have since changed my mind on that. I now believe intranets have an equally important role to play in not only improving operations (tasks) but also in enabling:
* innovations and improvements to occur within the organisation, and
* employee engagement.
These two things that are not easy to measure and yet can add considerable value to an organisation (up to 28% in the case of high employee engagement).
I think the 'time spent on the intranet' measure is perhaps a reflection that when an intranet offers functions such as online collaboration spaces, discussion forums, news feeds, blogging and microblogging - people use these as way of bouncing ideas off people, problem solving, building personal networks and building culture within an organisation. These are important activities but very difficult to measure an intranet's contribution towards them (other than perhaps time spent).
Below is a great case study that illustrates my point by an organisation who rated very highly in the WIC survey for questions about intranet interactivity.
Leveraging social tools to drive culture and adios 15,000 emails
http://nsynergyblog.com/2013/08/26/leveraging-social-tools-to-drive-culture-and-adios-15000-emails/
Posted by: Andrew Wright | August 28, 2013 at 12:06 AM