Digital Workplace and Intranet Employee Benchmarking

Intranets: the signal and the noise and the need for a 'content miner'

By Andrew Wright

In science and engineering, signal-to-noise is a measure that compares the level of a desired signal to the level of background noise. The term is also commonly used to describe a ratio of things that are useful (signal) to less useful (noise).

In intranets, it’s a useful analogy to describe the relationship between work-in-progress (WIP) type content such online collaboration & discussions (noise) and more formal content such as policies, procedures, work instructions, templates and forms (signal).

 

Signal V Noise - which is more valuable?

While online collaboration and the social possibilities offered by modern-day intranets are getting a lot of attention these days, it’s important not to forget the value that well structured and ‘qualified’ (ie. reviewed and approved) content provides to organisations.

In fact, according to the latest research data from the Worldwide Intranet Challenge (WIC)*, this qualified content is more important to the overall value of an intranet than unqualified and informal work-in-progress (WIP) type content.

The following chart shows the correlation of each WIC survey question with the overall value of the intranet. Questions at the top of the chart (eg. Mobile Access, Speed) have the weakest correlation to value while those at the bottom of the chart (eg. Time, Interactivity) have the strongest.

The arrows show the strength of the relationship between the ‘signal’ type content versus the ‘noise’ type content.

So what the chart is showing is that those questions related to collaboration and discussion (the noise), while important, are less critical to the overall value of an intranet than work instructions, policies and procedures, online forms and business applications (the signal).

When you think about it, this makes sense. The purpose of most collaboration or discussion is to either:

• Solve problems, or

• Discuss new ideas or innovations

The output from these discussions and collaboration should be new or updated processes along with supporting content such as templates, lists and/or applications.

Example

For example, let’s say a bank wants to introduce a new way to process personal loan applications. This idea may firstly be initiated and then debated & discussed through various online channels. A project site may then be set up on the intranet to implement the chosen solution. All this collaborative activity taking place is important but should ultimately result in new or updated businesses processes along with supporting content (eg. a new application or form).

The ‘signal’ and the ‘noise’ - the missing link

The following diagram shows the relationship between the different intranet content types.

It is important to note in this diagram the ‘content curation’ aspect. This is where ‘the noise’ type content is translated into the more meaningful, re-usable and findable ‘signal’ type content.

In many organisations, this function is often overlooked. This means that staff must wade through historical online discussions or conversations (the noise) to find that valuable nugget of information they need, instead of being able to find it easily in a well structured list of procedures (or other structured content).

It’s not that different to how email is used at the moment - that is, employees needing to search through past emails to find valuable information instead of it being available in a well indexed list of organisational wide content types.

What does it all mean?

What the above research suggests is that organisations need to:

• Clearly distinguish between the ‘signal’ and the ‘noise’ type content. These content types should be treated differently. ‘Noise’ type content will have less governance and should be branded in a way that staff recognise that the content is not ‘qualified’ - that is, it hasn’t been reviewed or ‘officially’ approved for widespread use within the organisation.
  
  ‘Signal’ type content on the other hand will have a greater level of governance (eg. sign-offs and approvals) and should also be clearly branded so that staff understand that this type of content is the ‘official’ and 'approved for use' within an organisation.

• Recognise that the purpose of ‘signal’ and ‘noise’ content shouldn’t be confused. In other words, collaboration spaces, such as team sites and project sites, should NOT be the place where procedures, policies, forms and reference documents such as case studies are found. This will confuse employees and has the potential for an organisation to end up with hundreds of mini intranet sites - ie. places where staff go to find the ‘official’ way of doing things. There should only be one place for ‘qualified’ content (ie. the intranet) but there can be many collaboration spaces.

• Recognise the need for a ‘content curation’ activity to take place. Just as ‘big data’ has created the need for a Data Miner to analyse, interpret and understand this data, so too does collaborative content require the need for a ‘Content Miner’ to review collaborative content and convert it into the more useful and findable ‘signal’ type content.

* The WIC is a web based survey that allows intranet end users to provide feedback about their own intranet. This feedback is then compared with other participating organisations. Over 45,000 intranet end users from more than 160 organisations have completed the WIC so far.

Related articles

10 lessons intranets can learn from Wikipedia

SharePoint V The Rest of the World

SharePoint V The Rest of the World

by Andrew Wright

There are literally dozens of technologies available that organisations can use to create and maintain their intranets. However Microsoft SharePoint has become the dominant technology with roughly half of all intranets built using SharePoint.

While there are many variables that can affect the value of an intranet such as quality of content, governance, leadership, culture, training, and change management, I thought it would be an interesting exercise to see how much of an impact, if any, technology might have on an intranet’s value.

So I analysed the data from the Worldwide Intranet Challenge (WIC) online benchmark survey to see whether there were any interesting findings - particularly in relation to SharePoint versus Other Intranet Software.

155 organisations have participated in the WIC and the breakdown of software used by participants is as follows,

• 24 SharePoint (version not known)

• 21 SharePoint 2007

• 15 SharePoint 2010

• 75 Other (comprised of 66 other intranet technologies with no clear second place)

• 20 Not known

To analyse the impact of the software on end user satisfaction, I filtered the results by the categories listed above and calculated the average score for each question. I then compared these averages to see whether there were any significant differences.

Note: Of the ‘Other’ intranet software, 4 was the next highest number of occurrences (Lotus Notes in case you were wondering). So the sample size is too small for any meaningful analysis to be done for software other than SharePoint. Grouping all non-SharePoint related software together in one category runs the risk that those products that perform particularly well in an aspect of intranet (eg. creating workflows or forms) may not be recognised because of this grouping together of products.

 

Key findings are below:

 

Overall value

Software doesn’t appear to affect the overall value of the intranet with minimal differences for each software category.

Time on the intranet

Staff spend approximately 5 minutes (around 15%) longer on SharePoint compared to ‘Other’ software (Is this a good thing? See the article If employees spend more time on the intranet, does this mean it's more valuable?)

Finding information

Again, there appears to be little difference in the ease of finding information, regardless of the software. 

Online collaboration

One area where SharePoint seems to have an edge is online collaboration. The average rating (out of 4) for SharePoint was 0.80 compared to Other 0.54 - almost double, a significant difference. The chart below shows a percentage breakdown of responses to this question.

This finding is supported to some extent by the results of the following surveys which indicate that the primary use of SharePoint is for supporting team based collaboration.

What are you doing with SharePoint (survey completed by 239 people)?
http://currents.michaelsampson.net/2010/08/what-sharepoint-overall.html

30 July 2010 (completed by over 750 SharePoint professionals)
http://www.colligo.com/media/document/survey-results.pdf

What does it all mean?

For many of the WIC survey questions, there were only minor differences between SharePoint and non-SharePoint ratings. This seems to indicate that there are many factors at play that affect the success of an intranet and that perhaps technology is not as big a part of the overall solution as many organisations may think.

Other key factors, besides technology, that will affect intranet value include:

• Leadership and support - are senior management convinced of the value of an intranet, do they actively use and support it?

• Financial support - are the funds available to sustain ongoing participation?

• Quality of and frequency of content provided - are people contributing content on a regular basis?

• Governance - are roles and responsibilities clear? Is content being maintained and updated regularly? Are there clear processes in place for keeping the intranet current?

• Organisational culture - is the culture one of collaboration and sharing of information? Or are people reluctant to share information?

• Change management and training - are people aware of what intranet functionality is available? Is training provided? Is it easy for people to contribute content?

You may also be interested in:

• If employees spend more time on the intranet, does this mean it's more valuable?

• Improve your intranet by 10%: how benchmarking can help

• Why measuring page views, time spent and user activity is a worthy measure of intranet success

• 10 lessons intranets can learn from Wikipedia

10 lessons intranets can learn from Wikipedia

         

by Andrew Wright

Convincing employees to use the intranet is one of the biggest challenges intranet managers face. One of the main reasons is the difficulty in persuading employees to contribute good content that other employees will find useful.

Just how difficult this task can be is highlighted by feedback from the Worldwide Intranet Challenge (WIC) online benchmark survey (the survey has been completed by over 45,000 intranet end users from more than 160 organisations).

The chart below shows the breakdown of responses to the 4 questions that ask employees how often they contribute content to the intranet.

As you can see, on average more than 90% of staff are not frequent contributors (content contribution methods include discussion forums, commenting, team sites, messaging, blogging, micro-blogging, as well as publishing traditional intranet content).

That’s a huge amount of handy hints, tips, information, business intelligence, insights, best practices, ideas, innovations, solutions, answers, knowledge and good advice that is not being properly shared or managed by organisations.

This lack of involvement represents a virtual gold mine of untapped knowledge lying dormant within many organisations.

How can organisations mine this gold?

Wikipedia began in 2001 and has since grown to become the 6th most popular internet site in the world (according to the Alexa website rankings). The Wikipedia statistics for June show that there were 11,071,642 pageviews per hour for the English Wikipedia. In anyone’s language, Wikipedia is a runaway success. And all the content is provided by volunteers!

How did this happen? Why do people contribute content? And can these same lessons be applied within an organisation?

The article, Why did Wikipedia succeed while other encyclopedias failed, explains that there were seven collaborative encyclopedias that aspired to Wikipedia-like dimensions before Wikipedia itself came along.

So why did Wikipedia succeed?

Focus on content - not the technology

Wikipedia focused on content development instead of technology. Its competitors, on the other hand, saw themselves more as technologists than as content providers. Jimmy Wales and Larry Sanger, the two Wikipedia founders, played the role of content evangelists, seeding Wikipedia with content they solicited from contributors — which, in turn, led to more content, and more contributors.

Lower barriers to participation

Wikipedia also lowered the barriers to participation. Editing Wikipedia is easy, instant, and virtually commitment-free. It’s possible to come along and do an anonymous edit and never make a contribution again. Over time, a sentence here and a paragraph there, turned into articles, which then turned into an encyclopedia. Without this relatively low barrier to entry, the community would be difficult to sustain.

In addition, there are no professional editors that review material before it is published. Instead, it’s the people who visit the pages who are the reviewers.

Why do people contribute content?

The article Online Motivational Factors: Incentives for Participation and Contribution in Wikipedia also examines why people are motivated to contribute to Wikipedia. There are many research based references and it’s well worth reading. Some of the key findings include:

• Users will contribute more to a community if they:

  • Believe that their contributions are important to the group’s performance,

  • Believe that their contributions will be identifiable, and

  • Like the group they are working with.

• Three conditions need to exist for this kind of peer work to perform well:

  • Cost for the contributors is low - needs to be easy to do

  • Tasks are ‘chunked into bite-size pieces’ - minor editing is available and boosts participation because the threshold for initial participation is low

  • Low cost of quality control

The article also notes that research has shown that a high level of both two-way and large scale community interactivity is a strong inducement to participate. There is a strong correlation between interactivity and motivation in online communities.

10 lessons that can be applied to intranets

Become ‘content evangelists’ - obtain ‘seed’ content from contributors Provide a platform that is quick and easy to use - lower barriers to contribution Make it possible for people to do edits on the run - chunking of tasks Consider the content review process - is it unwieldy and a barrier to contribution? Don’t get caught up in the technology - focus on helping users understand what you want from them rather than on dazzling them Provide tools to enable contributors to interact with each other Organise content community events and gatherings Explain how content contribution is important to the organisation’s future Use gamification with incentives to reward certain user behaviors Make sure your platform ‘invites’ rather than ‘challenges’

You may also be interested in:

• Improve your intranet by 10%: how benchmarking can help
• Why measuring page views, time spent and user activity is a worthy measure of intranet success

Interested in learning more about the WIC research?

I am giving three presentations in September where I'll be talking about some of the more interesting findings from the WIC data. I'll also be showing case studies of intranets that have succeeded in satisfying their most important stakeholder, the end user.

• 23-24 September - Utrecht - Congres SharePoint
• 25 September - London - Interaction 2013
• 26 September - Köln - IOM Summit

 

 

Improve your intranet by 10%: how benchmarking can help (latest research)

by Andrew Wright

“You can’t improve what you don’t measure” goes the old saying by I’m not sure who. This is certainly true in the case of the Worldwide Intranet Challenge (WIC) free online intranet benchmarking service.

Eleven organisations from the 150 organisations who have participated in the WIC have now completed the WIC survey twice. So what are the results? Have these organisations been able to improve their intranets the second time around?

It seems they have.

Every question in the online survey, on average, received a higher score for the second benchmark. The chart below shows the average score for each survey question - sorted in order of % improvement - for both the first benchmark and the second benchmark. Some interesting things to note:

• The average response for each question has improved by about 10% the second time around. Meaning that it would be reasonable to say that these intranets overall have improved on average by 10%.
• The average ranking for each question the first time the organisation completed the survey was 87.1 (from 170 surveys). The second time around this ranking improved dramatically to more than 20 places to 65.9 (from 170 surveys)
• The average time between the first and second survey is 539 days (nearly 1.5 years)
• The chart below:
  • Ranks the questions in order from biggest improvement (Collaboration 30.2%) to least improvement (Speed 1.7%)
  • Shows the average score for each question for both the first and second surveys
• Interesting to note that the question "Collaborate with other staff (eg. using online work spaces or team rooms)" has improved the most (30.2%). This perhaps indicates that the 'social intranet' is a priority for improving intranets 
• Also interesting to note that Mobile access has improved by almost 20% - indicating another priority area of improvement (though off-site access improved only 11.1%)
• So does it pay to benchmark? It would seem so. One of the big benefits that organisations mention when they participate in the WIC is that it helps focus the intranet team on measurable objectives. The fact that there are goals that can be measured is enough to focus and inspire teams to work towards these goals
• Interested in participating? Register for the Worldwide Intranet Challenge.

 

Why measuring page views, time spent and user activity is a worthy measure of intranet success

by Andrew Wright

This article argues that there are five core measures that organisations should be tracking (and benchmarking if possible) over time to determine the effectiveness of their intranets:

• Average time spent on the intranet per employee

• # of page views per employee

• % and frequency of employees that view content

• % and frequency of employees that contribute or update content

• Level of employee satisfaction with the intranet

Combined together and in conjunction with some kind of comparative data, these 5 measures can provide a valuable snapshot of intranet success.

However, not everyone necessarily agrees with this approach. There is a view that traffic, page views and the like are too simplistic and superficial as ‘real’ measures and don’t give an accurate or deep understanding of the underlying business value of an intranet. 

Approaches to intranet measurement

For example, Nigel Williams in his article Judging your intranet by hits alone is really a miss, says that “if you are judging your intranet by hits, stop!” He then provides a list of more than 20 other variables that intranet managers should also consider tracking (check out this interesting LinkedIn discussion about the article).

Gerry McGovern says in his article How to measure the success of your intranet”, “intranet teams are measuring the wrong things, like traffic, hits and page views. Is more traffic a good thing? Why?”. He goes on to say “Focus on your employees’ time. Be relentless in seeking to save it. If you do you will create a great intranet.” (The logical conclusion being that measuring time-saved is a better measure of intranet success than traffic).

In It’s not about the outputs, it’s about the outcomes, Jonathan Phillips says “Outputs such as page views, or time per visit, are interesting numbers but don’t get to the heart of the matter. It’s the outcomes, or the results of intranet activity, that’s most important.” He then says, “These numbers are much harder to generate and report, but do provide better clarity of return on investment and of purpose of your intranet.“

And Anna Rydne says in Why Miserable Intranet Stats May Indicate Success “effective communication means keeping the employees informed in the shortest period of time. And that’s why high click rate and a lot of time spent on the intranet doesn’t necessarily mean success.”

Is it worth looking beyond the raw data?

So it seems to really measure true intranet success, you need to look beyond just the raw numbers. You also need to look at business performance and outcomes.

I don’t necessarily disagree with the general intent of these articles but it does seem like a lot of hard work (as Jonathan points out). And is it overcomplicating something to the extent that it discourages organisations from even attempting to measure intranet value? Why bother measuring page views, usage and traffic at all if this data can be interpreted in different ways.  

In fact, many organisations don’t do any intranet measurement so asking them to commit time and effort in applying sophisticated techniques that measure business value of an intranet is a big expectation. Wouldn’t it be better to use some fundamental measures where the data is relatively easy to obtain, than nothing at all?

But does increased intranet activity necessarily = success?

From my research into effective intranets I have found that a good case can be made that tracking the 5 measures above - which is a relatively simple and straightforward task -  provides a reliable indicator that the intranet is also delivering business value. More reliable than perhaps the articles above acknowledge.

In the article, If employees spend more time on the intranet, does this mean it's more valuable? I reveal some interesting statistics that show that the more time staff spend on the intranet - both as viewers and contributors - the greater it's value.

While the statistics can't show whether there is any correlation between intranet value and business outcomes (the ideal measure of success) it's fair to say that an intranet that is perceived as being valuable by employees will most likely result in positive business outcomes.

It's similar to determining the success of a book. Lets say there are two books about SharePoint 2013 - one sells 10,000 copies and the other sells 100 copies. Which one is judged to be more successful? Which one has provided better business outcomes to the SharePoint community? You could possibly argue that the 100 copy book could potentially add more value. Maybe a single company that followed the advice in the second book was able save more money than all the companies combined that read the first book (almost impossible to measure). But I would say that, on balance, the chances are that the more popular book would also provide the most value (I know which one I would prefer to author).

Or compare two coffee shops - one is always busy the other has only a few people. Again, it could be argued that the one with the fewer people is making a bigger profit and is more successful. Perhaps they charge $50 for a cappucino. Again, I would argue this is unlikely and that the chances are that the busy shop is more successful.

Wright’s Rule for Intranet Success  

Based on this research, I’ve come up with a highly scientific formula that is a guide to intranet success. I’ve imaginatively called it: Wright’s Rule for Intranet Success.

The key word in this formula is ‘more’. What is more? More in comparison to what?  For this formula to work, the measures need to be compared to something. A baseline or benchmark. This could be past measures within your organisation or measures against competitors or against industry standards. As Nigel rightly points out in his article, statements such as  “I have a great intranet because we have 100,000 hits a month” are meaningless.

What is wrong is not so much the fact that it’s measuring hits but that the hits are not being compared to anything. A better measure might be something like number of page views per employee per month for the last 12 months. An even better measure would be average page views per month per employee against an industry benchmark. The ultimate measure would be to correlate these intranet measures with key organisational performance indicators such as profit/loss, revenue, market share, growth, research & development, and employee engagement (register for the Worldwide Intranet Challenge if you would like to benchmark the level of end user satisfaction with your intranet).

There may be certain situations where a case can be made that increased traffic or an increase in the number of users is a negative thing or a decrease is positive (read Anna’s article), but I would argue that these cases are not common and Wright’s Rule is a fairly solid guide.

Time spent as indicator of success

Time spent as an indicator of success is not a new idea. I was interested to see the following Tweet recently from Rupert Murdoch about one of the indicators the signalled the decline of MySpace.


Could the same 'hours spent' indicator be used for intranets?

And then there is this interesting interview with Jeff Weiner, the CEO of LinkedIn:

Interviewer: Are you succeeding in getting people to stay on LinkedIn longer during the course of the day, and is that a goal?

Jeff Weiner: Time spent has never been the primary objective. As we like to say, LinkedIn is not a service that enables you to pass the time, it's a service that enables you to save time. And that goes back to our mission, which is to connect the world's professionals to make them more productive and successful.

So, ultimately would we like our membership to become more engaged [ie. spend more time, click more things], yes, as a proxy for the value that we're delivering to them on a daily basis, and that's why we're introducing products like LinkedIn Today, that's why we have over a million groups now servicing LinkedIn members within a professional context.

What does it all mean?

While it’s a noble and worthwhile goal to look for more meaningful ways to measure and improve the effectiveness of your intranet, it’s important not to overlook or undersell the above 5 core measures as important, valid and legitimate. This is even more important for organisations that don’t measure anything.

So the big challenge for intranet professionals is to provide content and tools that employees need to be more productive and do their work more effectively. As Weiner notes, ‘time spent’ should not be a goal in itself but as “a proxy for the value that we're delivering to them on a daily basis”.

You may also like

• If employees spend more time on the intranet, does this mean it's more valuable?

• Are intranets becoming more collaborative and accessible?

Would you like to compare the level of staff satisfaction with your intranet?

If you are interested in benchmarking your intranet against 150 other organisations, why not register for participation in the Worldwide Intranet Challenge (WIC) free online benchmarking service. Read what participating organisations say about the WIC or find out more on the WIC FAQ page.