Too often the real value of a distribution channel, or site, goes unknown. Information is published with the expectation that the reader will find it. Unlike In a Field of Dreams, “Build it and they will come” is not necessarily accurate. People will visit a site once or twice but if they can’t get what they need don’t expect a third or fourth visit. They will find alternate channels to an answer, like developing a new site or checking with others whose information might be out of date.
But keeping people coming back to your site can be easy if you monitor a metric I call Search Efficiency. (Please note I’m not talking about optimizing a search engine but a way to put a numeric value to the performance of the search engine on your site.) Unfortunately calculating Search Efficiency can be a tedious manual process, but once the effort is completed you have a whole new understanding of your site and its visitors.
Search Engine Analytics
The results of search engines are great to understand why people are looking at your site or its content. It really allows you to normalize what you think you’re about versus what others thing you’re about. Looking at “BigMen” our search results show the follow search terms have been driving the most traffic to our site over the last 30 days:
- Sharepoint Competition
- EMC Magellan
- Sharepoint Oracle
- Sharepoint CMS
- Googlesites
Now while the search analytics were great I did need to massage the data. Specifically this result page did not consider “sharepoint oracle” the same as “oracle sharepoint”. Some search analytic tools may do so but read through the results to be sure.
Part of the series Calculating the Return on Investment with ECM.
What does this show? Our concept of “BigMen” is a general ECM site not specific to one vendor. We are obviously meeting that goal. But it seems that a lot of visitors are really looking to understand more about Sharepoint as a CMS and its competition. So we may want to write an article on Sharepoint and how it meets the competition.
For an internal site you are really checking two things. Is what people are looking for on your site what your site is about? If your top 10 searches don’t line up with information you planned to publish then your sites messaging is wrong. The other thing to look for is to ensure that the information they want is there. The most popular search requests should also be first articles that exist and then the articles with easiest access on the site. In fact the best thing to do may be to make the top ten search results, real result available on the “home” page.
Search Results
The next question you really want to understand is how easily do people get to the information you have on your site using search. Even in cases where you have the search engine in house, only a few of these analytics engines show you how the search terms relate to results, so this can be a tedious manual process.
The results is important to understand as it indicates efficiency. For a self help site you really want people to find the article that been written to answer their search term at the top of the list. If that answer doesn’t appear at the top then odds of them reading the article drop the further it is down the list, especially below the scroll.
Continuing the example using “BigMen”, since our analytic engine does show results we need to enter and calculate them manually. Searching “sharepoint competition” our article is the fifth search engine entry and searching for “emc magellan” our article is the second search engine entry. For an internal site search the results are rather poor but as this is an external site search, the articles have good placement and therefore good visibility.
Now if you didn’t have search engine analytics to begin with you could still get a handle on site search efficiency. All you need to do is to perform searches on key concept that is an accepted reason for people to visit the site. Then run each search and see where the published article appears in the results.
Calculating Search Efficiency
Probably the best way to turn this into a number that can be monitored is by tracking where the article returns on the results and averaging them. I call this number Search Efficiency. Basically you end up searching on each valid term and then entering the results in a spreadsheet. If it’s the first item in the results I put a “1” in the column. If it’s the fifth result, I put a “5” in the column and so on. A Search Efficiency close to 1 is ideal but a good ranking for an internal site will be between 2.5 and 3.5.
But I also take this a step further. I also make adjustments for other inefficiencies. For example, if to get to the result requires the user to scroll down then I add one more point to the rating for each scroll. I also look at error results, like similar articles and old articles before the actual article, and add points for each of them. For example if the actual article appears in the 8th position and it required the user to scroll down and there was an out of date article in the 5th position, I would give the article a 10 (8+1+1).
So following my example, here’s what I get:
- Sharepoint Competition – 5
- EMC Magellan – 2
- Sharepoint Oracle – 8 (7+1)
- Sharepoint CMS – 11 (10+1)
- Googlesites – does not appear in results
When I average the results (5,2,8,11) I get a Search Efficiency of 4.25. Again for an internal site search these results are poor, specifically the results around “sharepoint oracle” and “sharepoint cms”. Both of these would need to be much lower numbers. But as a measure of an external sites visibility, the results are not bad. Of course to achieve a more accurate number you would need to review more search terms.
Content Quality
An item that goes hand in hand with Search Efficiency is Content Quality. Content Quality is not a calculated value but a surveyed one. Just because a person finds the content that you intended them to find with their search request does not mean that content met their needs. The content could be confusing, inaccurate, or out of date.
But in order to survey Content Quality a feedback mechanism is required for the site. This can be as simple as a voting box for “was this content helpful” or multiple ranks on quality and even include comments for in depth reasons as to why it was not helpful. What I would not do is use a simple comment box as the only feedback mechanism. It is difficult for authors to read all the comments and by using a simple true/ false indicator for quality allow the author to limit their review to only those that readers feel need revision.
I like to use a five point scale with “1” being “very helpful”, “3” being “neutral”, and “5” being “not helpful at all”. The system should then keep track of the results, preferably at in individual level. All content should be at least a 3 with the top content ranking having an average of 1.5 to 2.5. Content ranked at 4 or 5 should be reviewed regularly.