Comments on: Enterprise Search Best Bets – a good enough practice? http://blog.leeromero.org/2009/02/03/enterprise-search-best-bets/ On Content, Collaboration and Findability Tue, 23 Feb 2010 18:35:33 -0600 http://wordpress.org/?v=2.8.6 hourly 1 By: Lee Romero » Blog Archive » Best Bet Governance http://blog.leeromero.org/2009/02/03/enterprise-search-best-bets/comment-page-1/#comment-19817 Lee Romero » Blog Archive » Best Bet Governance Tue, 23 Feb 2010 02:58:15 +0000 http://blog.leeromero.org/?p=339#comment-19817 [...] previously written about best bets and how I thought, while not perfect, they were an important part of a search solution.  In that [...] [...] previously written about best bets and how I thought, while not perfect, they were an important part of a search solution.  In that [...]

]]> By: Lee Romero http://blog.leeromero.org/2009/02/03/enterprise-search-best-bets/comment-page-1/#comment-2773 Lee Romero Thu, 05 Feb 2009 19:17:18 +0000 http://blog.leeromero.org/?p=339#comment-2773 Thanks for the kind words, Kas! As far as ROI - I can imagine how to measure (at a gross level) the investment side of things. What I would be challenged to measure would be the "return" side of the equation. My first thought is to measure what percentage of click-throughs in search results are from best bets? That gives you a general idea of their utility, anyway. That isn't sufficient because it does not measure how useful those results were (did the target meet the needs of the searcher?) It also does not provide any insight on the impact of letting those same items simply show up in the organic results as they normally would. If you make the leap that the items clicked on were valuable at all (that they were the "right answer" to the query), I could imagine disabling the "best bets" display for a time and seeing how affects which targets a user selects - if they still select the same items, managing best bets is not having much of an impact. Any other thoughts on measuring the utility of best bets directly? Thanks for the kind words, Kas!

As far as ROI – I can imagine how to measure (at a gross level) the investment side of things.

What I would be challenged to measure would be the “return” side of the equation. My first thought is to measure what percentage of click-throughs in search results are from best bets? That gives you a general idea of their utility, anyway.

That isn’t sufficient because it does not measure how useful those results were (did the target meet the needs of the searcher?)

It also does not provide any insight on the impact of letting those same items simply show up in the organic results as they normally would. If you make the leap that the items clicked on were valuable at all (that they were the “right answer” to the query), I could imagine disabling the “best bets” display for a time and seeing how affects which targets a user selects – if they still select the same items, managing best bets is not having much of an impact.

Any other thoughts on measuring the utility of best bets directly?

]]> By: Kas Thomas http://blog.leeromero.org/2009/02/03/enterprise-search-best-bets/comment-page-1/#comment-2769 Kas Thomas Thu, 05 Feb 2009 16:18:07 +0000 http://blog.leeromero.org/?p=339#comment-2769 Excellent post, Lee, as usual. I'm glad to see debate on this. Thanks for mentioning me. What I'm wondering at this point is how one would measure the effectiveness of best bets, and what an ROI calculation might show (taking into account the cost of manual intervention, of course). More blog-fodder, perhaps? Excellent post, Lee, as usual. I’m glad to see debate on this. Thanks for mentioning me.

What I’m wondering at this point is how one would measure the effectiveness of best bets, and what an ROI calculation might show (taking into account the cost of manual intervention, of course). More blog-fodder, perhaps?

]]> By: Jim S http://blog.leeromero.org/2009/02/03/enterprise-search-best-bets/comment-page-1/#comment-2728 Jim S Tue, 03 Feb 2009 18:21:53 +0000 http://blog.leeromero.org/?p=339#comment-2728 As stated above we use our WCM as the owner of the URL. We made decision to use the popular search terms in our search analytics to identify our key words. The goals if a terms is popular then we want to make sure that the results would be highly relevant to the majority of the audiance. In otherwords the users determine our keyword library. After reading your approach I'm beginnig to believe a hi-bred combination of both approaches might produce an interesting and highly relevant initial list. After 5 years working with this system I find that Monthly maintenance is simply running the search log of popular searches and identifying any new popular search terms. Takes less than 5 minutes. 3 or 4 times a year a new term will appear and we identify the appropriate key results. Usually less than 30 minutes. I'm guessing total maintenance invested is less then 3 or 4 hour annually. As stated above we use our WCM as the owner of the URL. We made decision to use the popular search terms in our search analytics to identify our key words. The goals if a terms is popular then we want to make sure that the results would be highly relevant to the majority of the audiance. In otherwords the users determine our keyword library.
After reading your approach I’m beginnig to believe a hi-bred combination of both approaches might produce an interesting and highly relevant initial list.

After 5 years working with this system I find that Monthly maintenance is simply running the search log of popular searches and identifying any new popular search terms. Takes less than 5 minutes. 3 or 4 times a year a new term will appear and we identify the appropriate key results. Usually less than 30 minutes. I’m guessing total maintenance invested is less then 3 or 4 hour annually.

]]>