The value of click-stream data derived from toolbars

Reading Greg Lindens blog is always pleasent if you want to dive deeper in the knowledge of search engines. In his last post he refers to an article by Jan O. Pedersen et al at Yahoo! called “Making sense of search result pages” [PDF] (published in a academic digital archive, not a peer-reviewed journal).

“Search engine query logs only reflect a small slice of user behavior – actions taken on the search results page. A more complete picture would include the entire click stream; search result page clicks as well as offsite follow-on actions”.

Yes, I’ve always been critical to the importance of query log data and click-stream data is of course a good expansion to quantitative studies on search beahvior. But, I think the heavy dependency on this kind of quantitative data is just an expression of the laziness of search engines to not use more qualitative research methods. For example naturlistic inquiry. Yes, it’s not possible to get data from as many users as in query log analysis, but I think the mix is important. Actually, I’m not quite sure how much qualitative reasearch search engines do today but among usability engineers it’s a matter of course.

But Pedersen also comment eye tracking:

“Ultimately a direct measurement of what the user perceives on a search results page would be most useful. A closely proxy of this ideal is measuring eye movement and fixations across the page followed by user interviews. Aggregate measures reveal which page elements get the greatest attention, while tracking individual sessions is very revealing about scanning behavior”.

User interviews! Good qualitative research, but not always easy. But remember what Jared Spool said about eye tracking in: “Eyetracking: worth the expense?“.

More of mixing quantitative data with qualitative in web search behavior research. This scientific field is overwhelmed by query log analysis papers.

Interview with Marissa Mayer, Google VP, Search Products & User Experience

Search Engine Land has started a weekly column on search and user behavior called Just Behave. Nice initiative as search behavior is maybe one of my main interests in my work. Last friday Marissa Mayer:”…the driving force behind Google’s Spartan look and feel from the very earliest days”, was interviewed by Gord Hotchkiss. Gord is CEO of Enquiro, a search marketing firm that produces search engine user eye tracking studies and other research.

Gord does some reviewing on Enquiros latest Eye-tracking study which compares Google, Yahoo and MSN. I’m not doing comments on it here, because Chris Sherman will soon on Search Engine Land. But I want to put some critical views on the importance of Eye-tracking studies by pointing to this article by Jared Spool: “Eyetracking: Worth The Expense?

Many interesting things is discussed in this interview. I will extract some of Marissas answers. Here’s about designing for the right resolution:

Yes, we are still seeing as many as 30% plus of our users at 800 by 600. My view is, we can view 1024 by 768 as ideal. The design has to look good on that resolution. It has to at least work and appear professional on 800 by 600. So all of us with our laptops, we’re working with 1024 by 768 as our resolution, so we try to make sure the designs look really good on that. It’s obvious that some of our engineers have bigger monitors and bigger resolutions than that, but we always are very conscious of 800 by 600.

Very interested thing is how search focused user avoid information:

Most users on our search results page don’t see the logo on the top of the page, they don’t see OneBox, they don’t even see spelling corrections, even though it’s there in bright red letters. There’s a single-mindedness of I’m going to put in my search, not let anything on the home page get in the way, and I’m going to go for the first blue left aligned link on the results page and everything above it basically gets ignored.

About the future of search she says:

I think the most pressing, immediate need as far as the search interface is to break paradigm of the expectation of “You give us a keyword, and we give you 10 URL’s”.

And further:

So, challenge one is how the searches get expressed, I think we really need to branch out there, but I also think we need to look at results pages that aren’t just 10 standard URLS that are laid out in a very linear format.

And comments on ads as relevant and not as disturbing:

I do think that for the 40 or so percent of page views that we serve ads on that those ads are incredibly relevant and usually do beat the search results, but for the other 60% of the time the search results are really the only reasonable answer.