Researchers Track Mouse Movements and Hesitations 116
lpctstr writes "Researchers from the University of Washington and Microsoft Research have found that cursor movements and cursor hovers can detect the relevance of a search result and whether a user may abandon the search. They use an efficient algorithm written in Javascript to silently record movements and clicks on Bing and find that computing relevance using movements + clicks works better than just clicks (the current state-of-the-art). They explain some of this due to cursor and gaze being closely aligned on the web, and especially so on search result pages. Is this the future of innovation in search ranking — Google and Bing tracking your every twitch and pause?"
Maybe it can tell me which I like better (Score:1)
Re:Maybe it can tell me which I like better (Score:5, Funny)
People like me (Score:2)
How would this work for people like me who don't move their mouse unless there's something they've decided to click on?
My mouse is not an extension of my eyes. It's a tool I use if and only if there's a task to be accomplished with it.
Re: (Score:1)
Likely their algorithms would be able to determine that you're the kind of user that only moves the mouse when you know where it's going, and disregard any data based on your mouse movements:
Re: (Score:1)
Yeah, instead they'll activate your webcam and track your eyeballs.
Scary (Score:1)
This would be scary.. if it wasn't potentially true.
How long, exactly, until major websites "require" for a flash / silverlight bug to play.. the sole purpose of which is to enable the camera to track your eyeballs. Resolution? Not that great... but matched with mouse movements and matched up against clicks.. it could well be data worth having.
Of course, we all use flashblock [mozilla.org], so it won't be a problem for us. Right?
Re:People like me (Score:5, Interesting)
Re:People like me (Score:5, Insightful)
Well, AFAIK, you're in the minority. I think most people do in fact move their mouse with their gaze, because it cuts the delay between when they decide to click on something and when they actually click on it. Think of it as a pre-loading or caching technique -- you don't pay much cost for moving your mouse around a little bit, but you can save time. At least that's my hypothesis.
I dunno...
I mean, obviously, they've got some kind of research to back it up... But it seems like this would be pretty useless to me. I mean, do people actually follow what they're reading with the mouse cursor?
I generally plant my mouse cursor in a chunk of whitespace so that it's out of the way while I'm reading. When I find something to click on, I go straight to that link and click. If there's multiple things I want to click on I'll generally hit them with the middle button to open multiple tabs. I don't generally pause my cursor over anything in particular.
And then there are the people who just don't stop moving their mouse cursor... The thing spins and swirls around the screen, slowly circling towards a link or a button that they want to click. I wonder what kind of data they could mine from that sort of behavior?
I will occasionally highlight a few random characters to act as a bookmark if I have to go do something else. What would this be interpreted as?
Re: (Score:2)
I'm blacklisting javascript on bing (using yesscript)
Dont do that, use a greasemonkey script to pick the most porny link and send hover data back to the server. Or just 'SEO' optimise your own sites with funny bits of javascript that talk back to bing when a user hits your site.
Re: (Score:2)
I mean, obviously, they've got some kind of research to back it up... But it seems like this would be pretty useless to me. I mean, do people actually follow what they're reading with the mouse cursor?
I know plenty of people who read using their fingers as a pointing device. I don't know, just because you (or I) don't need to do something like that doesn't mean the vast majority of the average user does.
I do, however, sometimes pick the mouse up a little and use the nervous tension in my arm to jiggle it up
Re: (Score:2)
This is pretty much the way I browse too. However, on some sites it gets tough to find a suitable blank area that doesn't pop up some stupid hovering elem
Re: (Score:3)
This whole article has had the "you are now breathing manually" effect on me. Now I'm watching my mouse cursor intently, trying to remember what the hell I usually do with it on instinct.
But yeah, I'm probably a lot like you; I just leave it in some whitespace where it faffs about until I need to click on something. I often see it in the periphery more than my direct focus because I have good vision and good reaction times.
Re: (Score:2)
I mean, obviously, they've got some kind of research to back it up... But it seems like this would be pretty useless to me.
This is modded "insightful", which it is. But the insight is entirely into the contents of your own head. If you bother to actually look at the data you might have something to say about the world outside yourself, which might get modded not just "insightful", but "interesting".
Why do people in the 21st century continue to repeat the failed behaviour of the past and report what "it seems like" to them as if it was remotely interesting or relevant to any given problem? We can look back at the past several
Re: (Score:2)
I think most people do in fact move their mouse with their gaze
Really? I'd say only the twitch-surfers who visit sites with little coherent text content would do so (e.g. Facebook twatterers). Well maybe the technologically inept, too. You know, those who take ages to click that menu item you've been telling them to click (and even pointing to with your finger) for the last 5 minutes.
Exactly. Most people.
Re: (Score:2)
I'm with you on the technologically inept. People that would never change their browser and search engine from the default.
Re: (Score:3)
It depends also on the mouse type, for example, if the mouse has a tracking wheel. I use mouse to scroll down the results in Google search. It would be cumbersome to simultaneously mouse over the links.
Re: (Score:3)
I do the same. My eyes can track the page and backtrack in a fraction of a second if there's a mistake, or something I need to double-check. The cursor is a high-cost movement, in that I would have to move my hand back.
I just experimented - my mouse cursor tends to be idle, then move directly for the only button I click, whether I'm in an app, on a website or just opening programs. Anything else is a bit of a waste of movement. "Fake" links and those of no relevance see no more contextual information fr
Re: (Score:2)
Re: (Score:3)
So you hover the link to make a decision about whether to click on it or not. Wouldn't that show up as a click, or not a click? And wouldn't suspicious websites end up as "potential click" candidates just as often as those that just don't grab your attention, or ones that have a funny URL?
The point is that the hover doesn't help you *differentiate* the user's intentions. It just lets you see where they happened to stick the mouse. The decision process concerning whether that site is "better" for the use
Re: (Score:2)
Really? You read the description of the page, thought it looked interesting, and then hovered over the link to see the URL. It sounds like I can tell a lot about that link from the fact that you hovered over it. You found it to be a potentially useful link, but then bailed on the URL. For other links, you dismissed them out of hand. If Google could figure out what in the page description interested you, and surfaced more pages similar to that page, they'
Re: (Score:2)
I generally try to rest the mouse cursor in some chunk of whitespace, so it's not in my way while I'm reading.
Re: (Score:2)
Re: (Score:1)
Google works for quickly selecting results using the arrow keys, so I find myself touching the mouse less and less. Oh wait, this was for Bing. Nevermind.
Re: (Score:2)
It obviously doesn't. Just like eye tracking doesn't work for people who are blind.
And it doesn't matter since something doesn't have to work with everyone to be useful.
Keyboard User (Score:2)
Exactly. I typically stick with the keyboard: either page up/down, or cursor up/down. I sometimes use the mouse to adjust the scrollbar, but seldom move within the window until I'm ready to click a link.
Re: (Score:3)
It doesn't need to work for them, they don't use Bing.
Re: (Score:2)
Something like this only has to be valid fora certain percentage of users in order for it to be useful. For you (and your stationary mouse) and me (and my constant text highlighting twitchiness, often highlighting text at random on a page while I read elsewhere), obvi
Oh well. (Score:1)
Re:Oh well. (Score:4, Interesting)
If you use Firefox there is an add-on called Scroogle [mozilla.org] that sidesteps these cursor-movement worries, plus leaving no tracks for Google to assimilate. It will add itself to the list of available search engines. I use it almost exclusively nowadays. Of course one must trust Pathetic Cockroach, the author, but the 5-star reviews speak loudly to me. I've never heard any criticism of it and would be interested if there is...
Re: (Score:3)
But then you have to trust Scroogle.
I prefer the approach of Searchfuscate.com [slashdot.org] - which continually does random searches that Google can not distinguish from your human powered searches - and you can code review the javascript yourself to prove the automated searches are indistinguishable from the human ones.
If you are searching for suspicious content, you'd probably never show up in the radar thanks to the zillions of innocent searches "you" also perform; and if you're asked,
Re: (Score:2)
Now we need a browser extension to help this effort track all the random data it likes.
"Future" is a mouse? (Score:3, Insightful)
They explain some of this due to [mouse] cursor and gaze being closely aligned on the web, and especially so on search result pages. Is this the future of innovation in search ranking — Google and Bing tracking your every twitch and pause?"
...Just in time for Web use to go mobile and touch-based.
Re: (Score:2)
Because everyone's going to migrate to smartphones? Unlikely. Touch screens are nice for environments where a mouse isn't practical but not my favorite for full on browsing. The two are entirely different animals, IMO.
I think the mobile browsing search can be/is covered by other methodologies that are probably equally effective - for example many people stay logged into things such as facebook and whatnot while willfully giving out piles and piles of their own personal information/preferences (look at fours
Consumer device (Score:3)
Yup, device equipped with touch screens are rather different beasts than desktops.
And browsing happens to be among the top activities for which tablets are designed. Keyboard-less, light devices are better suited to consume content, big machines with lots of inputs are better suited to create content.
Lots of browsing will occur on the current and future generation of tablets, and that's what is going to get mined through newer techniques (webcam secretely analysing gaze direction ?!?)
Meanwhile we will conti
Re: (Score:2)
"Keyboard-less, light devices are better suited to consume content, big machines with lots of inputs are better suited to create content."
You know - I never looked at it that way before, but I really like statement. Good perspective. :)
Re: (Score:2)
And touching a screen on a typical desktop/laptop would get tiring after a few minutes, even Jobs knows that when all the rumors of a touch-screen Mac were flying around. At best, a touchscreen will augment the existing mouse, but won't replace it on anything other than tablet PCs.
Now... (Score:1)
Re:Now... (Score:5, Funny)
Microsoft is going to patent hesitation
Google would have got in first but didn't want to rush it.
Re: (Score:2)
Microsoft is going to patent hesitation
Google would have got in first but didn't want to rush it.
Maybe they should patent procrastination.
This is news? (Score:2)
This is news?
Even my university notes make reference to using mouse gestures or gaze tracking to determine which search results are relevant, whether the user is hesitating or uninterested in the paragraph you're offering him.
How is this news?
Re: (Score:2)
Maybe it's news because people are doing it (in Javascript no less) not just talking about it?
In fifty years if somebody creates a FTL drive are you just going to sit around yawning, saying "Big deal, Alcubierre thought that up a long time ago?"
Google (Score:4, Interesting)
Re: (Score:2)
I've only run into that when I click the ad links that show up at the top of my results sometimes. It passes through doubleclick I believe. Is that what you're talking about?
Re: (Score:2)
Re: (Score:2)
You mean how each Google search results link sends a response to Google when clicked before redirecting you to the actual result?
Isn't that kind of vital for them in order to rank and prioritize search results? Wouldn't their service suck if everyone blocked that?
Re: (Score:2)
Hopefully you didn't think your google clicks were anonymous before you discovered it recently ;) You can take a look at google web history [google.com] to see just how much they've tracked. In theory you can turn it off, b
Yeah, I read with my cursor, too. (Score:2)
only records what appears interesting (Score:2)
the method will make spamming and gaming the system easier. well, not easier, but opens up a new route.
and of course, throw this out of the window on touchscreens.
So using vimperator (Score:2)
scroll wheel (Score:1)
Indeed
So I walk away... (Score:2)
So I walk away to, oh, I don't know take a piss or something, and when I come back wherever I bumped my mouse getting up is the most relevant thing in my search? Riiight.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
I read the PDF. They do not directly control for an unattended computer during the testing. Their numbers are statistically significant when averaged across their pool of users, though. It won't help with personalizing the results, but it seems it will help with overall ranking of results for particular queries over time and across the average pool of users.
One thing that is more interesting than anything mentioned in the /. summary is the idea of "good abandonment" of the search engine results. They made
Re: (Score:2)
So I walk away to, oh, I don't know take a piss or something, and when I come back wherever I bumped my mouse getting up is the most relevant thing in my search? Riiight.
Wow, you've sure demostrated with that anecdote that statistical analysis is a bunch of nonsense! Incredibly insightful!
So if I'm reading correctly, EVERY TIME you do a search you get up and take a piss, bumping the cursor in the process?
Or you think--for some undisclosed reason--that everyone does this kind of thing often enough to overwhelm any statistical power in these results?
Why is that, exactly? Where are the data on how often people get up, bump their mouse, etc? Why do you believe that accidenta
Re: (Score:2)
Way to read a thread, fucktard.
Re: (Score:2)
Way to read the thread, fucktard. Notice my reply to a reply was already posted an hour before you replied. Read it. Fucking asshole.
efficient algorithm written in Javascript (Score:2, Insightful)
From TFA:
They use an efficient algorithm written in Javascript.
Is it me or they are using two incompatible words in the same phrase?
Re:efficient algorithm written in Javascript (Score:4, Interesting)
have fun optimizing bubble sort in assembler ...
Re: (Score:3, Insightful)
Re: (Score:1)
Pointer vs Cursor (Score:2)
Thought I'ld comment on "It's not a cursor, it's called a pointer." but then WikiPedia says that MS tends to use carets to call cursors and cursors as pointers, and that some people use text cursor and mouse pointer to call the two respectively, to avoid ambiguities. So, that is new to me, as I've always called the blinking vertical line where text will be entered as cursor, and the arrow-like thing that moves when you move the mouse as pointer. Which is also what I teach my students. Now, I question why
Re: (Score:1)
In windows they are called carets (typing) and cursors (mouse) . I miss the really old days (win31) when you'd sometimes call your own caret create and destroy functions. You could have two! :)
Re: (Score:2)
They're both interchangeable and have been for decades.
The ZX Spectrum had "cursor keys" which were originally to move the text "cursor" around. Most games used the cursor keys to move pointers rather than a text cursor. By the time the +2 came out, the cursor keys were used to navigate menus in the default ROM. Even before that, someone re-used those keys in a keyboard-mapped joystick and called it a Cursor joystick. That then moved into all games I've ever played on the system calling it a cursor, not
Re: (Score:1)
I hear ya. I only started to know the terms cursor and pointer when I started teaching. And I have to teach according to syllabus. It'll be of no benefit to my students if I teach them other than what CIE advocates me teaching. So I'll stick to cursors and pointers. But, you are right and I agree that we, as people/consumers, will start calling things like the way we understand them most, even if they're not as exact as the original term/name, could even be opposite, e.g. Bruce Springsteen's 'Born in t
Re: (Score:1)
After 15 years of computer support, I'd have to say that the cursor is generally sitting at the desk looking at the screen or some other peripheral. You can usually gauge the level of difficulty the cursor is experiencing based on the volume of the cursing.
we need more front-ends to the front-ends (Score:4, Interesting)
I hate javascript. I knew from the very start it would be a source of abuse more than use. I knew it and avoided using it in my code whenever possible.
I turn it off at every chance and only allow it sparingly on some sites.
and now they want to meta-watch us. just wonderful. as if working around the 'helpful autocompletes' all over the place (I'm talking to you, google) isn't enough hassle to get the data we want.
typing into text boxes is a huge pita. SO much other processing goes not (even if you disable spellchecking) that I lose characters, find repeated characters and the typing lag is WORSE than it was 20 yrs ago with an 8 or 16bit cpu and less than a meg of ram. I'm not kidding - the more js that runs 'in the background' the laggier shit gets as you type. my 3ghz dualcore 'drops characters' like its no one business; and I know that I'm not alone since I see so many posts on so many forums with dropped and repeated chars. the web SHIT has taken over our computers and only gives us tiny slices of time to do OUR work in.
the solution is to go back to a batch oriented web again, for at least some things. I do NOT want 'journalling - saving!' happening WHILE I am typing in a text box. the only thing happening should be cursor moves, chars entered and cursor blinking. while in emacs or vi, my text speed is very fast; so why is it that web-based text areas are SLOW AS SHIT ?
its the javascript. the language for advertisers and webfuckers (what I call webmasters who fuck wtih your browser, thinking that THEY own the formating and content display on your system and not you).
do a view source and see all the crud that comes thru. how much is really need to query and display results? I could do that in simple forms/cgi's and bypass all the crud.
so, we need more 'submit' style front ends that sit there and do NOTHING until you hit submit. no animation, no character counting, no 'journalling - saved!' bullshit. no copying of my data to you 'in case'. just fucking sit there, take my text and when *I* hit submit, THEN you can bulk upload it to the main server.
javascript annoys the hell out of me. it has ruined what was once a nice responsive web. now, I drop characters as the background jscript tasks own more cpu than the foreground ones do. ;(
Re: (Score:1)
I've have started using NoScript since recently, and remove those cheesy Skybuttons that came with kopete, and it's amazing how responsive Firefox has now become :-)
Re: (Score:2, Funny)
Upgrade your computer more often. If you want to stay ahead of bloated software, you still need to spend money on serious hardware. My annual upgrade budget is $1000/year. I budget that in right below my health and car insurance in terms of priority. My current refresh is an i7 overclocked to 4.1 ghz, 12 gigs of ram, an intel X-25m ssd, and so on. Web seems plenty responsive to me.
A lot of my "searches" are not searches... (Score:2)
SEO (Score:1)
Queue the automated search that sends back fake interaction data.
Re: (Score:2)
Yeah, the more details they try to mine from their user base, the easier it is for people like us to completely fuck with their results. ;)
Microsoft NEEDS to track gestures for Windows UI (Score:4, Interesting)
What Microsoft REALLY needs to do a better job of is tracking mouse movement (specifically, acceleration, ballistics, etc) to do a better job of discerning intent when you go to grab something... and a better job of adaptively figuring out over time whether its assumptions about your intent are right or wrong. It really seems like every new version of WIndows leaves me fighting and frustrated with it a tiny bit more.
Here's a real-world concrete example. Suppose the mouse pointer is approaching the right edge of a window that's maximized to the left panel of a multi-monitor setup. The mouse pointer slows down, and seems to also be approaching the scrollbar. The left button gets pressed, and the mouse moves in a direction that's mostly upwards. Well, except the pointer overshot the edge a bit, and the left click technically occurred 2 pixels into the window on the monitor adjacent to the right. Taken in isolation, Windows has no real choice but to assume the user meant to click the pixel on the other monitor even though it contextually makes no sense. But combined with the observed ballistics (slowing down, slight arc towards the scrollbar, motion after left-click that makes more sense as a scroll-gesture than a... well... meaningless gesture), it's obvious what the user meant to click. And for the most part, Windows, seems to be completely oblivious to it.
Now, for a counter-example: trying to select text without adjacent whitespace. For me, Windows (Word and Outlook in particular) NEVER seems to get this right. I'll click at the right starting point, letting go and starting over if I'm not happy with it. Then I'll start highlighting. But way too often, it'll stop selecting a character or two short of where I want. If I keep moving the mouse, it'll grudgingly select the remaining characters... but feels compelled to ALSO ignore my hard initial-selection work and expand the other end of the selection too. Dammit. I'll then spend the next 10 seconds fighting with it trying to select the text I REALLY want. Half the time, I'm forced to give up, let it select the damn adjacent whitespace, and edit it away after I paste. It annoys me to no end.
In the end, it feels like Windows has simultaneously gotten worse in two directions. It forces its opinion on me without learning from its mistakes or giving me the option to beat it into submission so it quits interfering, and simultaneously forces selection with almost single-pixel precision to make increasingly-dense window gadgets work. I'll admit that Java is even worse in this regard, and Linux (or at least Gnome/Compiz) doesn't seem to be any better, but it's still annoying as hell.
Re: (Score:2)
I prefer a computer to do what I want to do. However, given that the computer cannot read my mind, it will probably do what it THINKS I want to do...
Re: (Score:1)
If they got feedback that the first time it was this slow, but then subsequently it took 2 seconds every time, great - we can chock it up to learni
Re: (Score:2)
but feels compelled to ALSO ignore my hard initial-selection work and expand the other end of the selection too.
Yeah, I remember that shit too, before the ribbon made me give up Office completely. There's actually an option somewhere called 'intelligent selection' or some stupid crap like that. Remove it and it will select from where you clic.
Re: (Score:1)
Do you have a smart phone? You'd love Swype [swypeinc.com]. It's basically what you describe but for the soft-keyboard on a touchscreen. You put your finger down somewhere near the letter your word starts with, drag it around in the general direction and neighborhood of the rest of the letters in the word, and then lift it up. Swype calculates a probability of what word you were trying to type based on the shape, even if you didn't quite touch all the letters in the word. If it's confident enough, it inserts the word into
ClickTale is doing this for sometime now (Score:1)
Can they tell ... (Score:2)
OCD (Score:2)
In more news... (Score:3)
Microsoft Research is found to be copying Google.
Roughly a year or two ago Google made it known they've long been searching the subject and have long had plans to bring this to market.
Suddenly both Apple and Microsoft are constantly trailing Google. You can say what you want about the three companies, but clearly both Apple and Microsoft have fallen behind Google as so far as real tech companies go. I'll leave it as a reader exercise to determine if its Apple or Microsoft which claims position two or three.
again? (Score:1)
http://tech.slashdot.org/story/10/07/27/1624251/Google-Nabs-Patent-To-Monitor-Your-Cursor-Movement
http://slashdot.org/comments.pl?sid=1734136&cid=33049918
Won't be long... (Score:2)
Re: (Score:1)
FPS will track your heart rate, so that if you are shot, you need to take cover, and let your heart rate go down to heal... (or something to that effect)
There, now we have mice with heart rate monitors.
Google's turn.
Frankly... (Score:2)
What surprised me more about this story is the fact that they found enough Bing users to get a decent dataset for their research.
Not always correct. (Score:1)
Hover events and usability (Score:1)
Re: (Score:1)
I have Tourettes Syndrome (srsly), and have a habit of randomly clicking my mouse buttons on empty areas of the screen while reading. Let's see you read into THAT, bitches!
You are obviously very interested in white space. Expect empty pages to get top rank in your searches!