Google Preps 'Smart Screenshots' Feature To Let You Search With a Screenshot

According to Abner Li from 9to5Google, Google is working on a new "Smart Screenshots" feature that integrates Google Lens abilities into the Google app's screenshot function. From the report: The Google app has long had an "Edit & share screenshots" ability where captures made within Search would reveal cropping and annotation tools. Meanwhile, Assistant has long maintained a "What's on my screen" capability that analyzes what you're currently viewing for search suggestions. Google app 10.61 reveals work on "Smart Screenshots" that combine those two features. Like before, a toolbar -- which interestingly uses a four-color light bar -- appears after you take a screenshot. A small preview is shown at the left with a pencil button overlaid. You can open the system share sheet, but the Google app also suggests a frequently used app.

The most interesting addition is Lens. "Exploring with Lens" could be intended as a "Screen search" replacement given that Lens is increasingly taking over visual lookup throughout first-party apps, like Chrome. After taking a capture, Smart Screenshots have an easy way to invoke Lens for search, OCR, and finding visually "similar items." The existing editing tools (Annotating, Cropping, and Sharing) will remain and this new functionality appears to even use the same settings toggle to enable. It's unclear if this functionality once live will again be limited to screenshots taken within Search, or if it will expand to be systemwide and invokable anywhere. A notification from the Google app could appear after capturing a screenshot.

Re: Of course

[Way Smarter Than You]

Uh what? I can hit two buttons, get a screen shot, and it's conveniently saved to Dropbox in a screenshot folder named by date and time. How is using my phone to take a picture of my screen better than that?? This is 2019. Join us.

Re:

[stealth_finger]

Because it's quicker, simpler and more useful.

And it makes you look like a twat.

Re:

[LenKagetsu]

Most games have a dedicated screenshot button, Steam overlay has a dedicated screenshot button that can be compressed or uncompressed and a massive amount of free space to host images, some games have dedicated screenshot functions, Firefox has a dedicated "Screenshot the entire page from top to bottom" button. Windows has a dedicated screenshot to clipboard button, Linux has a dedicated screenshot to clipboard, Macs have a dedicated screenshot function. If you take a photo of a desktop, you need to pack up

Re:

[grapesandwich]

I do that on occasion. It's not that I don't know how to take a screenshot, it's just that I need a quick photo and how pretty or nice it looks doesn't matter. Or I'm not the one logged in/using the computer.

IOW, Google will datamine your screenshots too

[khchung]

to find out what you are interested enough to take screenshots to send you more targeted ads.

Re:IOW, Google will datamine your screenshots too

[Shark]

They'll probably come up with the 'automatic screenshot to the cloud every 25ms' feature to improve on that ability soon enough.

Re: IOW, Google will datamine your screenshots too

[Way Smarter Than You]

Every 25ms? No. Real time. And to make sure we are not missing any of your life critical information we will also save every keystroke and mouse movement and everything on your microphone and camera. All real time. All your your benefit! And we promise not to be evil.

Re:

[tlhIngan]

Every 25ms? No. Real time. And to make sure we are not missing any of your life critical information we will also save every keystroke and mouse movement and everything on your microphone and camera. All real time. All your your benefit! And we promise not to be evil.

I think that was supposed ot be real time, as 50fps is 20ms and 60fps is 16.67ms. Excepting the new high refresh rate screens that are coming out, a screen shot every 16/20 ms is basically a screen capture.

The real question is whether the OP me

Re: IOW, Google will datamine your screenshots to

[Way Smarter Than You]

Until 99% of your cpu and net are devoted to google, it is't enough.

Re:

[eaglesrule]

Or determine the political affiliation of the candidate's yard sign you're interested in, so they can offer you some very personalized search results. There's precedent with their ML Fairness doctrine.

"it’s like a small company cannot do that."

First, fix image search

[Deep Esophagus]

Somebody posts a meme that is an obvious misinterpretation of the context in which the photo was taken, or a faked image. To demonstrate this, I do an image search (after cropping out the overlaid text). Here's what comes up every time:

* Thousands of pinterest pages containing the same or similar images without attribution or source links
* Google's guess at what the picture contains ("baby" "dragon" "smurf"), then thousands of images also tagged with some of those same keyword

Essaynews

[Vasyl Bodnar]

I am so happy I found https://samedayessay.com./ [samedayessay.com.] Now I can pay a small amount of money and got my essay done in few days. The writer is very professional and put all her efforts into my literature subject.