Forgot your password?
typodupeerror
AI Google Your Rights Online

Google Aims To Cull Child Porn By Algorithm, Not Human Review 306

Posted by timothy
from the safer-search dept.
According to a story at VentureBeat, "Google is working on a new database of flagged images of child porn and abuse that can be shared with other search engines and child protection organizations. The database will help create systems that automatically eliminate that sort of content. ... If the database is used effectively, any flagged image in the database would not be searchable through participating search engines or web hosting providers. And maybe best of all, computers will automatically flag and remove these images without any human needing to see them." Here's the announcement.
This discussion has been archived. No new comments can be posted.

Google Aims To Cull Child Porn By Algorithm, Not Human Review

Comments Filter:
  • by Electricity Likes Me (1098643) on Monday June 17, 2013 @08:09AM (#44027683)

    What is the point of automatically removing child porn so it's not searchable. That's not the problem with child porn.

    The problem with child porn is real children are being really abused to make it.

    Making it "not searchable" doesn't stop that. Arresting the people who are making it does.

    • by TheBlackMan (1458563) on Monday June 17, 2013 @08:13AM (#44027701)
      Exactly.

      Also, I am browsing the net since at least 12 years and i have NEVER found child porn by accident or whatsoever. I am thinking that child porn can be found only in the "dark internet".

      So that makes one wonder what Google's real motives are.
      • by ebno-10db (1459097) on Monday June 17, 2013 @08:49AM (#44027935)
        Realistically this is just a feel good effort. No one is going to seriously criticize Google for this, and they can say "we're doing our part". Not that their part really helps anything, but that's not Google's fault.

        So that makes one wonder what Google's real motives are.

        Good PR. I'm as cynical as the next person, but PR is often the only motive for these things. If they had a sinister motive, they'd just offer to help the NSA some more.

        • Ding, have a cookie.

          "Think of the children!" is the perfect answer to "Hey, why are you handing all the data to the government investigators?"

          • "Think of the children!" is the perfect answer to "Hey, why are you handing all the data to the government investigators?"

            Why do they need to answer that question? You're assuming the proles have any power. Besides, in 21st century America, "terrorism" trumps even "think of the children".

            • by Razgorov Prikazka (1699498) on Monday June 17, 2013 @09:37AM (#44028405)
              That is correct. The list looks a bit like this:
              1 - fighting terrorists
              2 - fighting child pornography
              3 - something with global warming
              4 - helping the government spy on ordinary people.
              • by fafalone (633739)
                You missed the other thing responsible for nearly as much civil rights loss as terrorism:
                - Fighting drugs
                The constitution has been gutted in the name of stopping people from voluntarily ingesting certain substances since even before 9/11. I guarantee someone is going to stand up and say, 'Why can't we use this information to stop those evil drug traffickers?' and no one will object.
          • There was a story about how much Google resents the NSA six days ago [slashdot.org], so I don't think you've hit the nail on the head here.
        • by Synerg1y (2169962)

          Perhaps they're tired of all the FBI /NSA requests and are trying to remove themselves out of that picture.

          I also heard that google content inspector is a job that requires therapy.

      • by cdrudge (68377) on Monday June 17, 2013 @08:49AM (#44027937) Homepage

        Also, I am browsing the net since at least 12 years and i have NEVER found child porn by accident or whatsoever. I am thinking that child porn can be found only in the "dark internet".

        Unfortunately it is out there. In a previous life as an intern I received a computer from a retail store that needed "fixed" as the store manager put it. Figuring it had some malware on it I booted it up to see what the damage was. Almost as soon as the computer was started numerous browser sessions autostarted with some of the most vile websites you wouldn't want to imagine. It wasn't a picture or two of some amateur girlfriend that might have been a little too young. They had the appearance of professionally designed and maintained websites just like any other porn website, but just happened to have kids 13- instead of 18+. I just turned off the computer, went to my boss, explained briefly what I found and said I wasn't dealing with it.

        That was 13+ years ago. I'm sure things have changed some since then, but I'm also not naive to think that child porn is just on the "dark internet" whatever that is.

        • dark internet

          FWIW, according to Wikipedia the term you're looking for is 'darknet' or 'deep web' [wikipedia.org]. I love clear terminology.

        • I was on the internet before it was the internet. My opinion is that the internet has actually become a powerful weapon against pedophiles. Denmark in the early 90's was the "tipping point" but since then many other western nations have started flushing some of these predators out of their own churches and state institutions. I don't know what the answer is since people who enjoy watching or participating in the act of dehumanizing and torturing a child, are by any definition 'sick'.
      • by aevan (903814)
        I have. Lot more common in the 90s (you'd see it as a banner ads on download sites for cracked software-and I'm talking porn, not just naked children like some russian nudism thing), but it's still out there. Wouldn't call those sites 'dark internet' either, as an altavista search could pull them up.

        More recently, remember that girl that did the 'my life sucks' then suicided? Uncensored autopsy pics got pulled up by Google- CP according to some definitions.

        Still, totally agree with grandparent post
      • by Splab (574204)

        Obviously, you have never hanged out on 4chan.

        It is very much alive and kicking on the regular net, else police wouldn't be apprehending so many pervs (tracking people on TOR is almost impossible).

    • by Ardyvee (2447206) on Monday June 17, 2013 @08:15AM (#44027709)

      The summary is a bit incomplete. I suppose that if the algorithm finds something, it will warm law enforcement.

      FTFA: "This will enable companies, law enforcement and charities to better collaborate on detecting and removing these images, and to take action against the criminals." "We can do a lot to ensure it’s not available online—and that when people try to share this disgusting content they are caught and prosecuted. "

    • Re: (Score:2, Interesting)

      by Anonymous Coward

      The problem with arresting the people who are making child porn is that it provides the government no excuse to monitor all internet traffic of innocent citizens.

    • by ebno-10db (1459097) on Monday June 17, 2013 @08:23AM (#44027779)
      True, which means this isn't a solution. It is about as much as a search engine can do though, so it's to Google's credit.
      • "Something must be done. This is something. Therefore, it must be done!"
    • You are absolutely correct that this won't make child porn disappear. But from Google's standpoint, it will help keep their top-notch search engine (and other search engines) from being used to find it. In addition, it's more than making it "not searchable"; RTFA. This will also have "hooks" into law enforcement and ISPs.

    • by Joce640k (829181)

      Making it "not searchable" doesn't stop that. Arresting the people who are making it does.

      Nope, it makes it more valuable to the people who distribute it - no more pesky freeloaders!

      (just like drugs, etc.)

    • Re: (Score:3, Insightful)

      by gweihir (88907)

      You are looking at this from the wrong angle. It is extremely likely that this is not about CP at all, but that Google wanted an effective image censorship system. They realized that this may be hard to sell to the public, so they found CP as an easy solution. They can even use it for that only for the first few months (which will be almost invisible, as there cannot be a lot of CP accessible via Google, if there is anything at all...), then they an start to put in other pictures that are "undesirable", lik

      • by Xest (935314)

        Yes, that's right, Google, the firm that publicises all DMCA requests it receives and flags up when it's been forced to censor search results by linking to the request on relevant searches, Google, that publishes the source code for many of it's products, Google, that produces a regular transparency report stating as much as it can about what data it's been requested to hand over, what it's been requested to censor and so forth has just arbitrarily decided one day that it wants to censor images. We don't kn

      • And if anybody protests, they can just report them for searching CP.

        At which point plausible deniability [twitter.com] adds reasonable doubt. See also CP (disambiguation) [encyclopediadramatica.se].

    • by c (8461) <beauregardcp@gmail.com> on Monday June 17, 2013 @09:22AM (#44028241)

      What is the point of automatically removing child porn so it's not searchable.

      Well, if it works to prevent people from seeing it unintentionally then it means the Google search engine provides more relevant search results. So that's a major improvement in Google's search engine.

      If it's automatically identified removed, then presumably Google would be able to purge ephemeral copies from their caches and whatnot, which is probably nice from a liability perspective.

      It might help to reduce casual interest in the subject if it's not easily searchable.

      It doubt it would prevent anyone actively trying to find it, and it certainly won't stop the kinds of people who would go to the length of producing it; at least, I can't imagine that fame through improved search engine results is a significant part of their motivation.

      The question is what is the impact on the people who might make a transition from casual interest (if they could view it by searching) to actual production? If it helps prevents that, it's a win. On the other hand, if these people deal with frustrated urges by just going ahead and making their own, we'd have to call it a major failure.

      Ideally, someone has actually done the research and determined that yes, blocking casual searches for child porn should amount to a net benefit.

      In practice it wouldn't surprise me if it's a move to reduce the threat from Attorney General's who see child porn in Google's search results as an easy PR and courtroom win.

    • Exactly. How about they automatically find out who's behind the website and arrest them in their home country or shut down the hosting company hosting it?
    • by J'raxis (248192)

      The point is to try to sell automated censorware to the public by saying it'll only be used against something "everyone" thinks ought to be censored. Once it's established, it's scope will be expanded to cover all sorts of other materials.

    • Making it "not searchable" doesn't stop that. Shooting the people who are making it does.

      Just a minor correction.
    • What is the point of automatically removing child porn so it's not searchable. That's not the problem with child porn.

      The problem with child porn is real children are being really abused to make it. Making it "not searchable" doesn't stop that. .

      The point, I would expect, is that by removing the channel by which it circulates puts a barrier between the demand and the source, and hence reduces the incentive to make it. That would reduces the amount which is made.

      Arresting the people who are making it does.

      I don't think that this proposal was intended to be instead of arresting the people who make it.

      With that said, your point "The problem with child porn is real children are being really abused to make it." is a good one. By that argument, any such material which was not produced using real

      • The point, I would expect, is that by removing the channel by which it circulates puts a barrier between the demand and the source, and hence reduces the incentive to make it.

        That, in fact, was the judicial reasoning on the constitutionality of child pornography laws. At this point, though, I have my doubts that this sort of economic argument is valid; I doubt that the consumers of child pornography are paying for it in any way. After all, we are supposed to believe that the Internet has ruined the MPAA's and RIAA's business, and child pornography is obviously not subject to copyrights.

    • Agreed, but they're both parts of a joint strategy. Google is helping make the stuff unfindable ... which makes it go more underground, raising its value, which gives a much bigger incentive to the dirtbags that do this stuff. So ... oops ... this is the digital equivalent of the Prohibition laws of the 1930s.

      Imagine if this leads someone to build a custom search engine that specifically goes out to find child porn, because the Google search engine is functionally crippled. Oops.

      I started this post think

  • by Mal-2 (675116) on Monday June 17, 2013 @08:10AM (#44027689) Homepage Journal

    If they mean "all underage" and not just "blatantly children", good luck with that. There are no characteristics that will distinguish between 17 and 18, or even older. What is the software going to think of Kat Young, for example? What about models who are just small?

    Also are they going to attempt to sort through drawings at all, considering they are legal in some jurisdictions and not others?

    I sense false positives and angry models in Google's future.

    • by Ardyvee (2447206)

      Yes, this seems to be a difficulty they may face, and something I would like to see addressed. While I have seen things I would rather have not (and not necessarily related to this topic), I'm not entirely sure I would deem it illegal. Maybe I'm just biased by finding it as something that just comes with the Internet as part of the package. That and as long as it's not shoved in my face I don't really mind it existing*

      *of course, child porn/abuse is illegal, so my personal view on it is kind of irrelevant

    • by Barny (103770)

      I for one can't wait until the Aussie government get in on this. Women with small breasts will be flagged and, as you said, drawings too.

      It is a good effort, but the world is really becoming just a little too fucked up to start trying to stop things now.

    • by gmack (197796)

      The summary is a bit off. The algorithm can't actually detect child porn, what this looks like is a system similar to the one Youtube uses where one item gets reported and it's blocked globally rather than have to have someone report each instance of the same image.

    • by ebno-10db (1459097) on Monday June 17, 2013 @08:36AM (#44027851)
      This is about detecting known images (presumably even if altered a bit), not automatically detecting if a heretofore unseen image is CP. From the Google announcement:

      Since 2008, we’ve used “hashing” technology to tag known child sexual abuse images, allowing us to identify duplicate images which may exist elsewhere. Each offending image in effect gets a unique ID that our computers can recognize without humans having to view them again . Recently, we’ve started working to incorporate encrypted “fingerprints” of child sexual abuse images into a cross-industry database. This will enable companies, law enforcement and charities to better collaborate on detecting and removing these images, and to take action against the criminals. Today we’ve also announced a $2 million Child Protection Technology Fund to encourage the development of ever more effective tools. [emphasis added]

    • As usually by the time it made it to the Slashdot headline it was completely miss-reported. They are working on a DB to share hashes of known images. This only prevents them having to review the image on each new URL, someone still has to have seen the image and added it to the DB (to be honest they had better be conforming these flags at least sometimes so that's two people etc.etc.)

    • by wvmarle (1070040)

      This system is not "looking" at images. It is a database of hashes of known offending files, against which found content can be compared. Matching content will be filtered.

      Of course this only works for known files (which are flagged by humans, I supposed, though that is not explicitly mentioned in TFA), and if a file is altered the hash changes. Though that doesn't happen too often, most people share content they find unaltered. And it doesn't work for new files, either. Those still need to be flagged - how

      • by rioki (1328185)

        If you look at how good / bad the YouTube content-id system works you can infer how this system works. Probably the same tech, which has some leeway for fuzzy mapping.

    • by Viol8 (599362) on Monday June 17, 2013 @08:49AM (#44027931)

      The age of consent in spain is 14, in the uk 16, in the USA 18 , so if there's a picture of a nude 15 or 17 year old in what country does it get to decided if its legal?

      While this may be a laudable effort I have the sneaking feeling the USA once again will be pushing its legal system and morality onto the rest of the world.

      • by rioki (1328185)

        age of consent != age to publish pornographic content

        AFAIK almost everywhere you have to be an adult (18/21) to view and publish pornographic content.

      • by jonbryce (703250)

        In the UK, the age of consent is 16, but it is illegal to publish pictures of the activity if you are under 18.

    • If they mean "all underage" and not just "blatantly children", good luck with that. There are no characteristics that will distinguish between 17 and 18, or even older. What is the software going to think of Kat Young, for example? What about models who are just small?

      This is a common matching problem for which a solution has been know for quite a long time: Probabilistic weights.

      Algorithms can simply apply a probabilistic weight to the matches. Kat Young alike = 0.001. Obvious toddler = 1.0. Barring the Kat Young/small model outliers, there are physical markers that can distinguish between a normal 17 year old from a 10 year old from a 4 year old.

      You, or rather we, cannot do anything about in terms of automatic (or even manual/visual) matching of physical characteris

  • This is a really good idea.
  • by Semmi Zamunda (2897397) on Monday June 17, 2013 @08:21AM (#44027765)
    How about instead you compile a list of where these images are HOSTED.....and then DO SOMETHING about that? Notify local law enforcement of the images and give all garnered info about said images to them.
  • by realsilly (186931) on Monday June 17, 2013 @08:29AM (#44027819)

    Let me be clear about this. I DO NOT condone child pornography at all; I find it foul and disgusting. But there is a over-reaching that I think may go on here. If I purchase a server and I engage in a P2P network, then it is not Google nor any one else's business what I transmit. If the server is a public server or one owned by a company (such as Google), then I would agree they have every right to remove such foul content from their servers.

    Yes I would rather that the people who engage in this be stopped. But whenever programs like this are created they tend to start out being put to use with the best of intentions, but will likely be used for other more nefarious purposes. If this algorithm is used to sniff out child pornography, it could be modified to sniff out a information about a political party and quell it, or news that a government agency doesn't want people to know about.

    With all that has recently come to light about the spying by the US Govt. can you really say that this with 100% certainty that this technology won't be abuse for other purposes? I can't.

    Again I DO NOT condone Child Pornography.

    • by jbmartin6 (1232050) on Monday June 17, 2013 @08:41AM (#44027889)
      My reaction was something similar. I question the value of a search engine when it is no longer neutral. Now I will only see what Google has decided it is in my interest to see. This technology will be used in the future to skew political searches for example, or to favor one company's products over another's. (If it isn't already.) Now if they said 'we are using Google's search engine to catch child pornographers' I would say good for you please continue.
      • Now I will only see what Google has decided it is in my interest to see.

        They've been doing that for years.

    • by wvmarle (1070040)

      Google and other search engines filter content already - like Google's "safe search" options to block images showing naked people to appear in their image search. The technology exists, and "safe search" appears to actually analyse images to judge the content, while this child porn database only compares file hashes against known offending content.

      The technology is there, it's not new, this is just a new application of it. And I have to say I'm quite confident that it's not being used for political purposes

    • This is certainly, unarguably, a useful tool that can be used in order to accomplish a worthy societal goal; I don't think our criteria for such things should be: "Well, it could be used for bad things, so we should stick our heads in the sand instead." No cars because they might be driven by bank robbers! No knives because they might be used to cut people instead of carrots! etc.

      In any case, content recognition algorithms already exist and are already used for nefarious purposes. Why not use those tools

    • The technology is out there. It will only get better (by a magnitudes of a 1000) in the next decade or so. It can be used by governments for all sorts of purposes - so the solution is not to limit the technology (which can't be done) but by limiting the government (which can be done).
    • by Daas (620469)

      Good grief, how hard is it to not be paranoid is this day and age? They don't want child porn to show up on their searches so they find a way to flag the images and not display them : that's it. If there comes a time when the technology is used malevolently to suppress political ideas, then you'll be able to bitch about it but going all "tinfoil hat" on it because there might me an hypothetical use which you don't like is just ridiculous.

      At the moment, Google uses mostly human beings to review flagged conte

  • by usuallylost (2468686) on Monday June 17, 2013 @08:37AM (#44027863)

    Removing child pornogragphy is a laudable goal.

    We just have to realize that it won't stop at that. From the what the article says it seems like that technology could be used for any image. At the very least I expect we'll see general copyright enforcement from this. Worst case we will see things like various regimes being able to use this to suppress images they don't like. Oh you have pictures of us slaughtering our opponents well we better put those on the bad list.

  • by gweihir (88907) on Monday June 17, 2013 @08:46AM (#44027913)

    This will increase child abuse. As soon as it becomes invisible, perpetrators are completely free to do whatever they like, as the public will not be aware it is a problem. The reason is that it addresses the wrong problem. Distribution of CP is a minor issue. Creation of CP (and all the child abuse that is not documented or does not end up on the Internet) is the real problem. It seems politicians have become so focused on distribution of CP, that nothing is being done anymore to fight actual child abuse. After all, distribution of CP gives nice and easy convictions and to hell with the children themselves.

    • This will increase child abuse. As soon as it becomes invisible, perpetrators are completely free to do whatever they like, as the public will not be aware it is a problem.

      This probably won't do squat to make it less visible, because it's already reasonably well hidden. Any poster of CP that's too dumb to keep it out of the range of search engines has probably already been caught. This is a feel good effort. I can't criticize it, but it won't have much effect one way or the other.

  • sheesh

    See the old story, by CM Kornbluth called the marching morons

  • The summary "By Algorithm, Not Human Review" implies that the algorithm is somehow evaluating pictures. In fact from TFA it is clear all it is doing is looking for copes of known existing images by hash-code. If it were examining images I would be worried about false positives, but as it just looks for know child porn I cannot see any down-side - this is a good move.
  • Judges have already declared that porn is basically undefinable, and I disagree with them that you know it when you see it.
    Added to this that you cannot tell the difference between a 15 yo and a 21 100% of the time, sure 95% you would get it right with that big of a range, but not always.
    And trying to tell the difference between 18 and 17 or 16 if more like a 50% chance of getting it right, regardless of if you are a computer or a human being.

    • Porn is undefinable partly because your culture has an important part in whether you consider something pornographic or not. Also, the intent of the individuals involved has a large impact. A naturist website might have pictures of underage nude children, but without being considered pornographic whereas a young teen (e.g. 15 years old) sharing a selfy of her breasts would be considered to be making CP.
    • Re:Impossible (Score:4, Interesting)

      by J'raxis (248192) on Monday June 17, 2013 @10:49AM (#44029321) Homepage

      A lot of people are misunderstanding what Google is developing here.

      This is not "automated" censorware that would make its own decisions as to what should be censored. It's not going to analyze images and decide to censor them on its own (which would result in the kind of false positives you claim).

      It's censorware that would, once a live human Google employee has viewed a piece of content and made the decision that that particular content is to be suppressed, globally remove all copies of the same content from Google's database. See, one of the major obstacles censors face nowadays is the so-called "Streisand effect," where suppressing a piece of offensive content results in dozens, sometimes thousands, of people mirroring the content and publicizing their mirrors. If people want the information, they'll get it; censorship always fails, and in many cases completely backfires.

      But, software like this will ensure that in the future, if a corporation or a government wants to suppress information, they will be able to do so.

      Of course, Google and others will only use this censorware to go after CP, which as we all know "everyone" hates, so I guess everything is okay.

      • OK, but if you actually read my post I was clearly saying that computers cannot do it, because it is simply impossible for anyone or thing to do it because they are indestrinquinable.
        Porn is often indistinguishable from non-porn, and children are indistinguishable from non-children. These are not just tiny edge cases, but significant portions of the content.

  • 1. Upload to Picasa picture of kids at birthday pool party holding balloon animals with long noses.
    2. End up on floor being beaten by local SWAT team.
    3. ??????
    4. Prison
  • If this system were 100% effective and preventing all known CP images from being searchable or even downloaded, then wouldn't that drive demand for brand new images to be created that don't trip the filters?

    • If this system were 100% effective

      It won't be, and no-one's expecting it to be. From the article it may be something as simple as storing hashes for specific files rather than specific images, which would hardly warrant the "new technology" status that the article has decided to give it. Even if it's more generic that, and is a hash for an image - allowing an image to be identified at different scales, exposures, or rotations - it's still not really new technology, nor is it particularly groundbreaking. It'll just be a handy tool which will

  • i think the problem with this system lies not in its intent but in its effects. I'm less concerned about whether it is searchable than about the abuses involved in creating it. I'm also concerned about the fact that we've seen charges involving 14-17 year old girls sending 14-17 year old boys their own pictures via their cellphones, marking them as felons and sex offenders for life. We need to figure out what is and isn't acceptable in our society and make it clear where that line is before matters get w

  • How come corporate Anti-Virus scanners don't scan (or have an add-on module to scan) for signatures of illicit images? If various government agencies have a collection of known infringing images, signatures could be generated, like viruses. Sure, there would likely be a way to fool it, but it would be step in the right direction.

  • In building an algorythim that can detect CP. Then they can combine it with their location detection methods to identify where it was produced. The big question is, are they then going to inform the LEO's based on the suspicion and ruin an individual who did not commit a crime? Are they becoming the new Thought Police ala "Orwells 1984"?

    Some of the things that Google has done are worthy efforts but this has the seeds of some serious abuse right from the beginning because if Google can do this for images, th

  • by Adrian Lopez (2615) on Monday June 17, 2013 @11:10AM (#44029567) Homepage

    What about borderline content such as non-pornographic nudity, sexually explicit drawings of imaginary minors, and pornographic images of adults who look like teenagers? It's likely these will be branded as "child pornography", leading to images being suppressed that are legal in many jurisdictions including the United States.

    Once service providers start censoring content based on third party reports of alleged child pornography, it becomes much easier to supress other content as well. Organizations such as RIAA and MPAA would love to be able to flag arbitrary content as infringing and have ISPs block such content automatically, bypassing even the need to file DMCA takedown notices. Think of how often YouTube videos are incorrectly flagged as examples copyright infringement and extend this to all ISPs who check against Google's database, and you can see the problem.

    ISPs who participate in this system delegate the right to make judgment calls on material that isn't obviously illegal to the maintainers of a central database whose judgment may or may not be consistent with local law. Anything in the database is assumed to be illegal regardless of its actual legal status, and the ISPs just follow along instead of deciding individually whether or not the content is likely to survive a legal challenge. Once the system becomes widespread, ISPs may even feel it is necessary to follow it to avoid secondary liability for content posted by their users.

    This is yet another example of a worrying trend, where content alleged to be illegal or infringing is removed without due process and often with little regard for the law and relevant jurisprudence. It's no way to run a network that for many has become a primary means of communication.

    Internet users deserve better than to have their content blocked according to extralegal judgments with perhaps no bearing on local law, little or no chance of appeal, and no way to establish legal precedents protecting certain kinds of content.

(1) Never draw what you can copy. (2) Never copy what you can trace. (3) Never trace what you can cut out and paste down.

Working...