What's Happening in This Picture?

Category: Search-Engines

It’s a question that human minds can answer instantly in most cases. Show me a photo of nine uniformed men standing on a diamond-shaped field and I’ll tell you, “It’s a baseball game.” Computers have a much harder time answering such questions. But this new technology just might be a breakthrough. Read on to learn more and try it for yourself...

What is Wolfram Image ID?

First, Stephen Wolfram gave us , a “computational knowledge engine” that provides computed answers to natural-language questions instead of merely Web pages that contain some of the words in the questions. Then he developed the Wolfram Language, a programming language that draws much of its power from the algorithms and databases underpinning Wolfram|Alpha.

Now, Stephen has added a simple function to his language, ImageIdentify, that answers the question, “What’s going on in this picture?”

Let's go back to that example of a baseball game. Make it nine little kids instead of grown men, and I’ll refine my answer to, “Little League game.” Such instant identifications are second nature to humans; few of us realize what vast stores of facts and billions of lightning-fast computations are required to pull of this simple feat.

Dog Eating Bubble

Artificial-intelligence experts know that image-recognition is incredibly complex, and it’s going to be utterly necessary in the future of self-driving vehicles, self-piloting drones, and other autonomous machines. Image-recognition is a problem that people like Stephen Wolfram are very keen to solve.

The ImageIdentify function is a simple way for a programmer to invoke a very complex program that operates on Wolfram|Alpha’s database of several trillion facts. This program has been exposed to over 10,000 “training images” and their associated identities or tags, e. g., “coffee mug.” Given an image as input via the ImageIdentify function, the program returns the tag of the image that is the closest fit to the input image.

You can try the ImageIdentify function on any image you like, at the Wolfram Language Image Identification Project Web site. You’ll also be helping the function expand and refine its knowledge base.

You can drag an image into the project’s input area, or upload an image file from your hard drive. The project will take just a few seconds to analyze the image and return a tag. Then you’ll be asked to rate the project’s performance on a scale labeled “Great,” “Could be better,” “Missed the point,” or “What the heck?!” If your rating is less than “great,” the project will offer second guesses and ask you to tell it what you think the tag should be.

...and All I Got Was This Lousy T-Shirt

Finally, you can leave your email address to be notified when ImageIdentify learns more about “your kind of image” and, more importantly, to be entered in a daily random drawing for an “I taught ImageIdentify” T-shirt. That ought to get you a few “What the hecks?” of your own!

My own tests produced very uneven but not unamusing results. ImageIdentify correctly recognized a flower and even gave me its genus. It deemed me a “person.” An origami fish was a “vertebrate.” ImageIdentify guessed correctly that one photo of a baseball stadium was a "ballpark" but thought that another one was a casino.

The FAQ attributes some bloopers to “training errors.” For example, ImageIdentify may have been trained to associate the word “Chihuahua” with several examples of the dog, one of which was wearing a party hat. So if you upload a picture of a party hat, it may be identified as a “Chihuahua.” Wolfram and company are working on such things.

There are other image-recognition Web sites to which you can compare Wolfram’s ImageIdentify. Their results are generally similar but can be amusing in their own ways. Examples include CamFind, Clarifai, MetaMind, Orbeus, and AlchemyAPI.

I expect this technology to become rapidly better. It has to… we don't want the robotic maid to throw all the chihuahuas in the trash after your kid's birthday party. Your thoughts on this topic are welcome. Post your comment or question below...

Ask Your Computer or Internet Question

  (Enter your question in the box above.)

It's Guaranteed to Make You Smarter...

AskBob Updates: Boost your Internet IQ & solve computer problems.
Get your FREE Subscription!


Check out other articles in this category:

Link to this article from your site or blog. Just copy and paste from this box:

This article was posted by on 26 May 2015

For Fun: Buy Bob a Snickers.

Prev Article:
Really, a $9 Computer?

The Top Twenty
Next Article:
Geekly Update - 27 May 2015

Most recent comments on "What's Happening in This Picture?"

Posted by:

26 May 2015

Dog biting soap bubble.

Posted by:

26 May 2015

Speak for yourself. Personally, I would be happy for Rosie to can all the Chihuahuas. (I hope that my niece doesn't read this!) Thanx for more great stuff, Bob. I'm going to have to try some of my pics.

Posted by:

26 May 2015

WOW THis is fun! It tastes terrible and disappears so fast but do it some more. Come one just a few more times chasing this weird strange ball you're tossing me!

Posted by:

26 May 2015

Thank you for the Education on this intriguing topic, Mr. Bob Rankin!

I may be wrong but I am inclined to think that Google is also deeply involved in this technology. I have been meaning to research into how "Google Images" website manages obtaining and categorization of all of the image database they have.
The Google President, Eric Schmidt, stated that Google Images search was created because of the desire to view Jennifer Lopez in her exotic green Versace dress. I had posted a blog on GoogleImages recently, as follows:
In 2000, Google Search results were limited to simple pages of text with links, but the developers worked on developing this further, realizing that an image search was required to answer "the most popular search query" they had seen to date: Jennifer Lopez's green dress.
As a result of this, Google Image Search was born.
In 2001, 250 million images were indexed.
In 2005, this grew to 1 billion.
By 2010, the index reached 10 billion images.
As of July 2010, the service receives over one billion views a day.
Google introduced a 'sort by subject/ feature for a visual category scheme overview of a search query in May 2011.
In addition, one of the features of the GoogleImages website is allowing users to drag-and-drop any image into the search field for identification, that seems amazing to me. YMMV!

Posted by:

26 May 2015

I wonder what they do to weed out the trolls who will inevitably flock to the system to deliberately try to train it incorrectly. Statistically, their efforts may not hugely skew the results if there's a lot more "honest" training that happens, but then we know how determined some trolls can be.

Posted by:

Nancy Teppler
26 May 2015

Maybe this will render those annoying "CAPTCHA" utilities ineffective and force websites to come up with something better to identify users as humans. Thanks for the interesting info, Bob.

Posted by:

26 May 2015

Go ahead and help big brother build it's 'brain'....not me. :)

Posted by:

26 May 2015

Amazing what tech is doing! I posted a Merlin magician and it said a person... Not bad but unprecise. Then I posted 2 unicorns and it said vertabrate... again not bad but unprecise... Thank you for your very interesting posts, especialy this one.

Posted by:

26 May 2015

Very interesting article Bob. I wonder how it will prove useful to individuals though. I can see the value to driverless cars, hovercraft and the like but not to ordinary citizens. I'm sure part of this is due to my present coffee deprived brain but I tried CamFind for its novelty and after numerous errors I stopped. And ultimately, under what circumstances would the software recognize something I can't identify?

EDITOR'S NOTE: How about identifying a flower, a bird, or a pill? (ex: Is this Tylenol or Lipitor?) You could ask if a particular spider is poisonous, or... I'm sure you can think of others.

Posted by:

26 May 2015

You were nicer than I am in what you submitted!

Apparently, no one had ever submitted pictures of bunk beds. It guessed wine cellar. But it identified candy corn correctly, identified a field a corn as corn stalks, and a pile of corn on the cob as "corn". So that was good.

A picture of myself (I am white) a black African boy sitting on my lap was identified as humanoid. Okay, that was at least in the right Universe. The subtleties of seeing two people and recognizing that one is on the lap of another is probably pretty complicated.

I submitted the pictures of a hatchet, axe, and tomahawk. There are significant differences in sizes and shapes, but those may be too subtle. But, for a name related item, I submitted a Tomahawk Cruise Missile. I thought it would get it since it had the name on the side. But, it thought it was a jet liner.

This should make us humans feel better about our brains!

Posted by:

26 May 2015

I just tried a few more to check for subtle differences. I uploaded 4 different pictures of copperhead snakes. Two were coiled, one was stretched out, and another was in-between. Patterns were distinct enough that they were 100% positive to be copperhead snakes. The Wolfram site correctly identified two of them. But, called another a python (another poisonous snake) and a corn snake (non-poisonous). I could almost understand the corn snake because the patterns are similar. But, I would hate to have to rely on them to be correctly identified by this site!

Posted by:

26 May 2015

AI. Whatever folks may wish it to mean, it is neither; it is not artificial nor intelligence. It is digital and available library information. The term should be buried in the same grave as earth-centric science.

Posted by:

27 May 2015

Well, Bob, it stuffed me right away (in your example). Being a citizen of the UK, I had no idea that a baseball park was diamond-shaped! lol.

Posted by:

27 May 2015

I had a picture of comet that Rosetta took close up. It guessed bacteria. Does kind of look like bacteria under a microscope.

Posted by:

27 May 2015

You didn't mention the new Flickr (flickr.com) which includes something called "Magic View". It categorizes all your uploaded photos (you can up load 1 TB for free!) into 10 groups and dozens of subgroups. It's pretty accurate and really cool.

Posted by:

27 May 2015

Bob, great article!

This is the start of AI (Artificial Intelligence) and it will take years, if not, a century or two, to get it right. Computers simply are not setup, to think like human brains. Computers have circuits, not neutrons and receptors, which make brains think.

I honestly, think in the future, AI was become a reality, but, not for a long, long time.

Posted by:

David Guillaume
27 May 2015

A very interesting article about a technology that is still very much in its infancy. However in the long term I can foresee that it will be used for relaying images of battle casualties straight to the operating theatre and space exploration just to name a couple of the many hundreds of applications that will be developed for its use in the future
David Guillaume

Posted by:

Romke Egbers
03 Jun 2015

Well, Wolfram believes that my wife (with elbows on table and hands under the chin) is a "House of Cards". Not quite unexpected: her arms form a triangle.
And lo! Cloudsight (CamFind) does recognise her as "vrouwen zwart shirt" (and even answers in my language).
Clarifi even thinks, she is famous. MetaMind, though, is convinced that she is a bookcase.....
I like jewel (Rekognition (orbe.us)) or the neutral person (Alchemy more.

Post your Comments, Questions or Suggestions

*     *     (* = Required field)

    (Your email address will not be published)
(you may use HTML tags for style)

YES... spelling, punctuation, grammar and proper use of UPPER/lower case are important! Comments of a political nature are discouraged. Please limit your remarks to 3-4 paragraphs. If you want to see your comment posted, pay attention to these items.

All comments are reviewed, and may be edited or removed at the discretion of the moderator.

NOTE: Please, post comments on this article ONLY.
If you want to ask a question click here.

Free Tech Support -- Ask Bob Rankin
RSS   Add to My Yahoo!   Feedburner Feed
Subscribe to AskBobRankin Updates: Free Newsletter
Copyright © 2005 - Bob Rankin - All Rights Reserved
Privacy Policy

Article information: AskBobRankin -- What's Happening in This Picture? (Posted: 26 May 2015)
Source: https://askbobrankin.com/whats_happening_in_this_picture.html
Copyright © 2005 - Bob Rankin - All Rights Reserved