Try These AI-Powered Image Recognition Tools

What's Happening in This Picture?

A few months ago, I wrote about Wolfram|Alpha, a “computational knowledge engine” that provides computed answers to natural-language questions instead of merely returning a list of web pages that contain some of the keywords in the questions. Stephen Wolfram, and the folks behind Wolfram|Alpha have a related service called ImageIdentify, that answers the question, “What’s going on in this picture?”

Let's go back to that example of a baseball game. Make it nine little kids instead of grown men, and I’ll refine my answer to, “Little League game.” Such instant identifications are second nature to humans. But few of us realize what vast stores of facts and billions of lightning-fast computations are required to pull of this simple feat.

Artificial-intelligence experts know that image-recognition is incredibly complex, and it’s going to be utterly necessary in the future of self-driving vehicles and accurate facial recognition systems. Image-recognition is a problem that people like Stephen Wolfram and others are very keen to solve.

The ImageIdentify function is a simple way for a programmer to invoke a very complex program that operates on Wolfram|Alpha’s database of several trillion facts. This program has been exposed to over 10,000 “training images” and their associated identities or tags, e. g., “coffee mug.” Given an image as input via the ImageIdentify function, the program returns the tag of the image that is the closest fit to the input image.

Thankfully, you don't have to be a computer programmer to use ImageIdentify. You can try the ImageIdentify function on any image you like, at the Wolfram Language Image Identification Project website. You’ll also be helping the function expand and refine its knowledge base. Drag an image into the project’s input area, or upload an image file from your hard drive.

It will take just a few seconds to analyze the image and return a tag. Then you’ll be asked to rate the project’s performance on a scale labeled “Great,” “Could be better,” “Missed the point,” or “What the heck?!” If your rating is less than “great,” the tool will offer second guesses and ask you to tell it what you think the tag should be. You can leave your email address to be notified when ImageIdentify learns more about “your kind of image.”

Casinos, Chihuahuas and Party Hats

My own tests produced very uneven but not unamusing results. ImageIdentify correctly recognized a flower and even gave me its genus. It deemed me a “person.” An origami fish was a “vertebrate.” I tried several photos of dogs eating ice cream, and it was able to reliably identify dogs, and in most cases the breed as well. But it wasn't smart enough to "see" the ice cream cone, or understand what the dog was doing. In one instance, the result was "carnivorous mammal." ImageIdentify guessed correctly that one photo of a baseball stadium was a "ballpark" but thought that another one was a casino.

The FAQ attributes some bloopers to “training errors.” For example, ImageIdentify may have been trained to associate the word “Chihuahua” with several examples of the dog, one of which was wearing a party hat. So if you upload a picture of a party hat, it may be identified as a “Chihuahua.” Wolfram and company are working on such things.

There are other online image-recognition tools to which you can compare Wolfram’s ImageIdentify. Their results are generally similar but can be amusing in their own ways. Here are some you can try:

• CloudSight


• Clarifai


• Amazon Rekognition


• Google Cloud Vision


• IBM Visual Recognition

Amazon's Rekognition tool did a very good job of identfying my "dog eating ice cream" test photos, returning these tags: Dessert, Food, Mammal, Pet, Golden Retriever, Canine, Animal, Dog, Ice Cream. Google Cloud Vision returned less tags (Dog, Dessert, Ice cream, Food) but got it essentially right. Clarifai identified a puppy, and was the only one to include tags such as Lick, Cute, Funny and Adorable.

I expect this technology to become rapidly better. It has to... we don't want the robotic maid to throw all the chihuahuas in the trash after your kid's birthday party. Your thoughts on this topic are welcome. Post your comment or question below...