December 16th, 2014 | 4 MIN READ

Ecommerce Trend: Keep Your Eyes on Visual Search

Written by author_profile_images Linda Bustos

Linda is an ecommerce industry analyst and consultant specializing in conversion optimization and digital transformation.

Visual search technology is one of the most exciting trends in ecommerce. Though the tech isn't new (Zappos had this feature in 2009 and Google bought in 2010), it's finding new life in ecommerce websites and mobile apps, with retail heavyweights Macy's and Neiman Marcus launching visual search apps for mobile this month.



Why should ecommerce professionals care about visual search?

Advantage over keyword-based search

Visual search "reads" images to identify color, shape, size and proportions, even text to identify brand and product names. This offers an advantage over keyword-matched search, in which results are only as good as the searcher's ability to describe an item.

With keyword search, keywords must exist in both the query and the product description or the page's metadata, and a match depends on product copy describing the product in the same or similar way.

The Web usability adage 'don't make me think' is satisfied by visual search when a picture is worth a thousand keywords. On mobile, visual search taken from photo can lead to quicker and more accurate results.

Visual search can also boost keyword search. Google Shopping's Visually Similar Items feature gives users a very contextual way to search deeper.



Support spearfishing

Customers 'spearfish' when they have a good idea of what they want and want to locate it quickly (vs. browse categories). Spearfishing with visual input provides better context than keyword search, and can deliver a tighter set of search results.

For example, a customer may be seeking a more affordable version of a couture handbag or pair of shoes discovered through social media.'s Sole Mates feature allows customers to upload photos of their own or link to image URLs to scour the site's vast catalog of footwear:



Some retailers build visual search apps for certain categories only. Tesco's wine finder app proved too much to maintain as an in-house R&D product, but now carries on through Cortexica's wine finder app (who partnered with Tesco to build the original).



Tilly's mobile app takes customers directly to online product pages from pictures snapped in its print catalog. Target's In a Snap app allows spearfishing from catalog, magazines and other printed ads. Both are examples of "exact match" applications of visual search.


Creative merchandising

Modista works how did before Google acquired it. As an affiliate-based shopping engine, users can explore the catalog and refine results by moving a slider more towards color or shape.


Online retailers can also quickly populate merchandising features based on color, shape or pattern. For example, Rakuten Taiwan's LGBT-themed feature:


Rakuten's product recommendations on quick look and product pages are also pulled from the visual search engine.


Merchandising beyond the storefront

Retailers that offer visual search can also provide it to 3rd parties as part of a shopping API strategy. Zappos provides affiliate developers access to its search similarity API so products can be remixed into new apps and experiences that Zappos doesn't build itself (one way to 'merchandise beyond the storefront').

Capturing showroomers

Amazon's Flow app is built to snipe sales from physical retailers by giving showroomers a quick way to locate products on Amazon.


While Amazon's is considered a "showrooming app," any retailer that offers visual search in mobile apps can combat showrooming, even benefit from customers showrooming other retail shops, and support 'endless aisle' for in-store customers looking to purchase items online (such as a different size or color of an item).

With visual search, virtually anything can be snapped-to-shop from the real world.

"With visual search you do not need to be in the retailer’s physical or digital store to have access to the products. The entire world is a showroom now, particularly with ‘snap to buy’ functionality" says Mark Elfenbein, CEO of Slyce, which powers Tillys' and Neiman Marcus' visual search apps.

Not just for fashion

“Fashion is a dominant market. However, home décor, consumer product goods advertising [for example] hairstyle and makeup products, and consumer electronics are all great use cases. There are also many uses of the technology for contextual search outside of purchases. You could snap a photo of an item to instantly add it to your mobile registry or to generate a mobile coupon; take a picture of a pair of jeans and get a Levi's coupon for example" says Elfbein.

How to get visual

There are several options for adding visual search to your ecommerce site. Slyce and Cortexica, as mentioned, along with Snap Fashion, which also has it's own marketplace and app.


Vivino and StubHub use Amazon's Firefly technology (which includes audio and video recognition as well as still images).

Amazon's Flow app has been criticized for not being able to handle certain types of products, but that doesn't mean all image recognition tools have the same problem. Slyce, for example, cites 90% accuracy. And popular iOS and Android app CamFind did a fine job identifying my test photo of a sample sized facial cream, reading the brand and product name amidst visual noise and poor lighting, correctly directing me to search results for the full product.


(CamFind also offers an API that developers can use in their own apps).

Will visual search be 2015's killer ecommerce app? We'll be "watching"...

Share on


Thanks for signing up!

You'll receive a welcome email shortly.

By submitting this you agree with our privacy policy.