February 25th, 2015 | 6 MIN READ

Beyond Product Recommendations: Big Data's Role in Personalization

Written by author_profile_images Linda Bustos

Linda is an ecommerce industry analyst and consultant specializing in conversion optimization and digital transformation.

If 48% of consumers spend more when their experience is personalized, it’s no wonder that 90% of marketers believe personalization is “the future” and are making it a strategic priority this year.

While an ecommerce experience can be targeted by immediate signals of context like geolocation, clickstream behaviour, site-searches, cookies and campaign referrals, true personalization relies on data to drive the experience. And for many ebusinesses, Big Data is part of the equation.

Big Data’s one of several key elements of personalization covered in FitForCommerce’s annual report Let’s Get Personal, Personalization in a Hyper-Connected World. How can online businesses harness and actually use Big Data to effectively personalize customer experience?

A brief history of personalization

Beyond welcoming a returning visitor with “Hi Joe!," the manual, rules-based product recommendation engine was ecommerce’s first and simplest personalization tool. A marketer could create associations between products A and products X, Y and Z for cross-sell and upsell opportunities, with data living in simple tables.

Automated, “collaborative filtering” pioneered by Amazon in the early 2000’s came next, recommending products based on actual customer browsing and buying behavior, rather than a merchandiser’s gut. The engineers behind this technology went on to found their own recommendation engine startup, and an industry was born. Online retailers have their choice of recommendation technology.

Beyond product recommendation

Product recommendation engines do a decent job of personalizing a user’s experience, but stand-alone, they only leverage on-site context (profile, clickstream and purchase data), often within a single or handful of visits (until the customer clears cookies, or changes devices, for example). But today’s marketers have access to much more contextual information. Email engagement and response, campaign referrals, mobile usage, social interactions, geolocation, customer service logs, in-store behaviour and optimization testing are just a few examples. And the freshness of this data is increasingly important for accurate targeting and personalization.

This means marketers have a lot more data to work with – more than a recommendation engine alone can handle.

Big Data driven personalization

Personalizing the customer experience across channels and beyond what you can target based on Web behavior and account history requires tapping into Big Data. Big Data involves the collection, management and analysis of multiple sources of data, both structured and unstructured, and each organization has its own custom blend of Big Data to work with.

It’s not for lack of data

FitForCommerce asked over 100 C-suite and key decision makers “do you have the data needed to truly personalize the shopper’s experience?” 35% say “we have data, it’s not usable and we don’t know how to use it.”


33% report they don’t have enough data, and the remaining 33% are happy with their data and believe it’s usable.

Of the 33% of respondents that believe they have enough usable data, a good chunk likely have data that could be used to extend and scale personalization efforts beyond the basics. A universal problem for organizations is how to truly leverage these disconnected but valuable data sources and bring them into the cross-touchpoint digital experience.

For more information on survey results, check out the infographic Omnichannel Personalization in a Hyper-connected World

How do organizations make Big Data usable?

Technologies like Hive, Hadoop and Aster Data are commonly used to help manage Big Data, and some retailers like Walmart go as far as to acquire tech startups like Kosmix to roll into their solution.

When it comes to making Big Data accessible to ecommerce applications, a universal API framework can be used to connect technologies like Hadoop and various sources of data with an ecommerce engine, CMS or marketing cloud solution, and allow developers to create new experiences leveraging this data.

Big Data and ecommerce in action

Kohl’s triggered in-store offers

Knowing that customers are more likely to respond to a push offer when they’re shopping vs when they’re at work, in class or at Starbucks, Kohl’s is testing triggered, personalized cross-channel offers in select stores. When a customer is lingering in the shoe department, for example, the customer can receive an offer for shoes viewed online but never bought.

Target social sentiment

Target taps the Twitter firehose to determine the “social sentiment” response to new product launches and collections. This arms marketers and merchandisers with otherwise hard-to-parse qualitative data quickly.

Moosejaw loyalty

Moosejaw Mountaineering uses big data to optimize its loyalty program for customer satisfaction and retention. It tracks the lifecycle between earning and redemption of loyalty points, and identify the top products and categories within the program using data gathered across channel, marketing and communication touchpoints. This helps Moosejaw deliver more targeted offers, including points-about-to-expire notifications, and through the use of predictive analytics, better optimize marketing programs like catalog, direct mail and email campaigns.

Victoria’s “Secret”

Shhhh….Victoria’s Secret has “used mobile phone numbers and advanced context-aware technology to alert people when their significant others were searching for or showing interest in a particular product.”

FitForCommerce’s 8 tips for getting personal with Big Data

While the above examples are advanced moves, if you’re just getting started make sure you’ve covered these basics:

1. Dedicate resources and make sure basic data management processes are solid. Personalization is hard and part-time efforts against mediocre data sets are not going to be rewarding. IT and business managers should work hand-in-hand to define the basic data elements needed to support the personalization strategy.

2. Evaluate existing data management processes. Evaluate the technology currently leveraged for data management processes and determine if it will support personalization efforts. Always pay attention to design, whether it is database, Hadoop structures or data update processes. Experts can be of real assistance here.

3. Start simple. Start with the home page, PDP (product detail page), email, retargeting and get the basics right before jumping into complex personalization efforts.

4. Test, test, test. Do not start personalization campaigns without a testing tool and be prepared to engage in perpetual testing.

5. Do not panic about Big Data. Keep an eye on Big Data, but do not devote lots of bandwidth or capital if traditional data processing techniques can provide the insights needed for personalization. Focus on capturing the right data (big or small) that is easy to get to and is actionable.

6. Let customers opt in/out of personalization. Opting in/out gives the customer more control of what data they share and can help reduce the “creepy” factor that overly effective personalization can induce.

7. Be transparent. Clarify with customers what data is collected in order to provide them a superlative shopping experience.

8. Understand personalization data collection. If leveraging a personalization cloud solution, be sure to understand what data it uses and how it gets that data: from the enterprise, from the industry, from everyone?

Check out Let’s Get Personal, Personalization in a Hyper-Connected World featuring results from FitForCommerce's retail and brand executive survey, interviews, and first-hand insights from FFC's consultants. In addition to Big Data, the report covers marketing, mobile, social media, site content, product delivery, customer service, and privacy and security's impact on personalization.

Share on


Thanks for signing up!

You'll receive a welcome email shortly.

By submitting this you agree with our privacy policy.