With such impressive results, I wondered if modifying the mobile menu could move the needle even more, as some of the menu icons may not be easily interpreted by new mobile users as to what they represent. Adam graciously agreed to test button labels against the existing icons and share the results with Get Elastic readers.
And the results are in! You can read Adam's complete coverage of the test and its results on his blog, and we cover the highlights.
Left image is the control -- notice the Home link is DI for Detailed Image, the menu uses the conventional 3-bar icon, Account is illustrated by a User icon, and Cart for cart. The challenger explicitly states which buttons lead where, save the cart icon, of course!
Each variation received over 33,000 mobile visits, and both unique events (whether or not a visitor engaged with the navigation bar) and total events (the number of times a user engaged with it) were measured. At 95% confidence, the test version's improvement range was 20% – 35% for unique events, and 15% – 24% for total events.
Users who saw the test version also engaged less with the search box than users who saw the control navigation, to the tune of -16% – -5.6%.
We can't forget the money-metrics, conversion rate lift was 17.44%, and AOV 9.44% -- however, the test did not reach statistical significance at that level of traffic. In order to reach 95%, the test would need to run 15 times longer. Nevertheless, Detailed Image believes the data supports that the test menu outperformed the control, and text labels have been rolled out to all mobile visitors.
Again, full details including testing tools used and measurement methodology are provided on Adam's blog.
Have you tested responsive design elements on mobile? We want to hear from you. Drop me a line at linda DOT bustos @ elasticpath.com.