Don’t Be Afraid of the Hamburger: A/B Test
Many months ago I ran a series of A/B tests. The idea was to see how much users engaged with the “hamburger” menu icon (compared with a number of variations).
The outcome freaked me out a little.
The data showed that the hamburger was not as well understood as first thought. Articles sprang up in agreement with this. It was as if an angry mob with pitchforks suddenly decided to lynch the hamburger icon.
Most people concluded that the hamburger icon was not a good thing, and users needed some additional clues.
That was then. What’s changed in 6-8 months?
Quick Recap – Previous Tests
All these tests were only run on mobiles.
20,000 visits. Compared hamburger icon variations.
The one with the border performed slightly better.
50,000 visits. Compared hamburger against the word MENU.
The word MENU only with a border performed best.
240,000 visits. Compared hamburger against the word MENU.
The word MENU performed 20% better.
The upshot was that users clicked (or tapped) the word menu more than they did the hamburger icon.
Now is a good time for a caveat:
These tests are conducted on a single website. There may be errors. I am not measuring a users intent, but simply button clicks. Draw your conclusions from the data. If in doubt measure and observe on your own site. I was not measuring an app, but a responsive website.
Glad we got that out of the way.
What The Data Says Now
The test had 434k mobile visits. Simple test of hamburger against the word MENU.
Hamburger is still weaker than the word MENU. This surprised me, as the hamburger is everywhere now. What’s going on here?
When I compared engagement stats (time on page, bounce, pages per visit) – there was no difference between hamburger and menu!
So, users are clicking MENU more but NOT going any further into the site. Did the word MENU make users more curious? But on clicking they didn’t want to go further? This is further indicated by the higher number of clicks compared to unique clicks (they tapped the MENU again to close the menu).
This kind of A/B test is weak, as my desired goals are increased engagement in the website. This is a nebulous goal to measure compared with this test – that showed an increase in bottom-line revenue (the winner was a colored button with hamburger + menu).
One other thing that has changed since the last test: Android users are clicking the hamburger more than MENU.
Hamburgers on Desktops?
I’m increasingly seeing the icon on larger viewport layouts. I have not tested the hamburger on larger layouts as it just didn’t make sense.
My feeling is the icon helps to conceal a list of functions that cannot fit onto a smaller layout (like a smartphone). Concealing all navigation options on desktop behind a hamburger may support a clean and spartan design, but (in my opinion) does nothing to assist navigation of the site.
If you want to hide all your navigation options – why have them at all?
Image from love2dev.com
The Overflow or Ellipses Icon
This icon appears to be growing in popularity. It indicates there are more functions – that presumably don’t fit. This is exactly what an ellipses has indicated for years and years.
However, it may not suit all use cases. This example from iTunes 12 is a case in point:
You want to find your apps?
Click the ellipses, THEN look for the apps icon. You have to do this every time you want to see your podcasts or apps (or whatever else is hidden in the ellipses). I have personally found this a frustrating aspect of the newer iTunes UI.
The good news is that you can edit the defaults, but this was not obvious at first.
The speed at which user interfaces change is breathtaking.
The only answer is to test. Then re-test, and stay with what works.