Back in September Flowr set off on the Grasshopper / KISSmetrics Phone Number Challenge. The idea was that they were going test placing a phone number on their home page to see if they could increase sign ups. The hypothesis was that by having a visible phone number on their home page, the trust factor would increase and therefore sign ups would too.
Jonathan Kay from Grasshopper Virtual Phone Systems, proposed the original concept of this challenge. He believed that:
“People feel more comfortable with brands that they can put a face behind. Even though you might purchase a product exclusively online, having a phone number on your site and the ability to talk to a real person (who cares) in turn makes you feel more comfortable taking out your wallet (or recommending someone else to) for this brand.”
TheFlowr.com Home Page Variants
Before we get into the results of this simple A/B test, let’s quickly look at the differences between the two home pages.
The image below is the original Flowr home page. If you look closely, you will see that there is no phone number on the page.
In the next image, you will see a screenshot of the home page variant with a phone number and the call to action “Want to have a chat? Call us at..” (look for the red asterisk).
The Results
Flowr ran a simple A/B test with one home page variation using KISSmetrics. Again, the only difference in the variation home page was the addition of a small phone number and some supporting call-to-action text. The results were as follows:
Test Conditions
- Test Duration: From September 9, 2011 to October 24, 2011. (approx. 6 weeks)
- Test Item: Website home page of Flowr vs. home page variation.
- Test Type: A/B Test (only difference between variation was a phone number and associated call-to-action).
- Test Goal: Increase software sign ups from home page.
Results
- 53.96% of sign-ups originated from the home page variation with the phone number.
- 46.04% of sign-ups originated from the original home page without a phone number.
- Conversion Increase: +.5% (half of a percent increase)
Statistical Significance
We didn’t hit a statistically significant threshold during the allotted time for the test. However, we like the trend that we saw and we’re going to try another test with a bigger phone number next.
Conclusions
The first thing we would like to mention is:
The Flowr didn’t follow instructions and they still got some sign up lift!
The rules explicitly said to have a highly visible phone number on their home page variation. As you can see the phone number is tiny (at least it was above the fold). But even with this tiny phone number, Flowr was able to increase their sign ups.
Davorin Gabrovec from Flowr concluded:
“Even though we didn’t receive a lot of calls I believe that having a phone number visible on the website gives more credibility to our product and trust to our visitors. When we re-design our website we will definitely include appropriate space for a bigger phone number.”
The bottom line is that having a phone number does bring peace of mind to consumers and people you do business with. If, at the very least, it instills trust in your visitors and removes any “fly-by-night-operation” fear they may have. If you run a Software as a Service (SaaS) business, we encourage you to try testing a phone number on your site (and let us know what happens!).
About the Author: Sean Work is the Marketing Director at KISSmetrics.
Awesome Infographics ♥ Marketing Guides ♥ User Feedback Software ♥ Customer Analytics Software
Pretty interesting Sean, I honestly didn’t expect their to be much of a bump, but it’s nice knowing that a number does make a difference.
It’s good to hear about a simple, practical test of something like this that people usually just assume works.
Putting my pedantic mathematician’s hat on, you might want to be a little careful about how you interpret the results:
You say that the results weren’t statistically significant. Given that, I think the correct conclusion is that the test hasn’t shown either way whether the phone number increases conversions. Strictly speaking, I think to conclude that adding the phone number did help a small amount is not quite right in this case (you didn’t have enough trials to know with confidence that it wasn’t just chance that more people signed up via the page with the phone number).
Perhaps I’m misunderstanding something, though.
Nice post otherwise. I look forward to hearing how the test with the more prominent phone number goes.
Yeah, I agree. With more trials and a more prominent number the real numbers would be clearer. I also think we’re missing part of the story. If they saw which calls came from the new number on the landing page versus calls originating from other places, they could see which sign ups not only happened because of just seeing the number, but also sign ups that came about only after getting phone help from someone on their team. My company, Callcap, has a feature called Webmatch that ties together phone calls and web hits.
Very interesting test! It is hard for me to see how having a phone number on a landing could be detrimental so I am going to give this a try as well. Thanks for sharing these results!
~Michael
To be fair, the phone number wasn’t very prominent on the website.
But why stop the test (or publish the results on a blog post, for that matter) if they didn’t hit statistical significance yet. Isn’t that the whole point of actually measuring?
Sorry to be negative (I think this A/B test is interesting) but you also mentioned “liking the trend” — which you are basing on something that’s not statistically significant.. ? :P
Anyways, on a more upbeat note: Did anybody call the phone number? And who answered, etc? Maybe there was some good that came from that as well, which the test does not take into account? (just a thought)
I’ll be looking forward to part 2 !! :)
Is the .5% increase based of the the number of signups in the 6 weeks prior to the trial? If not, how did you get that figure? I’m just curious, not trying to find faults.
I’m inclined to agree with Arnor on the prominence of the phone number. I think this is one of the biggest factors in testing phone conversions.
Based on the conversion data (56.96% vs. 46.04%) you provided, I calculated the difference as 17.2% and not 0.5% as in your article, which is a significant difference. You either made a mistake in numbers your provided, or you need to go back to school.
Those numbers are not conversion rates.
Out of all signups, 53.96% of the signups were originated from the homepage with the phone number.
Only if each variations had the exact same amount of visitors, then your calculation was correct.
We work on this stuff since 2006 and I can assure you that a web call back button converts even more than just a phone number. A call back button is a “call to action” button that increases the probability to receive a call from the user. We made lots of tests with our clients, with landing pages containing the number to call and the call back button as alternatives and during the campaign, the call back button was used for 80% of all the calls received.
I’m a little surprised by the results – perhaps it does have to do with the size of the phone number on the page. I found that I was drawn toward the page *WITH* the phone number even though I would probably never call.
In my experience the phonenumber does mean a big difference. We display all our phonenumbers even our CEO:s direct number in large digits on the first page our site and even though we don´t recive that many phone calls many of our customers mention that they have seen our phone numbers and that it made them feel good when shopping with us.
I found the same findings.
http://lesseverything.com/blog/archives/2011/02/17/adding-a-phone-number-to-lessaccounting-increased-our-paid-user-base/
If you do put a phone number on your landing pages, and you also use phone numbers in ads, you really should do some call tracking on them to know which piece generated which calls. I’m the UX designer for a call tracking company called Callcap and we even have a feature that lets you see what page the user was looking at when they called. More metrics means better decisions. :)
Very interesting.
As a phone number supplier I’m not surprised however I would want to test various phone numbers. For example some small business just list a mobile number. Others pretend to be big by having a non geographic number (0800 for example). I think both of these put potential callers off. I refuse to sell 08xx non geo numbers these days for this reason. Be an interesting test though……