Editor’s Note: Everyone wants to improve their conversion rates right? But are “conversion rates” the right indicators for steering you towards your most important goal: increasing revenues? The case study presented here is about how FoxTranslate.com (a document translation service), found that focusing on conversion rates alone can be misleading. Read this article to discover what they learned. You’ll be surprised and it may change the way you run and think about your website tests.
As most of you know, a popular platform for A/B testing new website features and landing pages is Google Website Optimizer. Google Website Optimizer uses conversion rate (goal reached / number of visitors starting the goal funnel) to determine the winner of A/B tests.
Over the past few months FoxTranslate has been running A/B tests to improve our website conversions. As we got involved with setting up and running A/B tests, we began to bury our heads in the sand by being enveloped with the busy work of testing.
It wasn’t until we decided to make a redesigned home page variation that we started to “see the light”. After a week of tweaking and improving what we thought a better home page would be, we came up with these initial assumptions as to why our new home page variation was more effective:
We felt the new home page:
- Highlighted our core service features.
- Segmented the data into digestible sections.
- Offered a slicker and professional feel.
We expected our redesigned homepage to be the clear champion.
Below is a screenshot of our original home page:
And here is a screenshot of our home page variation:
Our new homepage showed a 6% improvement in conversion rate vs. the old homepage. While the 6% was a nice win, the improvement was not capturing the full story.
We have a diverse customer base of businesses, law firms and consumers with very different spending habits. Even within these segments, transaction sizes vary greatly. The problem with focusing on conversion rate alone is that the conversion rate metric suffers from not telling you who and how much each visitor is spending.
Hoping that the new site was attracting higher spenders, a measure not tracked by conversion rate, we started tracking each page based on the revenue generated. We accomplished this by coding custom variables into each page and importing the by-page information into Google Analytics. For more information on custom variables, check out this Google Analytics overview.
Revenue Improvement of Current vs. Redesigned Homepage
With our new revenue measure, our new homepage really shined, generating a 17% improvement vs. the current homepage on similar impressions. We previously noted the 6% improvement in order volume (conversion rate), meaning that 11% of the improvement was due to higher transaction sizes.
Our use of custom variable tagging also allowed us to compare metrics on customer engagement available in Google Analytics.
On average, customers stayed longer on our site and were less inclined to immediately exit the site. While potentially a stretch, it appears that customers who spend more tend to do more research. Based on average time on site, our new homepage appeared to better engage higher spending customers.
What We Learned
Conversion rate is golden, but revenue generated is platinum. While the improvement in conversion rate was nice, it was really revenue generated that showed the vast improvement from our new homepage design.
While conversion rate is a good metric, the metric assumes that every transaction is of the same value. Therefore, an improved conversion rate may just be masking lower revenue due to a higher concentration of lower value customers, a negative to your company.
About the Author: Jason Thai is a marketing manager for FoxTranslate, a provider of certified translation services, specializing in translation of business, legal, immigration and academic documents in over 30 different languages.