Getting Things Done: How Moving Fast Doubled Our Feature Engagement

As a SaaS business, we regularly make improvements in our software product because we care about our customers. We also want to give our customers a competitive advantage with our customer data so they can make better business decisions.

When we started working on our new version of Live two weeks ago, we had a lot of discussion about whether we should rewrite the whole thing or just improve the visual designs. I’ll dive more into that a little later, but one of the big influencers for a rewrite was that we wanted to make a dramatic improvement in reliability and uptime — one that wouldn’t be possible with just a simple design upgrade. What’s a new design worth if it doesn’t work?

Funny thing is, when we finished building Live, our customers said it was fantastic…but still there was something missing. We were not getting the engagement or adoption levels we had hoped for.

What went wrong?

Instead of going back to the drawing board, we kept it simple. We figured we’d waste time making decisions and changing a lot of things. It was going to take too long to plan everything out again. We learned that, a lot of times, all it takes is small changes here and there to get that bump in engagement.

In our case at KISSmetrics, we increased our engagement by making small alterations in design thanks to our customer-driven data. Test quicker, faster, and get more things done. Here’s our story:

Building The New Version Of Live

New KISSmetrics Live

KISSmetrics Live lets you monitor top activity trends, filter for specific activity you’re looking for with a new launch or campaign, and see information for individual customers — all in real time.

When we set out to improve our Live tab (which provides people with a real-time data stream of customer activity), we first looked to our secret sauce — customer feedback.

Thanks to our awesome customers, we were able to define a list of requirements and use cases that our Live tool needed to help customers get their jobs done better and faster.

Some key requirements included:

  • Reliability*** — Flash was causing all sorts of trouble
  • A way to drill down on specific people, events, or properties
  • A better way to view your own activity AND monitor the live stream
  • Getting into individual customer profiles more obviously

Flash was a big offender. It caused loading problems. The different versions caused different errors. It crashed. Customers were not able to see it at all because of their device. Customers lost their whole session.

Customer Feedback Visualized

The old version of Live, complete with annotations, after customer feedback was summarized by Jason, our product manager. Oof.

Before we got to any visual design improvements, we started with the back end. If our customers couldn’t use our feature, there would be no point in updating the visuals or functionality. Our engineers did the hard part by building a robust back end that didn’t depend on Flash anymore. They were able to deliver the behind-the-scenes magic that powers our new Live tab now.

With reliability improved, we could confidently move forward and implement the rest of our improvements.

Getting To Customer Needs

What do customers actually need? To help answer this, we started with sketches, mockups, and wireframes.

New Feature Sketch Wireframe

One of the earliest sketches in our design phase from Eric, our support engineer.

We wanted to work with something low fidelity to show customers’ rough user experiences so we could see if our ideas were actually helping them solve their problems. This allowed us to focus on the jobs customers were trying to get done without having colors and major layouts get in the way of the feedback. It allowed us to differentiate what they needed (ways to filter, search, etc.) from what they wanted (button colors, perfect alignment, etc.).

UI Notes from Designer

Another idea from one of our designers, Jason, with UI notes.

As a company that helps other businesses get to know their people, it was obvious to us that we needed to keep in close contact with our customers. And we did just that. After several cycles of interviews and testing, we were able to get to a point where customers agreed that they would be able to do the jobs they wanted to accomplish with our new improved tool. So we started building.

Problem Solved! Or So We Thought.

Final Wireframe Mockup

Final design that was made by our lead designer, Ian, and implemented. Huzzah!

A job well-done, everyone! Let’s move on to the next thing, we thought. Not so fast. When we launched the feature two weeks ago, our feedback box started filling up with messages.

customer feedback

Some feedback messages all about stream activity being too big. Yowza. Thanks to everyone who sent in feedback!

A lot of the initial negative feedback focused on how the stream items were so large that it was impossible to scan for customer activity. Some people even wanted to switch back because it was not valuable without the ability to scan easily! Luckily, we received positive feedback with regard to reliability, search filters, and trend monitoring. All the jobs were accomplished (yeah!), but we had a design issue to solve.

Here’s How We Got The Crazy Boost In Engagement

Crazy Boost in Engagement

Yes, this is the actual metric chart with the huge jump. Since our data is tracked on a per-person basis, we knew something great happened.

The small bump in the middle of the graph was when we originally launched the new version of Live. We saw about a 50% increase in that week alone after an email to our existing customers to check it out and use it over the course of the week where it leveled off.

After the feedback wave, we hustled. We didn’t waste any time trying to come up with a quick change that would alleviate the main problem: customers wanted to see more activity and data in the stream.

New KISSmetrics Live

What we rolled out a week later as Version 2. We increased how much activity you could see, while maintaining some Version 1 design elements on hover.

In the old design, you could see two customer activities, maybe two and half activities, in the stream. In the new design we came up with, you can see ten activities in the same resolution.

Here’s how some of our customers responded:

customer feeback via Twitter

Valuable feedback turned into fix. Thanks Pejman!

best debugging tool

Happy customer with the new fixes and improvements. Thanks Evan!

Best part? We didn’t do any additional marketing or launch emails when we implemented the new version. Our customers organically started using the feature A LOT.

So What Did We Do?

1. We started with tracking customer data. We made sure we established tracking for our Live tab with KISSmetrics. Tracking on a per-person basis steers you away from dangerous vanity metrics and makes you start analyzing the behavior of real people. One event each from a million people is very different from a million events from just one person.

We also got customer feedback data and made sure their problems were solved by our solutions.

2. With customer data, we were able to look into the whole lifecycle of our engagement patterns, all the way back to the first occurrence, as well as drill down into specific customers, if necessary. We were able to benchmark our performance to measure against dips, or in this case, gains.

3. We made small design changes according to most common customer requests. And we did it fast. It’s hard for any business to get everything right the first time. Or the second time. And so on. If you iterate quicker, you’ll learn faster about what worked, what didn’t work, and how to prioritize what’s next.

What other ways have you moved the needle on engagement for your business? Tell us in the comments!

Do you want to move fast and get things done with us? We’re looking for awesome people to join our team.

About the Author: Chuck Liu is on the KISSmetrics Product team and loves to cook in his spare time. Find him on Twitter @chuckjliu and Quora.

  1. Assuming you guys were smaller and didn’t have a lot of customers that tweet you feedback etc, what would you have done to get feedback after noticing a problem?

    • Thanks for the question, Andrew!

      When we started building the new version of Live, we had a list of customers that we identified in KISSmetrics as having used the Live version a lot more than other customers. We leveraged these customers by asking them if they could spend 10-15 minutes with us over Skype. Some of them never replied, but the ones that did had a lot of valuable feedback for us.

      Check out this post on what we think are the best ways to get feedback:

      http://blog.kissmetrics.com/best-ways-to-get-feedback/

      Hope that helps!

    • Derek P. Collins Aug 28, 2012 at 1:52 pm

      I wanted to add that most of the feedback we received around Live came from our in-app feedback boxes (we talk about them in the post Chuck linked to). They’re a great way to get feedback regardless of how many (or how few) customers you have.

  2. Great post man!
    Thanks for sharing this wonderful post,keep on posting.

  3. I would like to thanks for the great information man.

Comments are closed.

← Previous ArticleNext Article →
Buffer