Metrics, Metrics: Measure to optimize

You can only manage what you can measure. You’ve probably heard this old saying before, and it is widely applicable when it comes to product management. You need to choose the right metrics to look at and have a good understanding of them in order to be able to optimize every single stage of your product development process. This is what the MVP Academy Batch of 2016 learnt this week in a workshop delivered by Bogdan Ripa, Director of Product at Vector Watch & MVP Academy Coach.

Entrepreneur at heart, Bogdan has Co-Founded his first startup when he was only 21 years old. He started by doing outsourcing work (consultancy for websites, web apps, ERP systems, analytics tools), but he got bored by doing repetitive things and decided to make a shift and start building his own products. 6 years later, the company he Co-Founded was bought by Adobe Romania, where he worked in engineering and product management until last year. Bogdan is an experienced product manager that enjoys creating products to help people deliver excellent online experiences for their customers. He is now Director of Product for Vector Watch, while offering hands-on product support to the startups that are currently accelerated in MVP Academy Batch of 2016.

But let’s get to the point! You’ve got an MVP and you’re committed to turn it into a full-blown product, scale and go big. You’ve probably already started collecting lots of qualitative and quantitative data, but the real question here is what do you actually do with it and how can you leverage it to guide your product development and optimize your processes. So where do you start?

First steps first: define your conversion funnel

The first thing to do, before actually choosing the metrics to look at, is to carefully define your conversion funnel (that is the stages your users pass through, from the first interaction they have with your product to the moment when they actually purchase and turn into your customers). The difficult part here is that there is no generally applicable model of conversion funnel and you have to understand your users’ journey to define the funnel that’s adequate for you (a couple of in-depth examples are provided in the embedded SlideShare presentation that Bogdan kindly shared with us).

“You can optimize your user experience at every level of your funnel, so it’s very important to clearly define it. However, limit the number of levels you include, otherwise it will be very hard to measure and optimize every single level”, recommends Bogdan.

Pirate Metrics - Funnel

Choose your metrics 

Once you’ve got your conversion funnel straight, you should proceed to choosing the right metrics to analyze. Closely monitoring these metrics will help you optimize your product development process based on the information provided by data.

Here are a couple of metrics you should take into consideration when making product decisions:

  • Monthly Active Users (MAU): number of users that are actively using your product over a period of one month (you can also take a look at DAU – daily active users, WAU – weekly active users, or YAU – yearly active users)
  • Churn rate: the percentage of customers that you lose over a specified period of time
  • Retention rate: this is the opposite of churn, reflecting the number of users that you keep over a specified period of time
  • Average lifetime of a customer: the average period of time that a customer keeps using your product / being active in your application
  • Net promoter score (NPS): indicator that measures customer satisfaction and the likelihood that your users will recommend your product to others
  • Virality factor: number of free users that starts using your product for one paid user. A virality factor higher than 1 will show exponential growth and will help you get the graphs investors want to see
  • Customer acquisition cost (CAC): how much does it cost to bring on board one more user
  • Average Revenue per User (ARPU): how much revenue every user brings you

For more information on how can you compute and interpret these indicators, check out Bogdan’s presentation (also embedded at the end of this post). And bear in mind that besides all these metrics, there are several other industry-specific indicators that you should consider in order to understand where you’re standing right now and how can you constantly improve your product and user experience.

Understand the data

Measuring the right indicators is not enough! You should have an in-depth understanding of their actual meaning and you should constantly test and analyze the results for being able to constantly optimize your processes. You can do so by using:

  • Landing pages: standalone pages that you design for a specific purpose (in this case for testing & validating your assumptions) with the main goal of generating leads that you’ll further push through your conversion funnel
  • Cohort analysis: analyzing the user behavior of different cohorts (clusters of users computed on a daily, weekly, or monthly basis, generally situated in the upper part of your funnel – the ones that have just had your first interaction with your product)
  • A/B Testing: comparing two different versions of the same feature in order to understand which one performs better for your target market

Test, test, test and constantly analyze the results to get an in-depth understanding of how you can better solve the problems of your users.

What tools to use?

There are several tools you can use to help you out along this process, starting from the very basic Microsoft Excel, that you are already familiar with us, and going up to several automated ones such as Google Analytics, MixPanel, KissMetrics or LaunchRock, to name just a few. Check out Bogdan’s presentation for a more comprehensive comparison between the tools that you have at your disposal.

Screen Shot 2015-04-09 at 13.28.02

“The tools you use for analytics are your personal choice. However, remember that their complexity has to evolve, as your understanding of the product grows”, recommends Bogdan.

Choosing the right metrics to analyze and understanding them to further optimize your product are not easy tasks for early stage entrepreneurs. Talking with experienced mentors and product guys can help you go smoother through the process, and we’re happy that Bogdan joined us and helped us shed some light on how startups should approach metrics & product development.

Interested in finding out more on the topic? Here are some insightful articles that you should take a look at:

The MVP Academy accelerator continues so stay tuned: we’ll get back with insights from our workshops and sneak peeks into mentoring, pitch practice sessions, and the dedicated activities we’re going to organize in the next weeks!

Add your comment