Ramblings of an Amateur PM : Part 4

Lets talk about Metrics baby!

We are taking a slight detour from my ritual of anecdotal writings of personal experiences interspersed with theory and will be getting straight to the point.

Every product decision undertaken by you has to be influenced and backed up by one thing. Data.

But even more important than that is how do you know if the decisions taken by you are being validated by the kind of results that you would like to see. And even if you can notice a trend of a particular number, how would you know if that is good or bad. That is where Metrics play a big role in an PMs role, as they do in most results oriented business from a strategy point of view. It is only more exacerbated as keeping a track of those numbers is very essential for a Product manager.

A good metric is:

• Understandable — It is intuitive and not something which is hard to remember or understand as ultimately it’s the insight that you can draw from raw data matters.
• A ratio or rate — Ratios or rate are more indicative than just absolute numbers. We can know our total registered numbers stand at 5000 today, but that wont tell us anything if we cant analyse it over a time period and then compare the growth rate of our users.
• Comparative — A good metric should be comparable across time periods, or demographic. Ex. Our retention rate has increased by 10% as compared to the last quarter. Just mentioning that our retention rate is 27% doesn’t tell you a lot more than just that sentence.
• Behavior changing — -A good metric will usually give you enough feedback to make changes. It’s good practice to keep track of all vitals but the more important metrics will be the ones that will make you take note, and work on any actionable insight from its trend. Because at the end of the day if any changes in that metric trend doesn't have any effect on your business, why do you have to measure it in the first place then?

Depending on your goals, metrics are generally classified into the following categories —

Acquisition — These metrics are the ones which keep you abreast regarding the new users that you are acquiring(or at least are trying to) answering the following questions —
> How many users we have?
> What is the rate at which the user base is growing? (Total New Users)/(Duration — month,week)
> What is the source of the growth? Where are the users coming from? (SEO,App store,InBound Links, Referrals)
> What channel has the lowest customer acquisition cost — cost per customer converted?

Activation — These are the metrics which tell you about the users who are performing the activity you want them to do. It could be signing up (like most businesses), or could be downloading that free version of a software that you will charge for after an extremely short trial period (y u do this balsamiq) , or could be ordering that fancy boba tea on your app, or using a particular feature X. Activation is the aha moment wherein a visitor has the first experience with your product. Usually the following questions are answered —
> How many users performed the activity?
> What percentage of people who visited your website then signed up?
> What is the conversion rate?

Note : There is overlap between some activation and acquisition metrics(No. of people signing up) as companies with different revenue models/business may like to set/define their important metrics in different categories, but the important takeaway is conversion happens at this stage. Activation in essence is converting a visitor to a customer.

Retention — Retention metrics deal with the number of people retained as your customers i.e. the ones who come back for your product after using it once. Rule of thumb — retention is always cheaper than acquisition. The better your offering is, better the retention, lower your costs on acquiring new customers, better chance of your product being a success. The opposite of it customer churn or most commonly known as the churn rate. Churn tells you about the number of customers leaving the system i.e. who dont come back or stop using your service beyond a certain point of time.
A simple mathematical equation will tell us a lot —
Customer Acquisition Rate - Customer Churn Rate = Customer retained
Customer Acquisition Rate > Customer Churn Rate = Growth
Customer Acquisition Rate < Customer Churn Rate = Rising Acquisition Costs!
Common questions —
> What is the Customer Acquisition Cost(CAC)?
> What is the Churn rate?
> Number of users resurrected?
> Daily Active Users/ Weekly Active Users/ Monthly Active Users?

Referral — Referral is one of the most effective methods for organic and cheap growth. It’s one thing to convert a visitor to a customer, but takes a great product to covert your customer into an advocate. The good bit is you save a ton of money in acquisition costs if you could just have your customers rave to their friends how amazing your product is. That is why it is imperative to have very set referral policies that incentivize customers to refer your product to their friends and families. For ex. the cash benefit on Google Pay whenever your friend uses a referral code to sign up on the payments app.

One of the most followed metric to measure advocacy of your customers is NPS — Net Promoter Score

On a scale of 1 to 10 with people answering how likely are they to recommend your product to a friend, people who mark 1–6 are classified as detractors, and people marking 9–10 are your promoters. NPS is calculated by subtracting your detractors from promoters. Any NPS> 0 is considered good. An NPS > 50 is excellent and will be a marquee sign indicating how good your product is.

Revenue — Revenue metrics are important because monetisation of any product/app determines its longevity. Even platforms like Quora or Instagram which started as non monetary platform now have in place a very robust advertising revenue model. Though if you have optimized the ‘AARR’ of the AARRR, the incoming cash flow shouldn’t really be your greatest headache.
A good measure of your revenue model is Customer Lifetime Value or CLV. CLV is the total amount of revenue a single customer will generate for your business until he is churned out. Ultimately it’s the CLV which helps determine how much can you afford to spend on your Customer Acquisition Cost CAC, which includes cost for marketing, sales, meetings, discount coupons or whatever it takes to get your customer to convert,considering all the other costs are taken care of.
Normal sense tells you that its good to have a CLV:CAC ratio more than 1. A good practice is to try to maintain that ratio at 3:1. Hence it’s important to retain customers for as long as you can as it translates to higher CLV.
Other good metrics would be —
> Average Revenue Per User(ARPU) — Usually measured in cycles.
> Customer Supporting Cost (CSC) — Costs incurred in hiring customer service reps, complaint resolution system etc.
> Revenue growth rate

Many of the above metrics shall also be measured in cohorts i.e. breaking down your customers into groups with respect to geography, demography, location, source channels, income group etc etc to generate more actionable insights relevant to each group. The above 5 categories are known as the AARRR framework for user metrics and are used to understand the funnel of your user’s journey as they interact with your product.

Another important metric or the metric that matters in most companies is known as the North Star Metric(NSM). It is usually the metric the company believes that captures best the core value that their product delivers to customers. Focusing on and optimizing company-wide efforts to improve your NSM is key to driving long-term sustainable growth in your company.

For example — Spotify has Time Spent Streaming as their NSM as that’s what they want to concentrate on increasing. For Facebook it is Daily Active Users (DAU), which is an interesting detour from MySpace’s NSM being Total Registered Users, working on which implies focusing on the Acquisition part of the funnel more. Facebook’s NSM being DAU implies the company focuses on making the customer return to their app daily, thus focusing on engagement and retention. We see what difference it has caused in the fates of both companies (Among other factors too).

Metrics indeed are a PMs best friend, though just as with friends, it’s always important to pick the right ones.

Do check out my other stuff here