Product Onions

note: I read a thread on Twitter the other day that was like solving a delicious product puzzle - each response led to more questions, like peeling back layers of an onion - so trying something different with today’s post and sharing my inner dialogue as I read the thread

---

Jason Knight 💔🇺🇦 @onejasonknight

#prodmgmt pop quiz! You have an API product providing data to your customers. They do "stuff" with that data but you don't get to see what happens for the ultimate end user. Leadership's only benchmark is revenue. What's your north star metric?

I managed an API product where leadership’s only benchmark was revenue! And on my first day on the job everyone was trying to figure out the north star metric! Is this guy reading my mind?!

Matthew Steer @otteradrift

@onejasonknight Hm. Your API exists so you can provide some kind of service. That service meets a need. You say "...providing data to your customers". I can't answer your question if I don't know what data. Because that data is the user need you are meeting. The north star metric is in there.

LOL. This guy is bringing product thinking to a revenue party. Your customer does “stuff”. “You don’t get to see what happens for the ultimate end users” - the constraint is spelled out in the question. It’s B2B2C. Maybe even B2D2C (your customer is a developer business like an ISV or SI). Don’t get me wrong, it’s the right instinct to try to connect the dots to the end user workflow, but this is Twitter - the answer is going to be short and sweet, not “do user research”.

Scott D. Witt @scottdwitt

@onejasonknight Followup Question: Why is Revenue leadership's only benchmark? And why is that? (5whys) Obviously Revs are important to the biz, but it may be unreliable b/c it's a function of the Pricing, which may also be faulty.

hahahahahaha - in B2B products, you have to connect to revenue (generation or preservation)

mirza besirovic 🇺🇦 @miryabesirovix

@onejasonknight It would have to be API calls, but we'd need to go granular to draw a line to a meaningful conclusion. What kind of call? What time of day? What customer? Verticals? Etc. Also, while leadership may not care about end users, I'd still go out and find a way to talk to them

We’re now assuming that API usage connects to revenue, but we haven’t confirmed that. I love, love, love “I’d still go out and find a way to talk to end users”, but you have to talk to your internal sales team first (probably quicker too).

Andrew Chant @chantmandrew

@onejasonknight My north star metric could be the number of daily active users, number of signups and retention and I'd also like to determine the customer lifetime value.

one metric! the quiz asked for a north star metric - this is a mix of leading indicators, lagging outcomes, and tension metrics

𝐃𝐚𝐧𝐢𝐤𝐚 🇪🇺🇺🇦 @dcerecedo

@onejasonknight How does the API make revenue? Is it requests, clients, other? NS can be revenue but you could use a proxy metric based on how your business model generates it. i.e. requests: measure daily reqests and you get daily feedback on how you will deliver revenue end of month.

Now we’re getting somewhere. If revenue is the goal, we need to understand the product usage → revenue link. And I love the idea of a proxy metric, because the product team can control product activation / adoption / engagement, but actual revenue is the result of many other factors beyond the product team’s sphere of influence. I was surprised no one said “revenue” is the north star - it makes a certain sort of sense, but would completely demoralize and gut the product team over the long-term.

Jason Knight 💔🇺🇦 @onejasonknight

@dcerecedo Let's assume the product is sold on a 2 year contract basis, unlimited usage

The next layer of the onion is revealed…Now we know usage has nothing to do with revenue.

Aya @ayaproductnj

@onejasonknight Time to first successful API call (reflects onboarding, documentation, support), uptime

Nice try, but no. Since we don’t care if the product is actually used, trying to optimize the activation journey is pointless. Note: I’m not saying it’s bad thinking, I’m saying this is a very constrained product problem and you can’t just redefine the parameters based on how you like to operate.

frostybyte @0xbad1dead

@onejasonknight If revenue is benchmarked and it’s an api metric the norstart metric has to be usage as pricing is usage based typically for api products

I’m pretty sure he said unlimited usage. How API products are “typically” priced has no bearing here. Many, many, many PMs can’t get over how things are “supposed to be”. Play the hand your dealt.

Andy Nyberg @AndyNybs

@onejasonknight I feel like you have to get to an engagement metric that short cuts the 2 year contract period - churn maybe? - and couple that with something qual like CSat.

Ok, I like the attempt to break out of the Matrix. Something revenue-ish (churn) to meet leadership where they are. But churn is not a great metric because it’s laggy. It’s about business health, not product focus.

Mike Chowla @mchowla

@onejasonknight I don’t see a good metric. API calls - making the API more efficient for devs may take 3 calls and turn it into 1. Metric goes down but value goes up Customers using - maybe the customers are only using it because of missing core product functionality and API is a workaround

I’ve seen it before - PMs just throwing up their hands and saying “no good metric”. It’s really, really hard to connect a product team’s work clearly to a top-level outcome (like revenue).

Dara Alsulayman @DAlsulayman

@onejasonknight Since the product offers unlimited usage and a 2 year contract I would potentially look at a function of acquisition and retention so something like a Quick Ratio (New+Resurrected)/Churned

So, let’s make the metric complicated and hope people don’t notice it doesn’t correlate to revenue generation / preservation?

Kirsten Minshall @KirstenMinshall

@onejasonknight Why wouldn't leadership be interested in the value the API provides to end users? Just because it's abstracted by a third party doesn't mean the value doesn't exist...

LOL. So much LOL!

Jason Knight 💔🇺🇦 @onejasonknight

@KirstenMinshall Let's assume they're not uninterested, but consider it difficult to measure and are using revenue ( / retention) as a proxy - which is fair enough but they have 2 year contracts so difficult to move that with shorter term product initiatives

OK, stop LOL-ing. Leadership is a hard job. Sometimes you’re painted in a corner with constraints the team can’t see / understand. The center of the onion is very pungent - not everyone can see / smell it without crying…

Sneha Shirsat @sneharshirsat

@onejasonknight 1. Daily GET requests to the API with 200 status 2. Daily volume of data delivered to users Idea is to see a consistent (WoW, MoM) increase in these metrics - since it translates to revenue and user retention

narrator: it doesn’t

Saeed Khan @saeedwkhan

@onejasonknight I'm not a big fan of a (single) North Star metric, b/c they are very contextually dependent & picking a poor one (even though you think it may be a good one) can lead to problems. But...for an NSM, focus on a customer value focused metric, not an internal operational one.

when in doubt, try to change the rules of the puzzle…

Sebastian Sandoval Similä @sebastian_j_ss

@onejasonknight I would look at unique users per day over unique requests, especially as sometimes the high number of requests can be an indication of an inefficient integration or endpoint. In fact a new endpoint can decrease usage of old ones, which would be a win

ever been in a roadmap review where the team insists that what they’re doing matters and leadership is like what on earth are you talking about?

Joe Blair @joeblairkey

@onejasonknight I'm gonna go with DAU, or whatever cycle fits the use case. Basically active engagement/retention.

yes, do what you know vs solve the puzzle ¯\_(ツ)_/¯

Sneha Shirsat @sneharshirsat

@onejasonknight 1. Daily GET requests to the API with 200 status 2. Daily volume of data delivered to users Idea is to see a consistent (WoW, MoM) increase in these metrics - since it translates to revenue and user retention

does it though? translate to revenue I mean…?

Xianhang Zhang @shalmanese

@onejasonknight What is your business model, under what foundational hypotheses did you arrive at your business model and how amenable is leadership to changing the business model?

change the business model! that’s definitely the answer - leadership will love that - and I’m sure totally do-able in 1 quarter…

Santoshi Nakamoto @blakraven7

@onejasonknight Btw it’s a very poorly constructed question. You should think and clear your head more before you post pop quizzes

and now we’re name calling…

🌻 christian crumlish @mediajunkie

@onejasonknight revenue

there you go, someone finally just said it: “revenue”…I don’t like it, but it is a valid answer to the puzzle…

ibrahim @ibscribe

@onejasonknight I say uptime is your north star - given the revenue focus, unlimited quota, and lack of visibility into the ultimate use case, the best thing you can do is prevent service interruption (which would disrupt revenue / breach contract) - utilization metrics are a res herring here

I have the advantage of having solved this puzzle before - in total honestly, everything everyone said I tried when I was peeling the onion myself - it’s much easier in hindsight / on Twitter

Atul @avaid82

@onejasonknight My 2c : Availability availability and availability of the API (it should be up and running and scalable as usage grows)

credit where credit is due, I wasn’t the only one with the “right” answer (it’s only “right” for right now)

Sagar Karvir @S_Karvir

@onejasonknight API performance and availability. But would be good to get involved in the discussion with feature team to understand overall business ask and impact.

and yes, deal with the constraints you have, but work towards a better model - that’s the job of a PM

---

As always, I’d also love to hear from readers about their attempts to peel product onions - please chime in via comments👇. And if you enjoyed this post, please consider subscribing.

---

further reading / references

• ISVs (Independent Software Vendors) and SIs (System Integrators) are common players in developer product ecosystems

• some more thoughts on how teams get tagged with South Star Metrics and the Metrics Malfunction behind this organizational anti-pattern

• if you like poking fun at execs / environments that push product teams into short-term revenue-oriented thinking vs long-term product-leverage, you might enjoy this satirical essay: The Effective Executive

---

childish drawing / interpretation