Curated Content May 2023
A few pieces of content I thought were worthwhile in the month of May.
Articles
Goodhart's Law Isn't as Useful as You Might Think
Cedric Chin
Commoncog is consistently one of my favorite places for content on the internet. If you're not subscribed there, go ahead and do yourself a favor and subscribe there.
This one focuses on Goodhart's Law and process improvement. The key highlights I see in it are the following (emphasis mine):
Broadly speaking, Amazon divides its metrics into ‘controllable input metrics’ and ‘output metrics’. Output metrics are not typically discussed in detail, because there is no way of directly influencing them. (Yes, Amazon leaders understand that they are evaluated based on their output metrics, but they recognise these are lagging indicators and are not directly actionable). Instead, the majority of discussions during WBR meetings focus on exceptions and trends in controllable input metrics. In other words, a metrics owner is expected to explain abnormal variation or a worrying trend (slowing growth rate, say, or if a metric is lagging behind target) — and is expected to announce “nothing to see here” if the metric is within normal variance and on track to hit target. In the latter case, the entire room glances at the metric for a second, and then moves on to the next graph.
How do you come up with the right set of controllable input metrics? The short answer is that you do so by trial and error.
And from earlier in the article:
Most of us, when faced with a goal, will fixate on the difference between the current output and our desired target. In other words, we think the way to hit our goals it to obsess over the goal. A common approach is to work backwards to figure out targets at specific checkpoints, and then say (either to ourselves or to our team) “HIT THOSE TARGETS, COME HELL OR HIGH WATER!”
...
This is a long winded way of saying that if you want to improve some process, you have to ignore the goal first, in favour of examining the process itself. On the face of it, this is common sense: you cannot expect your team to improve some metric if that metric isn’t directly controllable. No, you must first work out what set of controllable input metrics leads to the output metrics outcomes you desire, before you can even begin to talk about hitting targets. You’ll need to figure out what levers to pull in order to hit 10,000 units a month; you’ll need to figure out what drivers exist before you push for 100 new deals a quarter. The way you get to this state is nothing at all like obsessively watching your target and measuring how far off you are from it and yelling at your team about the underperformance — down that path lies Goodhart’s Law.
All of this is key and is part of what I was referring to You're Encouraging Your Team to Lie To You.
Ontology vs. Requirements
An article from Jimmy Koppel. This one is on philosophy of software design, and reflecting on how the definition of a given word can change it's definition.
However, the important thing is not the meaning of the world and ensuring it reflects it's relationship to the real world in a high fidelity way, but instead it's about the definition of the word relates back to your system's needs for the world and modeling it appropriately to facilitate that.
There is no Truth in Business, Only Knowledge
Cedric Chin
Cedric Chin of Commoncog with a number of excellent articles this month. (To be clear, I think he's one of a handful of folks worth continuously reading, but these two stood out for me.)
This one combines two of the more interesting business writers in my view, the first being Edward Demming, with his theory of profound knowledge, and then Clayton Christensen, with his Innovator's Dilemma, and how those two things come together in a really interesting way.
In that business is a domain where truth isn't some immutable thing, it's incredibly context dependent. You'll likely find some things that are slow to change or unlikely to, which are as close to truths as you'll get, but Demming's idea of the process being the link to the truth fits in nicely with this to me.
Large Limits to Software Estimation
This article by J. P. Lewis out of Disney's TSL, dating back to 2001, puts forward something that is important with an important caveat.
You can't develop actually interesting or novel software on a timeline, (and in my view anything that isn't interesting or novel ought to be a library).
Now this loops in the problem of terminal uniqueness in tech, but even if the problem has been generally solved before you still have the problem of integrating it into your own socio-technical system.
The Iron Imperative of writing: don't waste my time
Josh Bernoff
Very important when applied to writing, but I think this is just as important, but often not considered at all in the context of software engineering.
The Iron Imperative states that writers should treat the reader’s time as more valuable than their own. Applying this to software engineering allows you to take more of a forward looking view to yourself, and to future maintainers of the software, as well as reviewers of the code.
Books
Start With No - Jim Camp
In many ways was better to me than Never Split the Difference. In Start with No Jim starts by systematically explaining how win-win negotiation can be broken down when each party isn't operating from that shared framework, and then how to achieve a similar "win-win" outcome with less possibility of being taken advantage of.
Conf Talks
No conference talks made the list for me during May.
Podcasts
Lesley Sim on Skill Acceleration in Ultimate
Cedric Chin
At this point this month's update runs the risk of being a Commoncog fan letter, but Cedric puts out a ton of great content, and in this case his conversation here with Lesley Sim is incredibly interesting.
Lesley discusses different approaches to teaching, and ensuring that folks are learning the right lessons, along with the need for a defined pedagogy. She was also the inspiration for Cedric's post on playing to win versus playing to play, which is one of the best articles I've read, and strongly recommend to folks of all walks of life.
Tweets
another possibility is that they get it just fine...but incentives rule the day.
— John Cutler (@johncutlefish) May 3, 2023
GOODNESS what if you CAN'T figure out who completed that project?
What if people don't know what they will be working on in 6 months?
I think about this one often as it's problem that doesn't just exist in big tech. A lot of folks take comfort in the idea that you know what you'll be working on in 6 months, when for many organizations regardless of size, that's just not the case. It should be varying, and that the team is the unit that is delivering the work as part of the greater organization. Not the individual.