When metrics go wrong

  • “TELL ME HOW YOU WILL MEASURE ME, AND THEN I WILL TELL YOU HOW I WILL BEHAVE. IF YOU MEASURE ME IN AN ILLOGICAL WAY, DON’T COMPLAIN ABOUT ILLOGICAL BEHAVIOR.” – ELI GOLDRATT
  • Dee Hock - the tyranny of measurement: “ If we were to set out to deliberately design an efficient system for the methodical destruction of community, we could not do better than our present efforts to monetize all value, mechanize all societal organizations and reduce life to the tyranny of mathematical measurement, markets and the ever increasing centralization of power and wealth that result. Money, mechanism, measurement and markets have their place. They are useful tools indeed. We should use them carefully for beneficent ends. But useful tools are all they are. They do not deserve the deification the apostles of unrestrained acquisition insist that we give them. Only fools worship their tools.”
  • Some things are really hard to measure - eg the impact of saying good morning to people - and yet we can still all agree that it’s worth doing (this came from a conversation with Tito Sarrionandia but I can’t remember whether he was quoting anyone).

Accelerate Metrics

  • The four key metrics described in the “Accelerate” book are:
    • Lead time for changes
    • Deployment frequency
    • Time to restore service
    • Change failure rate (“a measure of how often deployment failures occur in production that require immediate remedy (particularity, rollbacks).”)
  • “Focusing on only these metrics … empower[s] organizations by having objective measures of determining if the changes they’re making have an actual impact”
    • The Accelerate book was very much focused on DevOps (infrastructure and deployment) and these metrics help you to focus on how good your infrastructure and deployment pipelines are.
    • More here