Let a 1,000 Flowers Bloom. Then Rip 999 of Them Out By The Roots

I recently read a pretty interesting piece by Peter Siebel, currently running Twitter’s Engineering Effectiveness group.

Let a 1,000 Flowers Bloom. Then Rip 999 of Them Out By The Roots

I’ve found Peter’s premise to be quite interesting:

As an industry we, for the most part, know how to scale up our software. […]We also know how to scale up our organizations, putting in the necessary management structures to allow thousands of people to work together more or less efficiently.

On the other hand, I’d argue that we don’t really yet have a good handle on how to scale that area that exists at the intersection of engineering and human organization. […] And, worse, it often seems that we don’t even understand the importance of scaling it as we go from that first line of code to thousands of engineers working on millions of lines of code.

Peter’s pieces consists of two main parts. The first part is a play-by-play history of Twitter’s code base and development methodologies which highlights the key areas where focus on “engineering effectiveness” would have helped.

The second part decomposes “engineering effectiveness” to three main areas:

  1. Reduction of tech debt first where it tends to accumulate the most (tooling) and then elsewhere in the code base
  2. Help in the dissemination of good practices (around code reviews, design docs, testing, etc.) and the reduction of bad practices
  3. Building tools which help engineers do their job better

In that second part Peter also suggests a model to determine the optimal level of investment in “engineering effectiveness” (ee):

ee-modelWhere “E” is total effectiveness (which we’re trying to maximize), “eng” is the total engineering headcount, “ee” is the engineering effectiveness group headcount, “b” is the boost that the first engineering effectiveness headcount brings to the rest of the engineering team and “s” is the scaling impact that each add’l engineering effectiveness headcount contributes (0<s<1 since we should assume diminishing returns).

Assuming b=0.02 (2% effectiveness boost) and s=0.7, for a total engineering headcount of 10, 100, 1000 and 10000, he gets an optimal ee headcount of 0, 2, 255, and 3773 respectively.   As the engineering org scales, a larger portion of the total headcount should be dedicated to making the rest of the engineering org more effective, with ~100 engineers being the inflection point of making the investment worthwhile (for these b and s values).

Another important aspect here is in providing guidance on the type of initiatives that such organizations should take on: breadth very quickly trumps depth – making 1000 engineers 2% more effective, has a much greater overall impact, than making 10 engineers 50% more effective.

This model is particularly interesting since it can easily be generalized for any other group whose mission is to help a larger part of the org be more effective. These support groups, in companies that are wise enough to have them,  tend to be staffed and funded based on a fixed headcount ratio to the total headcount of the org they support. Peter’s analysis suggests that when those organizations scale significantly, the traditional approach will lead to under-investment. Adopting this more refined methodology and having a thoughtful conversation about the appropriate “s” and “b” values for the particular use case, will likely lead to a better outcome.

 

Advertisements
Let a 1,000 Flowers Bloom. Then Rip 999 of Them Out By The Roots

The Cultural Manifesto

A recent piece by First Round Review recaps a talk Kevin Scott, LinkedIn’s SVP of Engineering and Operations gave at their recent CTO Summit:

How I Structured Engineering Teams at AdMob and LinkedIn for Success

There are many good nuggets of content in the piece, but at its core is an artifact referred to as the Cultural Manifesto: “a document or a set of materials to help your entire engineering team get on the same page about how you make and operate things, and how you function as a team”.

Not surprisingly, it has three main sections covering the organization concrete, documented opinions in three main areas:

  1. How we make things – code reviews, branching, deploying code, enabling features, etc.
  2. How we operate things – monitoring & alerting, production access, data integrity, etc.
  3. How we function as a team – role of engineering, culture and values, conflict resolution, etc.

cultural manifesto

 

It’s a phenomenal starting-point checklist to go through. If you don’t have a concrete, documented opinion on one of these line items – you probably should.

“Don’t let your CFO start capitalizing R&D labor” is my own personal favorite 🙂

The Cultural Manifesto

No Deadlines for You!

Continuing my short series of engineering-oriented posts, this lovely blog post by Dan Milstein of Hut 8 Labs has been leading my secret “most mis-titled blog posts” chart for quite a while now:

No Deadlines for You

This post has very little to do with deadlines and a lot to do with describing the behavior that sets apart senior engineers from junior engineers, in my opinion.

Dan argues that any definitive answer to the question:  “can your team get spec X done in Y months?”  is the wrong answer. The path to the right answer has to start by responding with a question of you own: “Why?”. This is an over-simplification, of course.

What Dan is really advocating for, is to start by developing a good enough understanding of the underlying business problem that led to the request in the first place.  Then, we have enough business context to figure out what work we really need to do in order to address it. And finally, we can present a realistic plan that takes uncertainty into account.

Trying to summarize Dan’s post in a single paragraph really doesn’t do it justice. Reading his entire post WILL be a good use of your time.

No Deadlines for You!

Speed in Software Development

As promised, something completely different.

Michael Dubakov, founder of TargetProcess wrote a pretty thoughtful post a couple of months ago called: Speed in Software Development.

It starts with a debate about sustainable development speed and a neat idea about structuring iterations more like interval training. But the part that I really liked is his bold attempt to map out the interactions between all the factors that ultimately affect development speed:

map2

It’s followed by a thoughtful discussion of the different factors and some actionable ideas on how to address them.

I think Michael can potentially be on to something very interesting here, but it also left me with two big, unanswered questions which I’ll address to you:

1. Is development speed really the ultimate goal/metric? Michael incorporated into his framework some of the typical contenders like quality, complexity, tech debt, etc. But is it really as simple as that?

2. How can it best be measured in a way that’ll encourage the right set of behaviors?

 

Speed in Software Development