Semantic Werks

Lean software development is an attempt to take ideas from lean manufacturing, popularized by the Toyota Production System, and apply them to software development. Here are a few of my thoughts after skimming the surface of the field.

Lean optimizes a system to eliminate waste. Waste in software production is undeveloped features, stories, requirements. If you have a backlog of requests, or user stories, then you have waste.

A corollary is that ‘time is money’. A lean software team seeks to optimize development to reduce the Lead Time, or time it takes to move a work item from idea to implemented (tested, accepted) feature.

A lean organization will have access to useful quantitative measurement tools, like cumulative flow diagrams. It is important to always understand how the process is moving: is the work item backlog too large? Where are the bottlenecks?

Lean sofware is built with a battery of tests. You cannot reliably develop software without testing. Constant refactoring is necessary to reduce complexity. My feeling is that this complexity is largely accidental, e.g. poor library choice, bad implementations, rather than essential. Presumably the essential complexity cannot be ‘refactored’ away (unless you are a rabid NIH fanboy).

There are several workflows that might help this process. Corey Ladas has an interesting set of posts on conceptualizing workflows as petri nets.

Limiting work in progress (WIP) can be a useful way to enforce lean principles (e.g., focus on lead time rather than feature completeness). One way to do this is to use a signalling mechanism, popularized as a Kanban board, that indicates when a work item can be pulled from the previous work phase.

Lean is not waterfall. Waterfall is the antithesis of lean: plan and estimate everything ahead of time. The problem for  people new to lean is that the lean process diagram looks very waterfall, in that it is a sequential series of steps. However, iteration and incrementalism are at the heart of lean. We don’t pretend to finish the entire product, but rather, a useable portion of it. If that portion is not to order, the appropriate work item goes back in the queue. Because everything is sequential at some level of granularity (a byproduct of the arrow of time!).

Estimation is evil and forces premature commitment; it is often treated by clients as set in stone guarantee, while the developers see it as a rough, and almost certainly changeable, guideline.

A challenge in a lean system is how to decide what subtasks constitute a work item. There is a time cost involved in assessing what a particular story demands (architecture, infrastructure, human resource). I’ve often seen this glossed over in discussions about lean or kanban, as the authors seem to assume that one has a useful backlog on hand.

Since time is the paramount cost, it often makes sense to apply what the Poppendiecks refer to as ‘set-based’ design: develop several competing options simulateneously, until circumstances force a choice. While there are other costs involved, the opportunity benefit from having the ‘right’ option available is typically higher.This is also known as deferring a choice until the ‘last responsible moment’ or real options theory.

The biggest risk in a lean initiative is that the organization will not engage completely with the dramatic change in thinking lean requires. Lean is not about tools or artifacts, it requires a paradigmatic shift in software development.

I still would like to see some studies on when lean isn’t applicable or useful. Can it work with the JPL processes? Embedded software?

A related question concerns how to teach lean in undergraduate software courses. At the University of Toronto, we’ve tried to introduce agile techniques, typically Scrum sprints, to mixed results (in my opinion). The cycle times and project sizes seem too small to make cowboy coding impossible. This is subject matter for another post, however.

Some references:

• the blog Lean Software Engineering
• David Anderson: agile management blog and book
• Derek Bailey’s article on the Theory of Constraints in software
• Implementing Lean Software Development by Mary and Tom Poppendieck
• #kanban on Twitter

Having just read a book on Toyota Production System (TPS), and following various developments in lean software development, I was curious to know what Toyota itself was doing with respect to software. Software forms a large part of new vehicles (I’ve heard figures like 50% of new car development costs), so this seems like a good place to use notions like continuous improvement, visual management, waste reduction, etc.

I gather as part of a “Lean” tour of Japan, various Lean software thought-leaders (Mary Poppendieck, among others) had occasion to meet with the head of Toyota’s software division. From the description of the meeting, it sounded like Toyota were doing fairly standard BRUF development — analyze, plan, implement, test, deliver. Some people call this ‘waterfall’ but personally, I don’t think that term has ever been well-defined.  Certainly I doubt anyone has ever said, “Hey, let’s do waterfall on this project!”

So from the initial description, fairly standard development, but, being Toyota, there’s more to the story. Mary Poppendieck elaborated on her understanding, suggesting that in the process control domain, the three month cycle is quite fast (close to some Scrum sprint lengths of 1 month, anyway). There is also some interesting follow-up discussion here and here. The takeaway seems to be that ‘context is everything’.  The other takeaway might be “Toyota isn’t perfect”.

via @agilemanager and @alinahsu

I just watched an excellent presentation by David Anderson at the Agile 08 conference. He talked about moving agile to more complex, enterprise-scale projects, and how the current agile practices work.

My big take-away was his characterization of requirements as ‘perishable’. In his view, unmet requirements are like unsold inventory, essentially liabilities until they can be turned into working systems. He borrows from the just-in-time philosophy of Lean manufacturing to emphasize how unsatisfied requirements ought to be taken off the shelf and implemented as soon as possible.

In his view, future software/system architecture decisions will be much higher-level, business-value decisions. He mentioned software product lines as an example: the architect is someone who can separate product variability and encapsulate common behaviour, rather than over-specifying lower-level designs. There will be an increased need for ‘customer intimacy’ to generate unique systems that capture a core competency or competitive advantage.

There’s a lot more in this talk, including some interesting ideas on community evolution, so it’s well worth the time to watch.