To me, "lean" software engineering means the following:
It produces a lean product, which does what it is supposed to do very well. It is designed for change to meet any new or changed requirements. It is composed from a set of cleanly designed modules which are well-separated in their design and implementations at all layers. There is no unnecessary complexity.
It uses a lean process. This is a more controversial issue; many managers would readily agree on the lean product, but they have difficulty agreeing on the lean process. They immediately associate a lean-process with extreme-programming (XP). And that means chaos to many managers who grew up with traditional waterfall models. I am a proponent of a "lean" process, but not XP, for that XP has its serious flaws and I do not subscribe to all of its tenets. That is a whole another discussion altogether.
The choice of development model should depend on the nature of the project. If time-to-market is an issue and if we have 6 months to deliver a fairly complex product, we know we cannot go through a traditional waterfall process. It doesn't mean that we should not follow some of what it specifies. We do need to start with requirements gathering, but what we cannot do is to sit around for 3 months gathering all nitty-gritty requirements, with the hope that we can get them all. We will need a much more agile and iterative process, which means rapid development, validation, enhancements, leading to final product delivery. On the other hand, if we are building a software that goes into a space shuttle, we may need much more structured waterfall / RUP process.
Finally, it employs a lean project team. Assign as many resources as needed, but not more. Identify right roles for every member and empower them by providing whatever resources they need to perform their tasks. Remember that men and months are not interchangeable (read the classic Mythical Man Month by Frederick Brooks); throwing extra resources blindly at a failing project will only make it worse.
Recently I was talking to the VP of Engineering of a 5-yr old "start-up" where nothing seems to go well. Even relatively simple projects have failed, and the prdictability is at all time low. After understanding the overall picture, I came to the conclusion that it all had to do with the integrity - integrity of the whole system that he is managing or trying to manage.
First, there is product integrity. This includes both technical integrity of the product as well as functional integrity. The product is very poorly designed, reusability is low, and has too many critical design flaws to enumerate. If complexity to functionality is a metric for software, this product would rank at the bottom 10%. In software, a design flaw could mean 50% more work just to circumvent it. Imagine driving a car where you hear parts rattling, wheels don't cleanly align with the road, and the steering is shaky.
Second, there is process integrity. The development process itself depends on the product being developed. Throw in a badly designed product and a bad architecture to work with, the process will be much more complex than is usually necessary. Make the process complex, and the managing that process becomes a huge overhead. As a result of increased complexity, the likelihood of process failure increases. All of a sudden, it turns into a situation where no one knows what to do next, they blame one another, and eventually leads to complete failure of the project.
Third, there is organizational integrity. In this specific case, managers weren't well aligned to the same organizational goals, which created conflict, and the engineers working for them couldn't work together. The organization had very good talent, but the talent was underutilized by inappropriate assignment of roles and responsibilities. In my view, a well designed organization is probably more important than everything else.
There is nothing new here, but losing integrity is completely antithetical to lean thinking. If you want to remain lean (and profitable), never lose sight of managing integrity, even if that means short-term pain.
This book on lean software spells out the exact mode of software development I prefer. I cannot emphasize enough on the basic principles of lean thinking proposed in this book.
Iterating towards excellence: software development as an exercise in discovery
Managing uncertainty: "decide as late as possible" by building change into the system.
Compressing the value stream: rapid development, feedback, and improvement
Empowering teams and individuals without compromising coordination
Software with integrity: promoting coherence, usability, fitness, maintainability, and adaptability
How to "see the whole"— even when your developers are scattered across multiple locations and contractors
Many managers have problems with rapid development-feedback-improvement iterations and delayed decision making to manage uncertainty. With respect to empowering teams and individuals, I have witnessed managers turning extraordinary engineers into average engineers by simply enforcing fat software engineering processes that completely weaken them.
XML has become ubiquitous, and it has become so for many good reasons. But, we can all find many cases of XML overdose as well. I have seen two modules in the same Java application exchanging data by passing complex XML documents, which are then translated into application objects, after going through DOM trees. And, if the developers are any more tool-lovers, they would use JAXB, Castor, and any other technology that's available to move data between XML and Java and vice versa. This all makes the application overly complex for no good reason.
Looking at a piece of raw BPEL code in XML syntax makes me cringe (especially the dataflow aspects). Some claim that, with BPEL, we can write applications without "coding" - we certainly don't need to code in Java when we can do that in XML. :-)
A colleague questioned my sanity after reading my rather insensitive comments on UML. How can I not like "models" and "model-driven architectures"? I spent a good part of my life building models for quite complex things, so I certainly don't dislike 'models' - they saved me more than once.
It is not that I dislike UML either. Any attempt to bring in some structure to software design / development process is a worthwhile attempt. I think UML offers a good diagramming tool for conveying one's design, and when it is used for that purpose, it serves well. However, a diagramming tool is different from a 'modeling' tool. In my opinion, trying to model a piece of software as very-detailed UML diagrams is a waste of time, because those models don't serve much purpose (law of diminishing returns). View them as diagrams for shared understanding between humans, if at all, and not between human and computer. Spend just as much time necessary for that purpose, not a second more.
Remember, when it comes to building a software application, a quick prototype is worth a million UML diagrams. A prototype is indeed a rough model of the final software.
