Thursday, March 6, 2008

Nuts & Bolts & Broken Things

During WWII, the US designed, built and fielded large a wide variety of aircraft in remarkably short timelines... and did so without sophisticated computer models or much experience in building planes. These days, it's taken a quarter of a century to field the F-22... and we've got all these great computer tools and mountains of experience. You'd think we'd be getting faster, but we're not.

There's a reason.

It turns out that computer models and an established record of success actually slow things down and make them more expensive. They also decrease the likelihood of making a significant improvement. Why is that? Because they a) decrease our tolerance of failure and b) remove us from reality. Put them together and you've got an environment that is hostile to learning.

See, when airplanes were new, we were in a fail-fail-fail-succeed mode. We experimented, tried things that didn't work, and ultimately got something in the air. But once we had some success under our belts, we're out of that mode and became hesitant to experiment, because experimentation leads to (near-term) failures. And there's something unpalatable about the fail-succeed-fail mode. We look at one success and think it should be followed with another. So do our bosses. So we throttle back on trying things where the outcome is unknown. But learning requires failure - and fewer failures mean less learning. Ultimately, hostility to failure is hostility to learning.

And computer models? They are safe places to fail, but they remove us from reality. As a rocket scientist pointed out in the latest issue of NASA's ASK journal, "In the model, pipes never leak. In reality, they always do." So models don't teach us as much as we think they do. Combine the distaste for failure with a preference for incomplete, potentially misleading models and you end up with an aircraft that takes 25 or more years to develop.

Sure, there are other factors at play... and models are useful... but if we want to really break new ground, we've got to be out in the real world of nuts and bolts and broken things. We've got to actually learn...


Mark said...

Great post - I agree 100%!

A related aspect is when development processes are focused on paperware - requirements documents, feature definitions, specifications, phase review checklists, etc, etc. Tons of work, but it is all before anything is actually built. Like you said, very detached from reality.

And the ironic thing, as you also pointed out, it that these are all intended to get faster successes. Sometimes, you just end up with suc(k)...

The Dan Ward said...

A 100% agreement from Mark!?! Whoa - I think that's a first! :)

I love the "paperware" concept. I'll definitely have to use that. One of the things they kept reminding us in my software class last quarter was: "The point of doing all this is to produce working code!"

Funny how we need to be reminded of that fact...