Last week I got a call from a friend and former colleague, who wanted to talk about stage gates. (You remember I've discussed them before here and here.) Specifically, he wanted my perspective on the question, "When do all the formal requirements have to be incorporated into a product? Can you wait till you release it, or do they have to be there before you start testing?"
Phrased like that the question sounds pretty innocuous, and also simple: It depends. Naturally all the requirements should be in place before you release the product, because otherwise you're not really done. And you'd like everything to be ready before you start testing, so that you can test it all together to make sure it works. If one feature isn't ready when you start testing, you can't test that feature; then when you finally get to it, there's always a risk you'll find a problem which impacts something else you thought was already buttoned-up. At the same time there are plenty of projects where this or that feature isn't quite ready when testing begins, and the project makes a conscious decision to proceed at risk. As I say, it depends.
Then as we talked some more, I began to understand that my friend had a real-life example in mind, one which made the discussion a little more pointed.
Last October, a GM Cruise self-driving taxi struck and dragged a pedestrian 20 feet.* This accident was only the latest of several last year, some of which got significant attention in the news.** After the October accident, the National Highway Traffic Safety Administration began an investigation; the State of California suspended Cruise's permission to operate autonomous vehicles; and Cruise recalled its entire fleet for reprogramming. Cruise's interaction with regulators appears to have been remarkably clumsy; while Cruise says they had no intention to mislead either the regulators or the public about what happened, apparently their first communications did exactly that. Since then Cruise has fired nine executives and let go a quarter of its staff. The CEO and one co-founder both resigned, and Cruise published a blog post blaming its corporate culture.***
So my friend's real question was something like, "What precautions do you have to take before you let an autonomous vehicle out on the public roads? Let's say you are still in the test phase; can you legitimately skip some of the safety features because your car is 'not released yet'? And if so, what stops you from ending up like Cruise?"
It's a good question, because it calls attention to the difference between formal requirements and pragmatic requirements. Often we think of formal requirements as more numerous than pragmatic ones: pragmatically, a hammer should drive nails; formally, there's an entire ISO standard of technical specifications for steel hammer heads. (That's ISO 15601:2000, in case you wondered.) But in a pre-release situation, sometimes the distinction reverses. Depending on how your Quality Management System is written, many product requirements might not be formally mandatory before release. But you may need them for pragmatic reasons, all the same.
To be clear, Cruise's vehicles had been fully released, because Cruise was operating a business and collecting fees. But there are plenty of other companies making autonomous vehicles, or wanting to make them. So my friend wasn't asking "What did Cruise do wrong?"—though it would be fascinating to dive into that question, if we had the data—but rather, "If Whizzbang Motors decides to build an autonomous car tomorrow, what's to stop one of their test drives from ending just as badly as this well-publicized accident?"
So I started at the beginning. Quality means getting what you want. And obviously you don't want to hurt people. So Quality means making sure that you don't. For a released product, especially in the automotive market, that means complying with a long list of federal regulations; and that, in turn, means proving that you comply with them by providing objective evidence like documented test results. For an unreleased product, you may not have all of that documentation yet. But you still have to avoid hurting people, and that means you have to figure out what it is going to take.
And when the day comes that you finally have to test on the public roads? Doubtless there will be public, legal requirements that you have to meet before you can put your car on the roads, even if it's not released yet. But beyond those legal requirements, you have pragmatic and moral requirements to have done everything in your power ahead of time to make sure the car is safe. Some people will insist that "Nothing can be made 100% safe," and in a literal sense they are right. But that is never an excuse for doing any less than you possibly can.
When it comes to the safety of any pre-release product, your first thought should never be "What do the rules make me do?" Ask yourself instead, "How do I avoid hurting people?"
__________
* See, for example, either of the following articles:
- GM Cruise robotaxi unit faces US probes over dragging incident, vows reforms | Reuters
- GM’s Cruise reveals dual US probes into grisly collision and company’s response | Self-driving cars | The Guardian.
** See for example this accident in April 2023, or this one in August. In all of 2023, Cruise vehicles were involved in 23 crashes, out of a fleet which numbered at least 950 at the time of the October accident.
*** Wow, where have we heard that before? 😀
No comments:
Post a Comment