Value-added auditing

Have you ever been at work when someone announced a new procedure that you knew right away was guaranteed to fail? What do you do about it? Of course if it's in your authority you can intervene immediately, but what if it's not? Is there any tool you can use to make your point?

Yes there is. Audits.

I used to work for a company who shipped products all over the world. Once we introduced a new product that had to be certified in each country where we shipped it, because one of the components was regulated as a communications device. This meant in each new market we had to supply a sample of the device, fill out a bunch of paperwork, and supply a fee; in exchange we got the right to import the product to that country for a certain period of time before we had to do it all again. But of course the expiration dates differed from country to country. So we had to keep track of the expiration dates for each country we shipped to, in order to make sure that we didn't inadvertently sell a product into a country where the certification had expired last week.

The design engineers were all in one office. The warehouse and the order-entry personnel were in a different facility, two time zones away. And when we recognized the problem posed by the need to certify this one component in country after country, someone proposed the following solution:

Let the Engineering Manager track the dates, because he's the one that has the specialized design knowledge to fill out all those government forms. He'll keep a big matrix that lists all the countries we ship to, and the current expiration date for each country. Then whenever he updates that list, he'll e-mail it to the Order-Entry Desk. That way they will always have a current list; and whenever they input an order for this product, they'll check the list to make sure we are really allowed to sell into that country. Foolproof!

Is it just me, or is it obvious to you, too, that this plan was guaranteed to fail? The most basic reason is that everything about this procedure was an exception. None of it was part of anyone's normal work routine.

The Engineering Manager had to remember to update the file in a timely way, even though that had almost nothing to do with the rest of his job. When he did update the file, he did it by hand.

The order-entry personnel had to check their emails in a timely way, and then fish out this file whenever it was updated. Then they had to put it somewhere, and remove the old outdated copies (to avoid confusion).  They did all of these steps by hand, with no system to guarantee even that the new files got stored in the same place where the old ones had been. (And of course if they accidentally stored different versions in different places, what would guarantee that they'd always find the latest one when they had to look for it?)

Finally, the Order Desk handled orders for thousands of different products. Among all those thousands of products, only this one had a rule that they had to stop everything and check a hand-built matrix to look up country and date before placing the order.

As I say, everything about the procedure was an exception. It was guaranteed to fail. But it was outside my area of responsibility, and it hadn't failed yet. So I couldn't really say anything.

What I did was to make notes to prepare for the next round of internal audits, which had already been scheduled for a few months later. When the time came, I asked the Engineering Manager for the latest copy of his matrix, and I scheduled an audit of the Order Entry Desk.

The people who operated the Order Entry Desk were really sharp, and we finished the routine part of the audit easily with no findings and lots of time to spare. So then I asked them for help. I reminded them of this procedure that was supposed to be in place—"Oh yes, we remember"—and then explained that I was afraid it was going to break down. Could they please help me? They opened the Order System in read-only mode, and searched for all orders for the product in question, starting with the latest one. For each order, they read me the country that it was shipped to and I checked it against the list. 

Within the first half dozen orders they read me, we found TWO that had shipped to a country where the certification had expired. Amazingly enough, that country's customs officials hadn't caught the shipments to reject them. But technically it was still a violation of a legal requirement. So I thanked the Order Entry auditees for their help, and wrote it up as a Major Nonconformity. I made a point to emphasize in the Closing Meeting that this finding was found while auditing the Order Desk, but it wasn't their fault: it was a system issue, and had to be addressed as such.

Soon after the company implemented a system of automated flags in the order tool itself, completely eliminating the manual exception-handling. It was a much better approach.