Category Archives: Statistics

QSR Required Procedures

I have identified 40 procedures required by 21 CFR 820: the Quality System Regulation

  1. § 820.22 for quality audits…
  2. § 820.25(b) for identifying training needs…
  3. § 820.30(a)(1) to control the design of the device in order to ensure that specified design requirements are met
  4. § 820.30(c) to ensure that the design requirements relating to a device are appropriate and address the intended use of the device, including the needs of the user and patient
  5. § 820.30(d) for defining and documenting design output in terms that allow an adequate evaluation of conformance to design input requirements
  6. § 820.30(e) to ensure that formal documented reviews of the design results are planned and conducted at appropriate stages of the device’s design development
  7. § 820.30(f) for verifying the device design
  8. § 820.30(g) for validating the device design
  9. § 820.30(h) to ensure that the device design is correctly translated into production specifications
  10. § 820.30(i) for the identification, documentation, validation or where appropriate verification, review, and approval of design changes before their implementation
  11. § 820.40 to control all documents that are required by this part
  12. § 820.50 to ensure that all purchased or otherwise received product and services conform to specified requirements
  13. § 820.60 for identifying product during all stages of receipt, production, distribution, and installation to prevent mixups
  14. § 820.65 for identifying with a control number each unit, lot, or batch of finished devices and where appropriate components
  15. § 820.70(a) that describe any process controls necessary to ensure conformance to specifications
  16. § 820.70(b) for changes to a specification, method, process, or procedure
  17. § 820.70(c) to adequately control these environmental conditions
  18. § 820.70(e) to prevent contamination of equipment or product by substances that could reasonably be expected to have an adverse effect on product quality
  19. § 820.70(h) for the use and removal of such manufacturing material to ensure that it is removed or limited to an amount that does not adversely affect the device’s quality
  20. § 820.72(a) to ensure that equipment is routinely calibrated, inspected, checked, and maintained
  21. § 820.75(b) for monitoring and control of process parameters for validated processes to ensure that the specified requirements continue to be met
  22. § 820.80(a) for acceptance activities
  23. § 820.80(b) for acceptance of incoming product
  24. § 820.80(c) to ensure that specified requirements for in-process product are met
  25. § 820.80(d) for finished device acceptance to ensure that each production run, lot, or batch of finished devices meets acceptance criteria
  26. § 820.90(a) to control product that does not conform to specified requirements
  27. § 820.90(b)(1) that define the responsibility for review and the authority for the disposition of nonconforming product
  28. § 820.90(b)(2) for rework, to include retesting and reevaluation of the nonconforming product after rework, to ensure that the product meets its current approved specifications
  29. § 820.100(a) for implementing corrective and preventive action
  30. § 820.120 to control labeling activities
  31. § 820.140 to ensure that mixups, damage, deterioration, contamination, or other adverse effects to product do not occur during handling
  32. § 820.150(a) for the control of storage areas and stock rooms for product to prevent mixups, damage, deterioration, contamination, or other adverse effects pending use or distribution and to ensure that no obsolete, rejected, or deteriorated product is used or distributed
  33. § 820.150(b) that describe the methods for authorizing receipt from and dispatch to storage areas and stock rooms
  34. § 820.160(a) for control and distribution of finished devices to ensure that only those devices approved for release are distributed and that purchase orders are reviewed to ensure that ambiguities and errors are resolved before devices are released for distribution
  35. § 820.170(a) for ensuring proper installation so that the device will perform as intended after installation
  36. § 820.184 to ensure that DHR’s for each batch, lot, or unit are maintained to demonstrate that the device is manufactured in accordance with the DMR and the requirements of this part
  37. § 820.198(a) for receiving, reviewing, and evaluating complaints by a formally designated unit
  38. § 820.200(a) for performing and verifying that the servicing meets the specified requirements
  39. § 820.250(a) for identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of process capability and product characteristics
  40. § 820.250(b) to ensure that sampling methods are adequate for their intended use and to ensure that when changes occur the sampling plans are reviewed

Targets Deconstructed

“Eliminate numerical goals, posters, and slogans for the work force, asking for new levels of productivity without providing methods.”

— Point No. 10 in Dr. W. E. Deming’s 14 points for management as written in “Quality, Productivity, and Competitive Position.”

A few weeks ago I had an excellent exchange on Twitter with UK Police Inspector Simon Guilfoyle on the topic of setting numerical targets. He asked “How do you set a numerical target without it being arbitrary? By what method?” Unfortunately, Twitter’s 140 character limit isn’t sufficient for adequately answering his question. I promised him I would write a post that explained my thinking.

When I was working for Samsung Austin Semiconductor (SAS) as a quality assurance engineer, one of my assigned responsibilities was to manage the factory’s overall nonconforming material rate. Over the course of my second year, the factory averaged a four percent* nonconforming material rate. The run chart for the monthly nonconforming material rate showed a stable system of variation.

As the year drew to a close, I began thinking about my goals for the following year. I knew I would continue to be responsible for managing the factory’s overall nonconforming material rate. What should I set as my target for it? Not knowing any better, I set it to be the rate we achieved for the current year: four percent. If nothing else, it was based on data. But my manager at the time, a Korean professional on assignment to the factory, mockingly asked me if I wasn’t motivated to do better. He set my target at two percent*; a fifty percent reduction.

What was the two percent number based on? How did he come about it? I had no insight and he didn’t bother to explain it either. From my perspective, it was an arbitrary numerical target; plucked out of thin air. I remember how incredibly nervous I felt about it. How was I going to achieve it? I had no clue nor guidance. I also remember how anxiety filled and frustrating the following year turned out for me. I watched the rate with a hawk eye. I hounded process engineers to do something whenever their process created a nonconforming lot. It was not a pleasant time for anyone.

Since then I’ve worked at several other companies in different industries. Nevertheless, my experience at SAS seems to be the norm when it comes to setting targets. This is regardless of the role, the industry or the culture. And, as far as I’ve been able to figure out, this approach to setting targets is driven more by tradition and arrogance than any objective thoughtful method. “Improve performance by 50% over last year!”, so the mantra goes. Worse still, no method is provided for achieving such arbitrary improvement targets. I’ve been told “You’re smart. You’ll figure out how to do it.”

So it’s not a surprise for me that folks like the good Inspector have become convinced all numerical targets are inherently arbitrary; that there is no objective and justifiable way to set them. Having been on the receiving end of such targets many times, I used to think the same, too. But just because people don’t know of a different way to set a target, one that is objective and can be justified, doesn’t mean there isn’t one. I believe numerical targets can be set in an objective fashion. It, however, requires thoughtfulness, great effort and understanding on the part of the person setting the target.

One way to set a target is to use the performance of a reference for comparison. In my case, the SAS factory I worked at had a sister facility in Korea. It would have been reasonable, albeit crude, to set my target for the nonconforming material rate to that achieved by the sister facility (if it was better.**) An argument could have been made that the target was achieved elsewhere, so it can be reached.

As part of our Twitter exchange, the Inspector made the point that regardless of whether these factories were defined to be sisters, there would still be differences between them. Therefore, they will generate a nonconforming material rate that is a function of their present system architecture. He is absolutely right! Setting a target for my factory based on the performance achieved by its sister facility alone will do nothing to improve the performance of my factory. It’s already doing the best it can.

But that’s not the point of setting the target: to operate the same system and expect an improved performance. The point of setting the target is to trigger a change in the system, a redesign in such a way as to achieve a level of performance that, in this case, has been achieved elsewhere. The sister system can be treated as a reference and studied. Differences between systems may be identified and eliminated. Along the way we may find out that some differences cannot be eliminated. Nevertheless, by eliminating the differences where possible the two systems are made more similar to one another and we will have improved the performance.

In the absence of a reference, simulations may be used to objectively define a target. The factory’s overall nonconforming material rate is the combined result of the nonconforming material rates of its individual processes. Investigating the performance of these inputs can help identify opportunities for improvement for each: stabilizing unstable processes, running stable processes on target, reducing the variability of stable on-target processes. All of this can be simulated to determine what is ideally possible. A justifiable target for the nonconforming material rate can then be set with the results. Best of all, the method by which it can be achieved gets defined as part of the exercise.

Finally, targets may be set by the state of the greater environment within which a system operates. All systems operate in a greater environment (e.g. national or global economy); one that is continuously changing in unpredictable ways. City populations grow or shrink. Markets grow or shrink. Polities combine or fragment. What we once produced to meet a demand will in a new environment prove to be too little or too much. A change in state of the external environment should trigger a change in the target of the system. A change in the target of the system should trigger a redesign of the system to achieve it. In Systems lingo, this is a tracking problem.

Targets are essential. They help guide the design or redesign of the system. They can be defined objectively in several different ways. I’ve outlined three above. They do not have to be set in the arbitrary way they currently are. But setting targets isn’t enough. Methods by which to achieve them must be defined. Targets, even objective ones, are meaningless and destructive without the means of achieving them. Failure to achieve targets should trigger an analysis into why the system failed. They should not be used to judge and blame workers within the system.

Sadly, people are like water, finding and using the path of least resistance. Setting arbitrary improvement targets is easier than doing all the work required to set objective ones. They have been successfully justified on the grounds of mindless ambition. No one questions the approach out of fear or ignorance. Positional authority is often used to mock or belittle the worker for not being motivated enough when the truth is something else: managerial ignorance and laziness to do their job.

* I’ve changed the numbers for obvious reasons. However, the message remains the same.

** As it turned out, the nonconforming material rate achieved at my factory was the best ever in all of Samsung!




  Total Design  the-diabolic why-a-guide-to-finding-and-using-causes


the-human-use-of-human-beings spc-at-the-esquire-club pmbok quality-or-else the-drunkards-walk qbq Critical Chain Willpower Beyond the Stable State Deep Work Andy and Me The Power of Habit Give and Take Systems Thinking For Social Change Toyota Kata


The Web Of Life The New Economics The Third Wave Six Easy Pieces Redesigning The Future Understanding Industrial Experimentation Deming Management Method Root Cause Analysis The Why Axis The Trust Factor The Man Who Discovered Quality What The Buddha Taught SpiritofKaizen Quality Wars 4thGenMgmt QPCP TheBonoboAndTheAtheist ManagerialBreakthrough Nonzero ATSPC TaaichiOhnosWorkplaceManagement Kaizen Confession of a Buddhist Atheist Future Shock Big Bang Longitude The Moral Animal The Black Swan Out of the Crisis OtherSideofInnovation Drive

Dealing with Nonconforming Product

A particular process makes parts of diameter D. There are 10 parts produced per batch. The batches are sampled periodically and the diameter of all the parts from the sampled batch is measured. Data, representing deviation from the target, for the first 6 sampled batches is shown in Table 1. The graph of the data is shown in Figure 1. Positive numbers indicate the measured diameter was larger than the target while negative numbers indicate the measured diameter was smaller than the target. The upper and lower specification limits for acceptable deviation are given as +/- 3.


Table 1. Data for six batches of 10 parts each. The numbers represent the deviation from the target.


Figure 1. Graph of the data from the table above. The most recent batch, batch number six, shows one part was nonconforming.

The most recent batch, sample batch number six, shows one of the 10 parts having a diameter smaller than the lower specification limit. As such, it is a nonconforming part.

The discovery of a nonconforming product triggers two parallel activities: i) figuring out what to do with the nonconforming product, and ii) addressing the cause of the nonconforming product to inhibit the nonconformance from occurring again.


Nonconforming product may be repaired or reworked when possible, but it can always be scrapped. Each one of these three options has its own set of complications and cost.

Repairing a nonconforming product involves additional steps beyond what are usually needed to make the product. This additional processing has the potential to create previously unknown weaknesses in the product e.g. stress concentrations. So repaired product will need to be subjected to testing that verifies it still satisfies its intended use. For this particular case, repairing is not possible. The diameter is smaller than the target. Repair would have been possible if the diameter had been larger than the target.

Reworking a nonconforming product involves undoing the results of the previous process steps, then sending the product through the standard process steps a second time. Undoing the results of the previous process steps involves additional process steps just as were required to repair a nonconforming product. This additional processing has the potential to create previously unknown weaknesses in the product. Reworked product will need to be subjected to testing that verifies it still satisfies its intended use. For this particular case, reworking is not possible.

Scrapping a nonconforming product means to destroy it so that it cannot be accidentally used. For this particular case, scrapping the nonconforming part is the only option available.


In order to determine the cause of the nonconformity we have to first determine the state of the process i.e. whether the process is stable or not. The type of action we take depends on it.

A control chart provides a straightforward way to answer this question. Figure 2. shows an Xbar-R chart for this process. Neither the Xbar chart (top), nor the R chart (bottom) show uncontrolled variation. There is no indication of a special cause affecting the process. This is a stable process in the threshold state. While it is operating on target i.e. its mean is approximately the same as the target, its within-batch variation is more than we would like. Therefore, there is no point trying to hunt down a specific cause for the nonconforming part identified above. It is most likely the product of chance variation that affects this process; a result of the process’s present design.


Figure 2. Xbar-R chart built using the first six sampled batches. Neither the Xbar chart nor the R chart show uncontrolled variation. There is no indication of a special cause affecting the process.

In fact, the process was left alone to collect more data (Figure 3.). The Xbar-R charts do not show any unusual variation that would indicate external disturbances affecting the process. Its behavior is predictable.


Figure 3. More data was collected and the control limits were recalculated using the first 15 sampled batches. The process continues to look stable with no signs of external disturbance.

But, even though the process is stable, it does produce nonconforming parts from time to time. Figure 4. shows that a nonconforming part was produced in sampled batch number 22 and one in sampled batch number 23. Still, it would be wasted effort to hunt down specific causes for the creation of these nonconforming parts. They are the result of chance variation that is a property of the present process design.


Figure 4. Even though the process is stable it still occasionally produces nonconforming parts. Sampled batch number 22 shows a nonconforming part with a larger than tolerable diameter while sampled batch number 23 shows one with a smaller than tolerable diameter.

Because this process is stable, we can estimate the mean and standard deviation of the distribution of individual parts. They were calculated to be -0.0114 and 0.9281. Assuming that the individual parts are normally distributed, we can estimate that this process will produce about 0.12% nonconforming product if left to run as is. Some of these parts will be smaller than the lower specification limit for the diameter. Others will be larger than the upper specification limit for the diameter. That is, about 12 nonconforming pieces will be created per 10,000 parts produced. Is this acceptable?

If the calculated nonconforming rate is not acceptable, then this process must be modified in some fundamental way. This would involves some sort of structured experimentation using methods from design of experiments to reduce variation. New settings for factors like RPM or blade type among others will need to be determined.

Entropy and Assignable Causes

Left alone, a process in the Ideal State will migrate toward the State of Chaos over time. Wear and tear of parts in use will lead to breakdowns and failures. Even mothballed machines cannot escape deterioration and decay caused by the environment. Entropy affects everything.

However, the effects of entropy can be repaired. Signatures of deterioration and failure made apparent by process control charts provide clues as to what repairs need to be made when. But, a process isn’t affected just by entropy. Other assignable causes keep it from operating in a predictable fashion. Replacing worn out parts might pull a process out from the State of Chaos back to the Brink of Chaos, but it’s just a matter of time before it returns to it.

Process managers, through the proper use of process control charts, must counteract the effects of entropy and assignable causes to help their processes achieve the ideal state and stay there. This is a never ending cycle.

Note: I learned this material from reading Dr. Wheeler’s writings. My post is intended to reflect my current understanding. None of the ideas herein are original to me. Any errors are my failures alone.


The State of Chaos

When a process is out of control and it is producing nonconforming product it is in a state of chaos. The State of Chaos is one of the four states a process can be in as shown in “What State Is Your Process In?“. The manufacturer cannot predict how much nonconforming product his process will produce in any given hour or day. At times the process will produce nothing but conforming product. Then without warning it will produce nothing but nonconforming product. It might seem as if there were ghosts in the machine.

A process in such a state is affected by assignable causes that are easily identified through the use of process control charts. The effects of these assignable causes have to be eliminated one at a time. Patience and perseverance are necessary. It is essential that the process be brought under statistical control and made predictable. Once the process has achieved stability further improvement efforts can be made to reach the ideal state.


Note: I learned this material from reading Dr. Wheeler’s writings. My post is intended to reflect my current understanding. None of the ideas herein are original to me. Any errors are my failures alone.


The Brink of Chaos

Of the four states a process can be in (see “What State Is Your Process In?“) the most sinister state is the one where it is producing 100 percent conforming product but is operating in an unpredictable way. That is, the process is not under statistical control. Such a process is, in fact, on the brink of chaos. But, hold on. There is no nonconforming product, therefore there is no problem, right? It is easy to get lulled into complacency by this happy circumstance.


But because the process is not under statistical control it is impossible to predict what it will do in the next instance. Various assignable causes are affecting the process in an unpredictable fashion. The effect of these causes could very well be the production of nonconforming product without any warning. When that happens the process has moved into a state of chaos.

The only way to address a process on the brink of chaos is to use process control charts to identify assignable causes and eliminate their effects one-by-one and bring the process under statistical control. You can then start other improvement efforts like moving the process mean to the process aim and reducing the process variation by minimizing the influence of common causes affecting the process.

Note: I learned this material from reading Dr. Wheeler’s writings. My post is intended to reflect my current understanding. None of the ideas herein are original to me. Any errors are my failures alone.


The Threshold State

A process that is predictable or in a state of statistical control, but producing nonconforming product can be described as being in the Threshold State. This is one of the four possible states that a process can be in as noted in “What State Is Your Process In?“. But, what might such a process look like?

A process in the threshold state might be operating with its mean higher than the process aim,


or it might be operating with its mean lower than the process aim,


or it might be operating with a process dispersion greater than the product specification window,


or it may be operating with some combination of a shift in its mean and breadth of its dispersion.

Nevertheless, the fact that such a process is in statistical control means that it will continue to produce consistent product so long as it stays in control. This in turn means that the producer can expect to produce a consistent amount of nonconforming product hour after hour day after day until a change is made in the process or a change is made to the specifications.

It is important to say here that exhorting your workers to work harder or to “Do it right the first time” or to show them the examples of nonconforming product from a process in the threshold state will not lead to improvements. They are not the cause for the failures. The causes for the nonconforming product are systemic and must be dealt with at the system level. Focusing on the worker will only serve to demoralize and frustrate them. It may lead to tampering with the process turning a bad situation worse.

You can always share your process data with your customer to demonstrate its stability and ask for a change in the product specifications. However, if specifications cannot be changed your only recourse is to modify your process to shift it from the threshold state into the ideal state. Adjusting the process mean to match the aim is usually relatively simple. In comparison, reducing the process variation requires an understanding of the common causes affecting the process and their respective effects – a much more involved activity.

While you are working on improving your process you are still producing nonconforming product. Until such time as you achieve the ideal state for your process, you must screen every unit or lot before shipping product to your customer – a 100 percent inspection and sort. This should be treated as a temporary stop-gap measure. You must recognize it as an imperfect quality control method and be mindful that defectives will escape.

Note: I learned this material from reading Dr. Wheeler’s writings. My post is intended to reflect my current understanding. None of the ideas herein are original to me. Any errors are my failures alone.


The Ideal State

In “What State Is Your Process In?” I noted that a process can be in one of four possible states. Here I write about the Ideal State wherein a process is predictable and is producing 100 percent conforming product.

A process that is predictable is one that is in a state of statistical control. The variability of the product from one unit to the next is randomly distributed about the average and bounded within statistically established limits – its natural limits (red solid lines in the figure below). So long as the process remains “in control”, it will continue to produce units within these limits.


Complete product conformity comes about when the process’s natural limits fall within the product’s specification limits (blue solid lines in the figure below). This depicts the ideal state for a process.


In order for a process to achieve this ideal state

  • The process must be inherently stable over time. This means that in the absence of external disturbances – what Dr. Shewhart referred to as assignable causes – the process’s natural variability does not change over time. (Note: There are processes that are inherently chaotic. An excellent reference to such processes is “Nonlinear Dynamics And Chaos” by Professor Steven Strogatz)
  • The process must be operated in a stable and consistent manner. The operating conditions cannot be selected or changed arbitrarily. Often machine parameters are tweaked in response to natural fluctuations in the process’s output. These actions add to the process’s natural variation disrupting its stability. Dr. Deming demonstrated the effects of such tampering through the “Nelson Funnel Experiment”. (Bill Scherkenbach has an excellent discussion of it in “Deming’s Road to Continual Improvement”.)
  • The process average must be set and maintained at an appropriate level. If you refer to the charts above, you can imagine the consequence of moving the process average up or down from its aim. The result is the production of nonconforming product either on the high or low side.
  • The natural tolerance of the process must be less than the specified tolerance for the product. This is obvious upon a first glance at the second chart above.

A process that satisfies these four conditions will be in the ideal state and the manufacturer can be confident that he is shipping only conforming product. In order to maintain the process in the ideal state he must use process control charts. He must act on the signals they provide to promptly identify assignable causes and eliminate their effects.

Note: I learned this material from reading Dr. Wheeler’s writings. My post is intended to reflect my current understanding. None of the ideas herein are original to me. Any errors are my failures alone.


What State Is Your Process In?

You can take one of two approaches to controlling the quality of your product. Once manufactured, you can compare it against its specifications and sort it as either conforming or nonconforming. This, however, will guarantee the greatest degree of variability between units as anything within specifications is considered acceptable. Alternatively, you can work to run the manufacturing process as consistently as possible to produce units that vary as little as possible from one another.

There is no path to improving performance with the first approach where you sort manufactured product as conforming or nonconforming. However, there is a clear path to improving performance through the use of process control techniques introduced by Dr. Walter Shewhart. But, how do you gauge improvement? One measure is achieving a state of statistical control for the process so that its behavior is predictable. Another measure is the manufacture of 100 percent conforming product by the process.

When these two measures are taken together, there are four possible states a process can be in (see figure below): The process is


Note: I learned this material from reading Dr. Wheeler’s writings. My post is intended to reflect my current understanding. None of the ideas herein are original to me. Any errors are my failures alone.