Category Archives: Statistics

On Variation and Some of its Sources

Diversity

Virtually every component is made to be assembled with its counterpart(s) into sub-assemblies and final assemblies.

If individual pieces of a given component could be made identical to one another, then they would either all conform or all fail to conform to the component’s design requirements. If they conform, then we could pick a piece at random for use in a sub- or final-assembly. It would fit and function without fail, as intended.

But material varies, machine operation varies, the work method varies, workers vary, measurement varies, as does the environment. Such variation, especially in combination, makes it impossible to produce anything identical. Variation is a fundamental principle of nature. All things vary.

Variation affects the fit, the form and the function of a component. And, it is propagated along the assembly line such that the final product is at times a mixed bag of conforming and nonconforming widgets.

Material Consider 316 Stainless Steel. It is used to make medical devices. Manufacturers buy it from metal producers in sheet stock or bar stock.

If we measured the dimensional attributes of the received stock, e.g. its diameter or length, for several different purchase orders, we would see that they were not identical between orders. They vary. If we measured these attributes for pieces received in a single order, we would see that they were not identical between pieces of that order either. If we measured these attributes for a single piece at different points in its cross-section, we would see that they, too, were not identical. If we then zoomed in to investigate the crystalline structure of the stainless steel, we would see that the crystals were not identical in shape or size.

The elemental composition, in percent by weight, of 316 Stainless Steel is: 0.08% Carbon, 2.00% Manganese, 0.75% Silicon, 16.00-18.00% Chromium, 10.00-14.00% Nickel, 2.00-3.00% Molybdenum, 0.045% Phosphorous, 0.030% Sulfur, 0.10% Nitrogen, and the balance is Iron. We see that the amount of Chromium, Nickel, Molybdenum and Iron are specified as ranges i.e. they are expected to vary within them by design!

These are some of the ways a specific raw material used in the production of medical devices varies. Keep in mind that a medical device isn’t a single component but an assembly of several components likely made of different materials that will vary in just such ways as well. All this variation affects the processing (i.e. machining, cleaning, passivation, etc.) of the material during manufacturing, as well as the device performance in use.

Machine One piece of equipment used in the production of medical device components is the CNC (Computer Numerical Control) machine. Its condition, as with all production equipment, varies with use.

Take the quality of the lubricating fluid: it changes properties (e.g. its viscosity) with temperature thus affecting its effectiveness. The sharpness of the cutting tool deteriorates with use. A component made with a brand new cutting tool will not be identical to one made from a used cutting tool whose cutting edges have dulled. The cutting is also affected by both the feed-rate and the rotation-rate of the cutting tool. Neither of which remain perfectly constant at a particular setting.

What’s more, no two machines perform in identical ways even when they are the same make and model made by the same manufacturer. In use, they will almost never be in the same state as each other, with one being used more or less than the other, and consumables like cutting tools in different states of wear. Such variability will contribute to the variability between the individual pieces of the component.

Method Unless there is standardized work, we would all do the work in the best way we know how. Each worker will have a best way slightly different from another. Variation in best ways will find its way into the pieces made using them.

These days a production tool like a CNC machine offers customized operation. The user can specify the settings for a large number of operating parameters. Users write “code” or develop a “recipe” that specifies the settings for various operating parameters in order to make a particular component. If several such pieces of code or recipes exist, one different from another, and they are used to make a particular component, they will produce pieces of that component that vary from one to another.

When and how an adjustment is made to control parameters of a tool will affect the degree of variation between one piece and another. Consider the method where a worker makes adjustment(s) after each piece is made to account for its deviation from the target versus one where a worker makes an adjustment only when a process shift is detected. Dr. Deming and Dr. Wheeler have shown that tampering with a stable process, as the first worker does, will serve to increase the variation in the process.

All such variation in method will introduce variability into the manufactured pieces.

Man There are a great many ways in which humans vary physically from one another. Some workers are men, others are women. Some are short, others are tall. Some are young, others are older. Some have short fat fingers, others have long thin fingers. Some have great eyesight, others need vision correction. Some have great hearing, others need hearing aids. Some are right handed, others are left handed. Some are strong, others not so much. Some have great hand-eye coordination, others do not. We all come from diverse ethnic backgrounds.

Not all workers have identical knowledge. Some have multiple degrees, others are high school graduates. Some have long experience doing a job, others are fresh out of school. Some have strong knowledge in a particular subject, others do not. Some have deep experience in a single task, others have shallow experience. Some have broad experience, others have focused experience.

Last, but not least, we all bring varying mindsets to work. Some may be intrinsically motivated, others need to be motivated externally. Some may be optimists, others may be pessimists. Some want to get better everyday, others are happy where they are. Some like change, others resist it. Some are data driven, others use their instinct.

All this variation affects the way a job gets done. The variation is propagated into the work and ultimately manifests itself in the variation of the manufactured component.

Measurement We consider a measured value as fact, immutable. But that’s not true. Measuring the same attribute repeatedly does not produce identical results between measurements.

Just like production tools, measurement tools wear from use. This affects the measurement made with it over the course of its use.

And also just like production tools, the method (e.g. how a part is oriented, where on the part the measurement is made, etc.) used to make a measurement affects the measured value. There is no true value of any measured attribute. Different measurement methods produce different measurements of the same attribute.

So even if by chance two pieces were made identical we wouldn’t be able to tell because of the variability inherent in the measurement process.

Environment Certain environmental factors affect all operations regardless of industry. One of them is time. It is not identical from one period to the next. Months in a year are not identical in duration. Seasons in a year are different from one another. Daytime and nighttime are not identical to one another. Weekdays and weekends are not identical to one another.

Even in a climate controlled facility the temperature cycles periodically around a target. It varies between locations as well. Lighting changes over the course of the day. Certain parts of the workplace may be darker than others. Noise, too, changes over the course of the day: quiet early in the morning or into the night, and noisier later into the day. There is variation in the type of noise, as well. Vibration by definition is variation. It can come from a heavy truck rolling down the street or the motor spinning the cutting tool in a production machine. Air movement or circulation isn’t the same under a vent as compared to a spot away from a vent, or when the system is on versus when it is off.

The 5M+E (Material, Machine, Method, Man, Measurement, and Environment) is just one way to categorize sources of variation. The examples in each are just a few of the different sources of variation that can affect the quality of individual pieces of a component. While we cannot eliminate variation, it is possible to systematically reduce it and achieve greater and greater uniformity in the output of a process. The objective of a business is to match the Voice of the Customer (VOC) and the Voice of the Process (VOP). The modern day world-class definition of quality is run-to-target with minimal variation!

Our approach has been to investigate one by one the causes of various “unnecessaries” in manufacturing operations…
— Taiichi Ohno describing the development of the Toyota Production System

Links
[1] Kume, Hitoshi. Statistical Methods for Quality Improvement. Tokyo, Japan: The Association for Overseas Technical Scholarship. 2000. Print. ISBN 4-906224-34-2

[2] Monden, Yasuhiro. Toyota Production System. Norcross, GA: Industrial Engineering and Management Press. 1983. Print. ISBN 0-89806-034-6

[3] Wheeler, Donald J. and David S. Chambers. Understanding Statistical Process Control. Knoxville, TN: SPC Press, Inc. 1986. Print. ISBN 0-945320-01-9

Reliability, Confidence, and Sample Size, Part II

In Part I the problem was to find the sample size, n, given failure count, c = 0, confidence level = 1 – P(c = 0), and minimum reliability = (1 – p’). The table giving sample size, n, with failures c = 0, for certain common combinations of confidence level and minimum reliability is reproduced below.

While I would like that none of the samples fail testing, failures do happen. Does that mean testing should stop on first fail? Are the test results useless? In this part I will flip the script. I will talk about what value I can extract from test results if I encounter one or more failures in the test sample.

I start with the binomial formula as before

It gives us the likelihood, P(x = c), of finding exactly c failures in n samples for a particular population failure rate p’. (Note that 1 – P(x c) is our confidence level, and 1 – p’ = q’ is our desired reliability.)

However, knowing the likelihood of finding just c failures in n samples isn’t enough. Different samples of size n from the same population will give different counts of failures c. If I am okay with c failures in n samples, then I must be okay with less than c failures, too! Therefore, I need to know the cumulative likelihood of finding c or less failures in n samples, or P(x c). That likelihood is calculated as the sum of the individual probabilities. For example, if c = 2 samples fail, I calculate P(x 2) = P(x = 2) + P(x = 1) + P(x = 0).

For a particular failure rate p’, I can make the statement that my confidence is 1 – P(x c) that the failure rate is no greater than p’ or alternatively my reliability is no less than q’ = (1 – p’).

It is useful to build a plot of P(x c) versus p’ to understand the relationship between the two for a given sample size n and failure count c. This plot is referred to as the operating characteristic (OC) curve for a particular n and c combination.

For example, given n = 45, and c = 2, my calculations would look like:

The table below shows a few values that were calculated:

A plot of P(c 2) versus p’ looks like:

From the plot I can see that the more confidence I require, the higher the failure rate or lesser the reliability estimate will be (e.g. 90% confidence with 0.887 reliability, or 95% confidence with 0.868 reliability.) Viewed differently, the more reliability I require, the less confidence I have in my estimate (e.g. 0.95 reliability with 40% confidence level).

Which combination of confidence and reliability to use depends on the user’s needs. There is no prescription for choosing one over another.

I may have chosen a sample size of n = 45 expecting c = 0 failures for testing with the expectation of having 90% confidence at 0.95 reliability in my results. But just because I got c = 2 failures doesn’t mean the effort is for naught. I could plot the OC curve for the combination of n, and c to understand how my confidence and reliability has been affected. Maybe there is a combination that is acceptable. Of course, I would need to explain why the new confidence, and reliability levels are acceptable if I started with something else.

 

 


Operating characteristic curves can be constructed in MS Excel or Libre Calc with the help of BINOM.DIST(c, n, p’, 1) function.

Once I have values for p’ and P(c ≤ 2), I can create an X-Y graph with X = p’, and Y = P(c 2).

Links
[1] Burr, Irving W. Elementary Statistical Quality Control. New York, NY: Marcel Dekker, Inc. 1979. Print. ISBN 0-8247-6686-5

Reliability, Confidence, and Sample Size, Part I

Say I have designed a widget that is supposed to survive N cycles of stress S applied at frequency f.

I can demonstrate that the widgets will conform to the performance requirement by manufacturing a set of them and testing them. Such testing, though, runs headlong into the question of sample size. How many widgets should I test?

For starters, however many widgets I choose to test, I would want all of them to survive i.e. the number of failures, c, in my sample, n, should be zero. (The reason for this has more to do with the psychology of perception than statistics.)

If I get zero failures (c = 0) in 30 samples (n = 30), does that mean I have perfect quality relative to my requirement? No, because the sample failure rate, p = 0/30 or 0%, is a point estimate for the population failure rate, p’. If I took a different sample of 30 widgets from the same population, I may get one, two, or more failures.

The sample failure rate, p, is the probability of failure for a single widget as calculated from test data. It is a statistic. It estimates the population parameter, p’, which is the theoretical probability of failure for a single widget. The probability of failure for a single widget tells us how likely it is to fail the specified test.

If we know the likelihood of a widget failing the test, p’, then we also know the likelihood of it surviving the test, q’ = (1 – p’). The value, q’, is also known as the reliability of the widget. It is the probability that a widget will perform its intended function under stated conditions for the specified interval.

The likelihood of finding c failures in n samples from a stable process with p’ failure rate is given by the binomial formula.

But here I am interested in just the case where I find zero failures in n samples. What is the likelihood of me finding zero failures in n samples for a production process with p’ failure rate?

If I know the likelihood of finding zero failure in n samples from a production process with p’ failure rate, then I know the likelihood of finding 1 or more failures in n samples from the production process, too. It is P(c ≥ 1) = 1 – P(0). This is the confidence with which I can say that the failure rate of the production process is no worse than p’.

Usually a lower limit is specified for the reliability of the widget. For example, I might want the widget to survive the test at least 95% of the time or q’ = 0.95. This is the same as saying I want the failure rate to be no more than p’ = 0.05.

I would also want to have high confidence in this minimum reliability (or maximum failure rate). For example, I might require 90% confidence that the minimum reliability of the widget is q’ = 0.95.

A 90% confidence that the reliability is at least 95% is the same as saying 9 out of 10 times I will find one or more failures, c, in my sample, n, if the reliability were less than or equal to 95%. This is also the same as saying that 1 out of 10 times I will find zero failures, c, in my sample, n, if the reliability were less than or equal to 95%. This, in turn, is the same as saying P(0) = 10% or 0.1 for p’ = 0.05.

With P(0) and p’ defined, I can calculate the sample size, n, that will satisfy these requirements.

The formula can be used to calculate the sample size for specific values of minimum reliability and confidence level. However, there are standard minimum reliability and confidence level values used in industry. The table below provides the sample sizes with no failures for some standard values of minimum reliability and confidence level.

What should the reliability of the widget be? That depends on how critical its function is.

What confidence level should you choose? That again depends on how sure you need to be about the reliability of the widget.

Note: A basic assumption of this method is that the failure rate, p’, is constant for all the widgets being tested. This is only possible if the production process producing these widgets is in control. If this cannot be demonstrated, then this method will not help you establish the minimum reliability for your widget with any degree of confidence.

Links
[1] Burr, Irving W. Elementary Statistical Quality Control. New York, NY: Marcel Dekker, Inc. 1979. Print. ISBN 0-8247-6686-5

Whose Measurement is Right?

Every company I’ve worked for inspects the product it receives from its suppliers to determine conformance to requirements. The process is variously referred to as incoming inspection or receiving inspection.

Sometimes the receiving inspection process identifies a lot of product that fails to conform to requirements. That lot is subsequently classified as nonconforming material and quarantined for further review. There are many reasons why a lot of product may be classified as nonconforming. Here I wish to focus just on reasons having to do with measurement.

Once a company discovers nonconforming parts, it usually contacts its supplier to share that information. It is not unusual, however, for the supplier to push back when their data for the lot of product shows it to be conforming. So, how can a given lot of product be both conforming and nonconforming? Who is right?

We need to recognize that measurement is a process. The measured value is an outcome of this process. It depends on the measurement tool used, the skill of the person making the measurement and the steps of the measurement operation. A difference in any of these factors will show up as a difference in the measured value.

It is rare that a measurement process is the same between a customer and its supplier. A customer may use different measurement tools than its supplier. For example, where the customer might have used a caliper or micrometer, the supplier may have used an optical comparator or CMM. Even if both the customer and the supplier use the same measurement tool, the workers using that tool are unlikely to have been trained in its use in the same way. Finally, the steps used to make the measurement, such as fixturing, lighting and handling the part, which often depend on the measurement tool used, will likely be different, too.

Thus, more often than not, a measurement process between a supplier and a customer will be different. Each measured value is correct in its context—the supplier’s measurement is correct in its context, as is the customer’s measurement in its context. But because the measurement process is different between the two contexts, the measured values cannot be compared directly with one another. So it is possible that the same lot of product may be conforming per the supplier’s measurements and nonconforming per the customer’s measurements.

But why are we measuring product twice: once by the supplier and again by the customer? Measurement is a costly non-value adding operation, and doing it twice is excess processing–wasteful. One reason I’ve been told is this is done to confirm the data provided by the supplier. But confirmation is possible only if the measurement process used by the customer matches the one used by the supplier.

Besides, if we are worried about the quality of supplier data, we should then focus efforts on deeply understanding their measurement process, monitoring its stability and capability, and working with the supplier to improve it if necessary. With that we should trust the measurement data the supplier provides and base our decisions on it, and eliminate the duplicate measurement step during receiving inspection.

Links
[1] Eliminate Waste in Incoming Inspection: 10 ideas of where to look for waste in your process http://www.qualitymag.com/articles/92832-eliminate-waste-in-incoming-inspection Retrieved 2017-06-29

QSR Required Procedures

I have identified 40 procedures required by 21 CFR 820: the Quality System Regulation

  1. § 820.22 for quality audits…
  2. § 820.25(b) for identifying training needs…
  3. § 820.30(a)(1) to control the design of the device in order to ensure that specified design requirements are met
  4. § 820.30(c) to ensure that the design requirements relating to a device are appropriate and address the intended use of the device, including the needs of the user and patient
  5. § 820.30(d) for defining and documenting design output in terms that allow an adequate evaluation of conformance to design input requirements
  6. § 820.30(e) to ensure that formal documented reviews of the design results are planned and conducted at appropriate stages of the device’s design development
  7. § 820.30(f) for verifying the device design
  8. § 820.30(g) for validating the device design
  9. § 820.30(h) to ensure that the device design is correctly translated into production specifications
  10. § 820.30(i) for the identification, documentation, validation or where appropriate verification, review, and approval of design changes before their implementation
  11. § 820.40 to control all documents that are required by this part
  12. § 820.50 to ensure that all purchased or otherwise received product and services conform to specified requirements
  13. § 820.60 for identifying product during all stages of receipt, production, distribution, and installation to prevent mixups
  14. § 820.65 for identifying with a control number each unit, lot, or batch of finished devices and where appropriate components
  15. § 820.70(a) that describe any process controls necessary to ensure conformance to specifications
  16. § 820.70(b) for changes to a specification, method, process, or procedure
  17. § 820.70(c) to adequately control these environmental conditions
  18. § 820.70(e) to prevent contamination of equipment or product by substances that could reasonably be expected to have an adverse effect on product quality
  19. § 820.70(h) for the use and removal of such manufacturing material to ensure that it is removed or limited to an amount that does not adversely affect the device’s quality
  20. § 820.72(a) to ensure that equipment is routinely calibrated, inspected, checked, and maintained
  21. § 820.75(b) for monitoring and control of process parameters for validated processes to ensure that the specified requirements continue to be met
  22. § 820.80(a) for acceptance activities
  23. § 820.80(b) for acceptance of incoming product
  24. § 820.80(c) to ensure that specified requirements for in-process product are met
  25. § 820.80(d) for finished device acceptance to ensure that each production run, lot, or batch of finished devices meets acceptance criteria
  26. § 820.90(a) to control product that does not conform to specified requirements
  27. § 820.90(b)(1) that define the responsibility for review and the authority for the disposition of nonconforming product
  28. § 820.90(b)(2) for rework, to include retesting and reevaluation of the nonconforming product after rework, to ensure that the product meets its current approved specifications
  29. § 820.100(a) for implementing corrective and preventive action
  30. § 820.120 to control labeling activities
  31. § 820.140 to ensure that mixups, damage, deterioration, contamination, or other adverse effects to product do not occur during handling
  32. § 820.150(a) for the control of storage areas and stock rooms for product to prevent mixups, damage, deterioration, contamination, or other adverse effects pending use or distribution and to ensure that no obsolete, rejected, or deteriorated product is used or distributed
  33. § 820.150(b) that describe the methods for authorizing receipt from and dispatch to storage areas and stock rooms
  34. § 820.160(a) for control and distribution of finished devices to ensure that only those devices approved for release are distributed and that purchase orders are reviewed to ensure that ambiguities and errors are resolved before devices are released for distribution
  35. § 820.170(a) for ensuring proper installation so that the device will perform as intended after installation
  36. § 820.184 to ensure that DHR’s for each batch, lot, or unit are maintained to demonstrate that the device is manufactured in accordance with the DMR and the requirements of this part
  37. § 820.198(a) for receiving, reviewing, and evaluating complaints by a formally designated unit
  38. § 820.200(a) for performing and verifying that the servicing meets the specified requirements
  39. § 820.250(a) for identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of process capability and product characteristics
  40. § 820.250(b) to ensure that sampling methods are adequate for their intended use and to ensure that when changes occur the sampling plans are reviewed

Targets Deconstructed

“Eliminate numerical goals, posters, and slogans for the work force, asking for new levels of productivity without providing methods.”

— Point No. 10 in Dr. W. E. Deming’s 14 points for management as written in “Quality, Productivity, and Competitive Position.”

A few weeks ago I had an excellent exchange on Twitter with UK Police Inspector Simon Guilfoyle on the topic of setting numerical targets. He asked “How do you set a numerical target without it being arbitrary? By what method?” Unfortunately, Twitter’s 140 character limit isn’t sufficient for adequately answering his question. I promised him I would write a post that explained my thinking.

When I was working for Samsung Austin Semiconductor (SAS) as a quality assurance engineer, one of my assigned responsibilities was to manage the factory’s overall nonconforming material rate. Over the course of my second year, the factory averaged a four percent* nonconforming material rate. The run chart for the monthly nonconforming material rate showed a stable system of variation.

As the year drew to a close, I began thinking about my goals for the following year. I knew I would continue to be responsible for managing the factory’s overall nonconforming material rate. What should I set as my target for it? Not knowing any better, I set it to be the rate we achieved for the current year: four percent. If nothing else, it was based on data. But my manager at the time, a Korean professional on assignment to the factory, mockingly asked me if I wasn’t motivated to do better. He set my target at two percent*; a fifty percent reduction.

What was the two percent number based on? How did he come about it? I had no insight and he didn’t bother to explain it either. From my perspective, it was an arbitrary numerical target; plucked out of thin air. I remember how incredibly nervous I felt about it. How was I going to achieve it? I had no clue nor guidance. I also remember how anxiety filled and frustrating the following year turned out for me. I watched the rate with a hawk eye. I hounded process engineers to do something whenever their process created a nonconforming lot. It was not a pleasant time for anyone.

Since then I’ve worked at several other companies in different industries. Nevertheless, my experience at SAS seems to be the norm when it comes to setting targets. This is regardless of the role, the industry or the culture. And, as far as I’ve been able to figure out, this approach to setting targets is driven more by tradition and arrogance than any objective thoughtful method. “Improve performance by 50% over last year!”, so the mantra goes. Worse still, no method is provided for achieving such arbitrary improvement targets. I’ve been told “You’re smart. You’ll figure out how to do it.”

So it’s not a surprise for me that folks like the good Inspector have become convinced all numerical targets are inherently arbitrary; that there is no objective and justifiable way to set them. Having been on the receiving end of such targets many times, I used to think the same, too. But just because people don’t know of a different way to set a target, one that is objective and can be justified, doesn’t mean there isn’t one. I believe numerical targets can be set in an objective fashion. It, however, requires thoughtfulness, great effort and understanding on the part of the person setting the target.

One way to set a target is to use the performance of a reference for comparison. In my case, the SAS factory I worked at had a sister facility in Korea. It would have been reasonable, albeit crude, to set my target for the nonconforming material rate to that achieved by the sister facility (if it was better.**) An argument could have been made that the target was achieved elsewhere, so it can be reached.

As part of our Twitter exchange, the Inspector made the point that regardless of whether these factories were defined to be sisters, there would still be differences between them. Therefore, they will generate a nonconforming material rate that is a function of their present system architecture. He is absolutely right! Setting a target for my factory based on the performance achieved by its sister facility alone will do nothing to improve the performance of my factory. It’s already doing the best it can.

But that’s not the point of setting the target: to operate the same system and expect an improved performance. The point of setting the target is to trigger a change in the system, a redesign in such a way as to achieve a level of performance that, in this case, has been achieved elsewhere. The sister system can be treated as a reference and studied. Differences between systems may be identified and eliminated. Along the way we may find out that some differences cannot be eliminated. Nevertheless, by eliminating the differences where possible the two systems are made more similar to one another and we will have improved the performance.

In the absence of a reference, simulations may be used to objectively define a target. The factory’s overall nonconforming material rate is the combined result of the nonconforming material rates of its individual processes. Investigating the performance of these inputs can help identify opportunities for improvement for each: stabilizing unstable processes, running stable processes on target, reducing the variability of stable on-target processes. All of this can be simulated to determine what is ideally possible. A justifiable target for the nonconforming material rate can then be set with the results. Best of all, the method by which it can be achieved gets defined as part of the exercise.

Finally, targets may be set by the state of the greater environment within which a system operates. All systems operate in a greater environment (e.g. national or global economy); one that is continuously changing in unpredictable ways. City populations grow or shrink. Markets grow or shrink. Polities combine or fragment. What we once produced to meet a demand will in a new environment prove to be too little or too much. A change in state of the external environment should trigger a change in the target of the system. A change in the target of the system should trigger a redesign of the system to achieve it. In Systems lingo, this is a tracking problem.

Targets are essential. They help guide the design or redesign of the system. They can be defined objectively in several different ways. I’ve outlined three above. They do not have to be set in the arbitrary way they currently are. But setting targets isn’t enough. Methods by which to achieve them must be defined. Targets, even objective ones, are meaningless and destructive without the means of achieving them. Failure to achieve targets should trigger an analysis into why the system failed. They should not be used to judge and blame workers within the system.

Sadly, people are like water, finding and using the path of least resistance. Setting arbitrary improvement targets is easier than doing all the work required to set objective ones. They have been successfully justified on the grounds of mindless ambition. No one questions the approach out of fear or ignorance. Positional authority is often used to mock or belittle the worker for not being motivated enough when the truth is something else: managerial ignorance and laziness to do their job.


* I’ve changed the numbers for obvious reasons. However, the message remains the same.

** As it turned out, the nonconforming material rate achieved at my factory was the best ever in all of Samsung!

 

Books

2017

        Total Design  the-diabolic why-a-guide-to-finding-and-using-causes


2016

the-human-use-of-human-beings spc-at-the-esquire-club pmbok quality-or-else the-drunkards-walk qbq Critical Chain Willpower Beyond the Stable State Deep Work Andy and Me The Power of Habit Give and Take Systems Thinking For Social Change Toyota Kata


2015

The Web Of Life The New Economics The Third Wave Six Easy Pieces Redesigning The Future Understanding Industrial Experimentation Deming Management Method Root Cause Analysis The Why Axis The Trust Factor The Man Who Discovered Quality What The Buddha Taught SpiritofKaizen Quality Wars 4thGenMgmt QPCP TheBonoboAndTheAtheist ManagerialBreakthrough Nonzero ATSPC TaaichiOhnosWorkplaceManagement Kaizen Confession of a Buddhist Atheist Future Shock Big Bang Longitude The Moral Animal The Black Swan Out of the Crisis OtherSideofInnovation Drive

Dealing with Nonconforming Product

A particular process makes parts of diameter D. There are 10 parts produced per batch. The batches are sampled periodically and the diameter of all the parts from the sampled batch is measured. Data, representing deviation from the target, for the first 6 sampled batches is shown in Table 1. The graph of the data is shown in Figure 1. Positive numbers indicate the measured diameter was larger than the target while negative numbers indicate the measured diameter was smaller than the target. The upper and lower specification limits for acceptable deviation are given as +/- 3.

NCM-Tab1

Table 1. Data for six batches of 10 parts each. The numbers represent the deviation from the target.

NCM-Fig1

Figure 1. Graph of the data from the table above. The most recent batch, batch number six, shows one part was nonconforming.

The most recent batch, sample batch number six, shows one of the 10 parts having a diameter smaller than the lower specification limit. As such, it is a nonconforming part.

The discovery of a nonconforming product triggers two parallel activities: i) figuring out what to do with the nonconforming product, and ii) addressing the cause of the nonconforming product to inhibit the nonconformance from occurring again.

PRODUCT DISPOSITION

Nonconforming product may be repaired or reworked when possible, but it can always be scrapped. Each one of these three options has its own set of complications and cost.

Repairing a nonconforming product involves additional steps beyond what are usually needed to make the product. This additional processing has the potential to create previously unknown weaknesses in the product e.g. stress concentrations. So repaired product will need to be subjected to testing that verifies it still satisfies its intended use. For this particular case, repairing is not possible. The diameter is smaller than the target. Repair would have been possible if the diameter had been larger than the target.

Reworking a nonconforming product involves undoing the results of the previous process steps, then sending the product through the standard process steps a second time. Undoing the results of the previous process steps involves additional process steps just as were required to repair a nonconforming product. This additional processing has the potential to create previously unknown weaknesses in the product. Reworked product will need to be subjected to testing that verifies it still satisfies its intended use. For this particular case, reworking is not possible.

Scrapping a nonconforming product means to destroy it so that it cannot be accidentally used. For this particular case, scrapping the nonconforming part is the only option available.

ADDRESSING THE CAUSE

In order to determine the cause of the nonconformity we have to first determine the state of the process i.e. whether the process is stable or not. The type of action we take depends on it.

A control chart provides a straightforward way to answer this question. Figure 2. shows an Xbar-R chart for this process. Neither the Xbar chart (top), nor the R chart (bottom) show uncontrolled variation. There is no indication of a special cause affecting the process. This is a stable process in the threshold state. While it is operating on target i.e. its mean is approximately the same as the target, its within-batch variation is more than we would like. Therefore, there is no point trying to hunt down a specific cause for the nonconforming part identified above. It is most likely the product of chance variation that affects this process; a result of the process’s present design.

NCM-Fig2

Figure 2. Xbar-R chart built using the first six sampled batches. Neither the Xbar chart nor the R chart show uncontrolled variation. There is no indication of a special cause affecting the process.

In fact, the process was left alone to collect more data (Figure 3.). The Xbar-R charts do not show any unusual variation that would indicate external disturbances affecting the process. Its behavior is predictable.

NCM-Fig3

Figure 3. More data was collected and the control limits were recalculated using the first 15 sampled batches. The process continues to look stable with no signs of external disturbance.

But, even though the process is stable, it does produce nonconforming parts from time to time. Figure 4. shows that a nonconforming part was produced in sampled batch number 22 and one in sampled batch number 23. Still, it would be wasted effort to hunt down specific causes for the creation of these nonconforming parts. They are the result of chance variation that is a property of the present process design.

NCM-Fig4

Figure 4. Even though the process is stable it still occasionally produces nonconforming parts. Sampled batch number 22 shows a nonconforming part with a larger than tolerable diameter while sampled batch number 23 shows one with a smaller than tolerable diameter.

Because this process is stable, we can estimate the mean and standard deviation of the distribution of individual parts. They were calculated to be -0.0114 and 0.9281. Assuming that the individual parts are normally distributed, we can estimate that this process will produce about 0.12% nonconforming product if left to run as is. Some of these parts will be smaller than the lower specification limit for the diameter. Others will be larger than the upper specification limit for the diameter. That is, about 12 nonconforming pieces will be created per 10,000 parts produced. Is this acceptable?

If the calculated nonconforming rate is not acceptable, then this process must be modified in some fundamental way. This would involves some sort of structured experimentation using methods from design of experiments to reduce variation. New settings for factors like RPM or blade type among others will need to be determined.

Entropy and Assignable Causes

Left alone, a process in the Ideal State will migrate toward the State of Chaos over time. Wear and tear of parts in use will lead to breakdowns and failures. Even mothballed machines cannot escape deterioration and decay caused by the environment. Entropy affects everything.

However, the effects of entropy can be repaired. Signatures of deterioration and failure made apparent by process control charts provide clues as to what repairs need to be made when. But, a process isn’t affected just by entropy. Other assignable causes keep it from operating in a predictable fashion. Replacing worn out parts might pull a process out from the State of Chaos back to the Brink of Chaos, but it’s just a matter of time before it returns to it.

Process managers, through the proper use of process control charts, must counteract the effects of entropy and assignable causes to help their processes achieve the ideal state and stay there. This is a never ending cycle.

Note: I learned this material from reading Dr. Wheeler’s writings. My post is intended to reflect my current understanding. None of the ideas herein are original to me. Any errors are my failures alone.

References

The State of Chaos

When a process is out of control and it is producing nonconforming product it is in a state of chaos. The State of Chaos is one of the four states a process can be in as shown in “What State Is Your Process In?“. The manufacturer cannot predict how much nonconforming product his process will produce in any given hour or day. At times the process will produce nothing but conforming product. Then without warning it will produce nothing but nonconforming product. It might seem as if there were ghosts in the machine.

A process in such a state is affected by assignable causes that are easily identified through the use of process control charts. The effects of these assignable causes have to be eliminated one at a time. Patience and perseverance are necessary. It is essential that the process be brought under statistical control and made predictable. Once the process has achieved stability further improvement efforts can be made to reach the ideal state.

ChaosProcess

Note: I learned this material from reading Dr. Wheeler’s writings. My post is intended to reflect my current understanding. None of the ideas herein are original to me. Any errors are my failures alone.

References