Some Observations and Thoughts on Design Controls

In my role as a quality engineer supporting product design and development at various medical device manufacturers I got practical experience with each company’s design and development process. As a matter of regulation[1], each medical device manufacturer has procedures that control the design of their products. Unfortunately, they are not particularly useful.

I’ve observed that the Quality function at these companies develops and deploys all the procedures that the Quality System regulations require[2]. However, professionals in the Quality function typically don’t have the subject matter expertise in a particular function such as product design and development or manufacturing to develop usable procedures for that function.

Here I share an example product design and development procedure typical of those I have seen deployed:

This type of process, laid out in the order of the text of the regulation, would suggest that product design and development is a sequence of steps executed in series.

At first glance it seems logical and sensible. First you catalog the user needs. Next you convert those user needs into design inputs (i.e. engineering requirements.) You then transform the design inputs through the design process into design outputs (i.e. drawings or prototypes.) Those design outputs are then verified (i.e. inspected and tested) against the design inputs. After that the design is validated by the user in the actual or simulated use environment. And finally, the design is transferred to manufacturing for mass production.

It wrongly suggests, albeit implicitly, that these steps also represent phases of design and development where a review is conducted after each block, and that a single traceability matrix, with columns corresponding to each block, is enough to capture the activity of the total design effort.

I have tried to figure out how this would work for a design involving multiple components that are assembled together, but I cannot find a way. This type of design for the product design and development process is fatally flawed as it doesn’t model the real nature of products which is often components/systems embedded within systems. Trying to map the total design effort into this format is like trying to fit a square peg in a round hole, an impossible and ultimately frustrating exercise.

Just because language is linear, in that ideas are expressed one after the other as the regulation does, doesn’t mean that the process being described is linear, too. In fact, the design and development process is most certainly not linear. It is deeply iterative with iterations built within iterations!

The FDA’s “Design Control Guidance for Medical Device Manufacturers”[3] provides an explanation of the iterative nature of the design and development process. The guidance includes a simplified process flow chart, but it does not adequately communicate the complexity that makes up the actual design and development process. The guidance even explicitly says so.

In practice, feedback paths would be required between each phase of the process and previous phases, representing the iterative nature of product development. However, this detail has been omitted from the figure…

The language of the guidance in the above paragraph unfortunately implies that each block of the waterfall design process is a phase. It clarifies this further on where it says:

When the design input has been reviewed and the design input requirements are determined to be acceptable, an iterative process of translating those requirements into a device design begins. The first step is conversion of the requirements into system or high-level specifications. Thus, these specifications are a design output. Upon verification that the high-level specifications conform to the design input requirements, they become the design input for the next step in the design process, and so on.

This basic technique is used throughout the design process. Each design input is converted into a new design output; each output is verified as conforming to its input; and it then becomes the design input for another step in the design process. In this manner, the design input requirements are translated into a device design conforming to those requirements.

While the regulation does not prescribe a method for designing and developing a product, the guidance does point in a particular direction. The best representation I could find that captures the direction in the guidance is this graphic adapted from “The House of Quality” by John Hauser and Don Clausing[4]:

The first “house” shows the “conversion of the requirements [Customer attributes] into system or high-level specifications [Engineering characteristics]”. The body of the house allows for the verification that “high-level specifications conform to the design input requirements”. The engineering characteristics then “become the design input for the next step in the design process, and so on.

It’s obvious from the linked houses and the guidance that verification is not a one time or single type of activity. It is performed at each step of the design and development process wherever inputs are converted to outputs. Implicit in this point is that the type of verification is unique to the particular step or phase of the design and development process.

Each house may be thought of as a phase of the design and development process. The houses offer natural breaks. The design process of the next phase, converting inputs into outputs, depends on the successful completion of the previous phase, so it is nearly impossible to move too far down the process as gaps will be immediately apparent!

Each house can be considered its own traceability matrix where every design output is tied to one or more design inputs. And because the houses are all linked to one another it is possible to trace an attribute of the manufactured product all the way back to the customer need it helps address.

While they may not have a firm conceptual understanding of the design and development process, and thus cannot explain it, I believe most engineers have an instinctual feel for it in practice. But a poorly designed design and development process creates unnecessary and insoluble problems for project teams. The teams I’ve been on have responded to such hurdles by running two parallel processes: one that is the practical design and development effort, and the other is the documentation effort—a hidden factory. I don’t think it’s possible to calculate the cost of such waste.

Links
[1] 21 CFR Part 820.30 (a) https://www.ecfr.gov/cgi-bin/text-idx?SID=a018454b01dab73d0d1cef9f95be36a9&mc=true&node=pt21.8.820&rgn=div5#se21.8.820_130 Retrieved 2017-07-05
[2] QSR Required Procedures https://shrikale.wordpress.com/2017/05/18/qsr-required-procedures/ Retrieved 2017-07-05
[3] Design Control Guidance For Medical Device Manufacturers https://www.fda.gov/RegulatoryInformation/Guidances/ucm070627.htm Retrieved 2017-07-05
[4] The House of Quality. Harvard Business Review, pages 63-77, Vol 66 No 3, May 1988.
[5] Product Design and Development, 5th Edition. McGraw Hill, 2016.

Advertisements

Whose Measurement is Right?

Every company I’ve worked for inspects the product it receives from its suppliers to determine conformance to requirements. The process is variously referred to as incoming inspection or receiving inspection.

Sometimes the receiving inspection process identifies a lot of product that fails to conform to requirements. That lot is subsequently classified as nonconforming material and quarantined for further review. There are many reasons why a lot of product may be classified as nonconforming. Here I wish to focus just on reasons having to do with measurement.

Once a company discovers nonconforming parts, it usually contacts its supplier to share that information. It is not unusual, however, for the supplier to push back when their data for the lot of product shows it to be conforming. So, how can a given lot of product be both conforming and nonconforming? Who is right?

We need to recognize that measurement is a process. The measured value is an outcome of this process. It depends on the measurement tool used, the skill of the person making the measurement and the steps of the measurement operation. A difference in any of these factors will show up as a difference in the measured value.

It is rare that a measurement process is the same between a customer and its supplier. A customer may use different measurement tools than its supplier. For example, where the customer might have used a caliper or micrometer, the supplier may have used an optical comparator or CMM. Even if both the customer and the supplier use the same measurement tool, the workers using that tool are unlikely to have been trained in its use in the same way. Finally, the steps used to make the measurement, such as fixturing, lighting and handling the part, which often depend on the measurement tool used, will likely be different, too.

Thus, more often than not, a measurement process between a supplier and a customer will be different. Each measured value is correct in its context—the supplier’s measurement is correct in its context, as is the customer’s measurement in its context. But because the measurement process is different between the two contexts, the measured values cannot be compared directly with one another. So it is possible that the same lot of product may be conforming per the supplier’s measurements and nonconforming per the customer’s measurements.

But why are we measuring product twice: once by the supplier and again by the customer? Measurement is a costly non-value adding operation, and doing it twice is excess processing–wasteful. One reason I’ve been told is this is done to confirm the data provided by the supplier. But confirmation is possible only if the measurement process used by the customer matches the one used by the supplier.

Besides, if we are worried about the quality of supplier data, we should then focus efforts on deeply understanding their measurement process, monitoring its stability and capability, and working with the supplier to improve it if necessary. With that we should trust the measurement data the supplier provides and base our decisions on it, and eliminate the duplicate measurement step during receiving inspection.

Links
[1] Eliminate Waste in Incoming Inspection: 10 ideas of where to look for waste in your process http://www.qualitymag.com/articles/92832-eliminate-waste-in-incoming-inspection Retrieved 2017-06-29

Above All, Don’t Wobble

In walking, just walk. In sitting, just sit. Above all, don’t wobble.
– Yúnmén

The companies I’ve worked for have been neurotic. They dither. When decisions are made they have an irrational and anxious quality about them.

My experience of work can be described as a shuddering paralysis. In an effort to take everything into account teams I’ve been on enter into an infinite regression of analysis that often takes us off course, delaying action. (I have been guilty of contributing to this.) However, the essence of a business is to act, to do.

When we do act, we don’t just act, but worry about whether that action is the best possible; we complain about all the flaws we find in the method; we even wonder whether the goal is the right goal. So our attention is split, bouncing between acting and thinking. Instead of moving gracefully toward our goal, we wobble. I wobble.

Perhaps Yúnmén wouldn’t mind if I rephrased his quote as “In planning, just plan. In doing, just do. Above all, don’t wobble.”

Retraining Can’t Fix This

In the course of an average workday we make hundreds of decisions. Some of those decisions require engaging our conscious awareness. In my previous post I described how the quality of those decisions deteriorate as that awareness or willpower fatigues with use.

However, there are decisions where human error occurs with certainty even if our attention is totally focused on the task. Consider the Muller-Lyer[1] illusion below:

The two vertical lines are of the same length. Even after knowing this, we all continue to perceive the line on the left to be longer than the line on the right. The “fact” that the two lines are of different lengths is simply obvious to us. Because of its obviousness we don’t stop to check our judgment before acting on it. Such actions, based on erroneous perception, are likely to produce faulty outcomes.

This error in our human perception/cognition system is hard-wired into our brains. No amount of retraining or conscious effort will correct it. So corrective actions that identify retraining as the way to prevent recurrence of this type of error won’t be effective. It will only serve to demoralize the worker. What, then, is an effective corrective action for such errors?

We can develop and use tools and methods that circumvent the brain’s perception/cognition system, for example with an overlay (red lines in the figure below), or actually measuring each line and comparing those values to one another. This does add a step to the evaluation process; an after-the-fact fix to a faulty design. Ideally, though, we would want our designs to take into account human limitations and avoid creating such illusions in the first place.

Links
[1] Muller-Lyer illusion https://en.wikipedia.org/wiki/Muller-Lyer_illusion Retrieved 2017-06-22

Human Error

Often, investigators identify the root cause of a problem as human error. But what exactly is human error?

An action may be judged as an error only in relation to a reference or standard. So first a standard on how to perform the task must exist. Sometimes such a standard is defined in a documented procedure. On occasion it may also be taught by a master to an apprentice on the job. Most times we just figure it out through a combination of past experience, current observations, and some fiddling. Human error, then, is action by a human that deviates from the standard.

When we judge the root cause of a problem as human error we’re making certain assumptions: 1] that a standard exists, and 2] the standard, if it exists, is adequate to the degree that mindfully following it produces the expected outcome.

Let’s grant that both the above assumptions are true, and even grant that the root cause of a problem was the failure of the worker to follow the standard. What, then, should the corrective action be that will prevent the recurrence of the problem? In my experience it has almost always been defined as “retraining”. But such a corrective action assumes that the worker failed to follow the standard because they don’t know it. Is this true? If not, retraining is pure waste and won’t do a damn thing to prevent the recurrence of the problem.

If a proper standard exists and the worker has been trained to it, then there must be some other reason for their failure to follow it. Skill-based errors (i.e. slips and lapses) can occur when the worker is unable to pay attention to or focus on performing the task they are otherwise familiar with. So it’s not a training issue. In my previous post I wrote about how willpower, our conscious awareness, is like a muscle. It can fatigue from use. As willpower is depleted the mind resorts to mental shortcuts or habits. This is how errors creep in.

We should not rely only on our ability to remain attentive and focused to ensure that the task is performed without failure. For that we must design tasks in such a way that failure is unlikely, if not impossible, to occur. Through design thinking we can develop tools, methods, and systems that help us perform better.

Links
[1] Understanding human failure. http://www.hse.gov.uk/construction/lwit/assets/downloads/human-failure.pdf Retrieved 2017-06-15

Personal Willpower, Communal Impact

Meditators

We seem to make decisions in more impulsive ways than before. Many of us don’t seem to practice any reasonable amount of self-control. I feel this may be because most of us today just don’t have strong willpower.

Last year I read a book called “Willpower”[1] by Roy F. Baumeister and John Tierney. In it the authors liken willpower to muscle. And just like a muscle willpower can wear out from fatigue. When willpower is worn out, we behave more impulsively. How quickly we drain our willpower depends on how strong it is.

In using our willpower to make decisions we’re using our conscious mind or “System 2” as Daniel Kahneman refers to it in “Thinking, Fast and Slow”[2]. Conscious decision making or thinking is hard! It requires effort and uses a lot of energy in the process.

The body, however, has a limited store of energy. When we are low on energy, this conscious decision making process shuts down and decision making is shunted to the brain’s default decision making process or “System 1.” It doesn’t require much energy; it’s automatic and occurs outside of our conscious awareness. Many of the decisions we make in the default mode are driven by habit.

Conscious decision making generally produces reliable outcomes. We make better decisions with it. Not so with automatic decision making, which has been shown to be error-prone, often in systematic ways. So it’s important that we exercise our willpower; build it up, and make it stronger.

No one can make you exercise your body or mind. That’s a choice you make for yourself. But the results of your choice affects your behavior which in turn affects society. We live in communities and we have an obligation to them: to be the best version of ourselves.

Links
[1] Baumeister, Roy F., John Tierney (2012). Willpower: Rediscovering the Greatest Human Strength.
[2] Kahneman, Daniel (2011). Thinking, Fast and Slow.

The Context for Concepts

In my last post I might have left the impression that conceptualizing the real place is bad or that we should avoid it. This is not a correct impression.

We cannot avoid conceptualizing the real place. It’s automatic; part of our biological structure and the structure of our language. Concepts are how we make sense of the real place. They provide insights into the real place. We need those insights to respond appropriately to the real place. But we shouldn’t lose sight of the fact that concepts are the mind’s representations of the real place and not the real place itself! We can call them images, idols, models, data, or symbols.

D. T. Suzuki[1] shared, “To point at the moon a finger is needed, but woe to those who take the finger for the moon…” Alfred Korzybski[2] wrote in Science and Sanity, “A map is not the territory it represents, but, if correct, it has a similar structure to the territory, which accounts for its usefulness.” George E. P. Box[3], in Statistics for Experimenters, put it pithily that “all models are wrong; some models are useful.” These reminders, to be consciously aware of the difference between the real place and our mind’s abstractions of it, is the thread that runs through science and religion.

Problems only arise when we hold onto a concept long after it has stopped representing the real place and a gap has developed between what is and what we conceptualize it to be. To know what is, we must first “go and see” the real place. Without that direct experience with the real place, we cannot hope to act in ways appropriate to it. This is my understanding of what Zen and lean teach.

Links
[1] D.T. Suzuki
[2] Alfred Korzybski
[3] George E.P. Box

The Real Place

My study of Buddhist thought, and especially Zen, have so far taught me that I am often unaware of the real place. Decades of schooling and acculturation to society have taught me to ignore the real place in favor of concepts manufactured by the human mind; to create and be hypnotized by images and models. Right, wrong, god, devil, me, you, husband, wife, mother, father, boss, servant, friend, enemy, success, failure, good, bad, us, and them are all concepts. These are all creations of the mind. It gives them meaning. They’re not real.

Concepts are static–unchanging and easy to grab a hold of and cling to, while the real place is dynamic–ever changing; sometimes in predictable ways, most times in unpredictable ways. The real place offers nothing to grab on to; nothing to cling to. It is inevitable then that the two will eventually diverge from one another. I believe that that gap between what I see and what I think I see is the source of much, if not all, my suffering–frustration, anxiety, feelings of helplessness, exhaustion, and such. To experience the real place, I must let go of concepts, or rather I should not cling to them. Only then will my actions be appropriate or right for the real place.

Zen has been useful in ferrying me back to the real place every time my mind drifts to concepts.

My most direct experience of this gap, or at least one that I am most aware of, has been in the workplace. Data, charts, procedures, policies, concepts abound. Again, most, if not all, are disconnected from the real processes and systems. How work actually happens. However, like me, organizations remain mostly unaware of the disconnect. They thus suffer in a mire of internal conflict and frustration, too.

Lean can be useful to get organizations back to the real place.

PDCA During Product Development

I used the concept of the PDCA cycle (below) during a few new product introduction projects. The teams realized many tangible and intangible benefits from it.

Plan  The engineering concept for a part is converted into a detailed drawing. It provides a graphic representation of the part along with all its engineering requirements/specifications. Among other things, it defines the geometry, dimensions, tolerances, and material for the physical part. The drawing of the part acts as the plan for the manufacturer to follow when making the physical part.

Do  The manufacturer uses the drawing of the part (plan) to make the physical part (do).

(Note: If you outsource the manufacturing of the part, lead times could be as much as 14 weeks or 3+ months. So, it’s a good idea to involve the manufacturer in the planning phase of the part to address any foreseeable issues as early as possible.)

Check  The physical part is inspected (checked) against the drawing of the part (plan) e.g. as part of a first article inspection (FAI) or receiving inspection. Discrepancies between the physical part and its drawing are identified.

Act  Decisions are made for each discrepancy to determine whether the part must comply with the existing drawing specifications or whether the drawing specifications—typically the tolerances around an attribute—should be changed.

If it’s decided that the drawing specifications are to be changed, e.g. the tolerances for one or more attributes is to be loosened, then the drawing is revised. This results in another loop through the PDCA cycle with the new drawing or plan.

Quality People, Stop Holding Product Hostage

Any process will inevitably generate nonconforming product at some point in its operation. Companies typically define the method for handling nonconforming product in a formal procedure that is part of their quality management system. As part of such a procedure, when nonconforming product is discovered, usually during the inspection step, it is quarantined. Two separate but related questions must then be answered: 1] What do we do with the nonconforming product? and 2] How do we prevent it from recurring?

I have observed a troubling pattern across multiple companies in how their quality control professionals are managing nonconforming product. They are holding it hostage—not allowing it to flow after it has been properly dispositioned—in order to compel others to comply with the other requirements of the formal procedure for handling nonconforming product. Specifically, the requirement to determine the root cause of the nonconformity and put in place countermeasures to prevent its recurrence.

I have also identified several reasons why quality control personnel are taking this counterproductive approach. Almost all feel, with good reason, that without it they cannot comply with all the requirements of the formal procedure or the standards and regulations they are designed to meet. They don’t believe that their coworkers are intrinsically motivated to take ownership for investigating the cause of the nonconformity and putting in place the appropriate countermeasures. Nor do they believe that they are externally incentivized to do so. And, of course, there are some quality control personnel who use this tactic to assert themselves in an environment that they feel otherwise does not respect them.

The effect of such behavior though is to reinforce the perception non-quality people have that quality professionals create blocks or bureaucratic hurdles instead of working with others to support the company’s objectives by helping to improve process and product quality. I wonder whether quality control personnel are aware that nonconforming product is still counted as inventory, and holding properly dispositioned nonconforming product hostage has a myriad unintentional consequences like lost sales, inaccurate accounting of assets, using up precious storage space, wasted manpower to monitor and manage the material, etc.

When people do give in to such arm-twisting, they do it with resentment, to meet a quality function demand, and not because they see the value in fixing the process. This is a pyrrhic victory. Pressured, resentful and motivated by the wrong goal, how thorough or accurate can their root cause investigation be? Countermeasures developed in response to sloppy cause analysis will at best address the symptoms of the nonconforming product. So recurrence is all but assured! And it’s quiet possible that these countermeasures may destabilize a process and increasing its variation leading to the creation of more nonconforming product.

Holding properly dispositioned nonconforming product hostage is not the right way to improve the performance of the nonconforming product handling process. Not only are you not adding value by doing that, your actions are costing the company. So please stop doing that. There are other better ways to improve the performance of quality system processes that support the company objectives.