In the course of an average workday we make hundreds of decisions. Some of those decisions require engaging our conscious awareness. In my previous post I described how the quality of those decisions deteriorate as that awareness or willpower fatigues with use.
However, there are decisions where human error occurs with certainty even if our attention is totally focused on the task. Consider the Muller-Lyer illusion below:
The two vertical lines are of the same length. Even after knowing this, we all continue to perceive the line on the left to be longer than the line on the right. The “fact” that the two lines are of different lengths is simply obvious to us. Because of its obviousness we don’t stop to check our judgment before acting on it. Such actions, based on erroneous perception, are likely to produce faulty outcomes.
This error in our human perception/cognition system is hard-wired into our brains. No amount of retraining or conscious effort will correct it. So corrective actions that identify retraining as the way to prevent recurrence of this type of error won’t be effective. It will only serve to demoralize the worker. What, then, is an effective corrective action for such errors?
We can develop and use tools and methods that circumvent the brain’s perception/cognition system, for example with an overlay (red lines in the figure below), or actually measuring each line and comparing those values to one another. This does add a step to the evaluation process; an after-the-fact fix to a faulty design. Ideally, though, we would want our designs to take into account human limitations and avoid creating such illusions in the first place.
 Muller-Lyer illusion https://en.wikipedia.org/wiki/Muller-Lyer_illusion Retrieved 2017-06-22
An action may be judged as an error only in relation to a reference or standard. So first a standard on how to perform the task must exist. Sometimes such a standard is defined in a documented procedure. On occasion it may also be taught by a master to an apprentice on the job. Most times we just figure it out through a combination of past experience, current observations, and some fiddling. Human error, then, is action by a human that deviates from the standard.
When we judge the root cause of a problem as human error we’re making certain assumptions: 1] that a standard exists, and 2] the standard, if it exists, is adequate to the degree that mindfully following it produces the expected outcome.
Let’s grant that both the above assumptions are true, and even grant that the root cause of a problem was the failure of the worker to follow the standard. What, then, should the corrective action be that will prevent the recurrence of the problem? In my experience it has almost always been defined as “retraining”. But such a corrective action assumes that the worker failed to follow the standard because they don’t know it. Is this true? If not, retraining is pure waste and won’t do a damn thing to prevent the recurrence of the problem.
If a proper standard exists and the worker has been trained to it, then there must be some other reason for their failure to follow it. Skill-based errors (i.e. slips and lapses) can occur when the worker is unable to pay attention to or focus on performing the task they are otherwise familiar with. So it’s not a training issue. In my previous post I wrote about how willpower, our conscious awareness, is like a muscle. It can fatigue from use. As willpower is depleted the mind resorts to mental shortcuts or habits. This is how errors creep in.
We should not rely only on our ability to remain attentive and focused to ensure that the task is performed without failure. For that we must design tasks in such a way that failure is unlikely, if not impossible, to occur. Through design thinking we can develop tools, methods, and systems that help us perform better.
 Understanding human failure. http://www.hse.gov.uk/construction/lwit/assets/downloads/human-failure.pdf Retrieved 2017-06-15
We seem to make decisions in more impulsive ways than before. Many of us don’t seem to practice any reasonable amount of self-control. I feel this may be because most of us today just don’t have strong willpower.
Last year I read a book called “Willpower” by Roy F. Baumeister and John Tierney. In it the authors liken willpower to muscle. And just like a muscle willpower can wear out from fatigue. When willpower is worn out, we behave more impulsively. How quickly we drain our willpower depends on how strong it is.
In using our willpower to make decisions we’re using our conscious mind or “System 2” as Daniel Kahneman refers to it in “Thinking, Fast and Slow”. Conscious decision making or thinking is hard! It requires effort and uses a lot of energy in the process.
The body, however, has a limited store of energy. When we are low on energy, this conscious decision making process shuts down and decision making is shunted to the brain’s default decision making process or “System 1.” It doesn’t require much energy; it’s automatic and occurs outside of our conscious awareness. Many of the decisions we make in the default mode are driven by habit.
Conscious decision making generally produces reliable outcomes. We make better decisions with it. Not so with automatic decision making, which has been shown to be error-prone, often in systematic ways. So it’s important that we exercise our willpower; build it up, and make it stronger.
No one can make you exercise your body or mind. That’s a choice you make for yourself. But the results of your choice affects your behavior which in turn affects society. We live in communities and we have an obligation to them: to be the best version of ourselves.
 Baumeister, Roy F., John Tierney (2012). Willpower: Rediscovering the Greatest Human Strength.
 Kahneman, Daniel (2011). Thinking, Fast and Slow.
In my last post I might have left the impression that conceptualizing the real place is bad or that we should avoid it. This is not a correct impression.
We cannot avoid conceptualizing the real place. It’s automatic; part of our biological structure and the structure of our language. Concepts are how we make sense of the real place. They provide insights into the real place. We need those insights to respond appropriately to the real place. But we shouldn’t lose sight of the fact that concepts are the mind’s representations of the real place and not the real place itself! We can call them images, idols, models, data, or symbols.
D. T. Suzuki shared, “To point at the moon a finger is needed, but woe to those who take the finger for the moon…” Alfred Korzybski wrote in Science and Sanity, “A map is not the territory it represents, but, if correct, it has a similar structure to the territory, which accounts for its usefulness.” George E. P. Box, in Statistics for Experimenters, put it pithily that “all models are wrong; some models are useful.” These reminders, to be consciously aware of the difference between the real place and our mind’s abstractions of it, is the thread that runs through science and religion.
Problems only arise when we hold onto a concept long after it has stopped representing the real place and a gap has developed between what is and what we conceptualize it to be. To know what is, we must first “go and see” the real place. Without that direct experience with the real place, we cannot hope to act in ways appropriate to it. This is my understanding of what Zen and lean teach.
My study of Buddhist thought, and especially Zen, have so far taught me that I am often unaware of the real place. Decades of schooling and acculturation to society have taught me to ignore the real place in favor of concepts manufactured by the human mind; to create and be hypnotized by images and models. Right, wrong, god, devil, me, you, husband, wife, mother, father, boss, servant, friend, enemy, success, failure, good, bad, us, and them are all concepts. These are all creations of the mind. It gives them meaning. They’re not real.
Concepts are static–unchanging and easy to grab a hold of and cling to, while the real place is dynamic–ever changing; sometimes in predictable ways, most times in unpredictable ways. The real place offers nothing to grab on to; nothing to cling to. It is inevitable then that the two will eventually diverge from one another. I believe that that gap between what I see and what I think I see is the source of much, if not all, my suffering–frustration, anxiety, feelings of helplessness, exhaustion, and such. To experience the real place, I must let go of concepts, or rather I should not cling to them. Only then will my actions be appropriate or right for the real place.
Zen has been useful in ferrying me back to the real place every time my mind drifts to concepts.
My most direct experience of this gap, or at least one that I am most aware of, has been in the workplace. Data, charts, procedures, policies, concepts abound. Again, most, if not all, are disconnected from the real processes and systems. How work actually happens. However, like me, organizations remain mostly unaware of the disconnect. They thus suffer in a mire of internal conflict and frustration, too.
Lean can be useful to get organizations back to the real place.
Plan The engineering concept for a part is converted into a detailed drawing. It provides a graphic representation of the part along with all its engineering requirements/specifications. Among other things, it defines the geometry, dimensions, tolerances, and material for the physical part. The drawing of the part acts as the plan for the manufacturer to follow when making the physical part.
Do The manufacturer uses the drawing of the part (plan) to make the physical part (do).
(Note: If you outsource the manufacturing of the part, lead times could be as much as 14 weeks or 3+ months. So, it’s a good idea to involve the manufacturer in the planning phase of the part to address any foreseeable issues as early as possible.)
Check The physical part is inspected (checked) against the drawing of the part (plan) e.g. as part of a first article inspection (FAI) or receiving inspection. Discrepancies between the physical part and its drawing are identified.
Act Decisions are made for each discrepancy to determine whether the part must comply with the existing drawing specifications or whether the drawing specifications—typically the tolerances around an attribute—should be changed.
If it’s decided that the drawing specifications are to be changed, e.g. the tolerances for one or more attributes is to be loosened, then the drawing is revised. This results in another loop through the PDCA cycle with the new drawing or plan.
Any process will inevitably generate nonconforming product at some point in its operation. Companies typically define the method for handling nonconforming product in a formal procedure that is part of their quality management system. As part of such a procedure, when nonconforming product is discovered, usually during the inspection step, it is quarantined. Two separate but related questions must then be answered: 1] What do we do with the nonconforming product? and 2] How do we prevent it from recurring?
I have observed a troubling pattern across multiple companies in how their quality control professionals are managing nonconforming product. They are holding it hostage—not allowing it to flow after it has been properly dispositioned—in order to compel others to comply with the other requirements of the formal procedure for handling nonconforming product. Specifically, the requirement to determine the root cause of the nonconformity and put in place countermeasures to prevent its recurrence.
I have also identified several reasons why quality control personnel are taking this counterproductive approach. Almost all feel, with good reason, that without it they cannot comply with all the requirements of the formal procedure or the standards and regulations they are designed to meet. They don’t believe that their coworkers are intrinsically motivated to take ownership for investigating the cause of the nonconformity and putting in place the appropriate countermeasures. Nor do they believe that they are externally incentivized to do so. And, of course, there are some quality control personnel who use this tactic to assert themselves in an environment that they feel otherwise does not respect them.
The effect of such behavior though is to reinforce the perception non-quality people have that quality professionals create blocks or bureaucratic hurdles instead of working with others to support the company’s objectives by helping to improve process and product quality. I wonder whether quality control personnel are aware that nonconforming product is still counted as inventory, and holding properly dispositioned nonconforming product hostage has a myriad unintentional consequences like lost sales, inaccurate accounting of assets, using up precious storage space, wasted manpower to monitor and manage the material, etc.
When people do give in to such arm-twisting, they do it with resentment, to meet a quality function demand, and not because they see the value in fixing the process. This is a pyrrhic victory. Pressured, resentful and motivated by the wrong goal, how thorough or accurate can their root cause investigation be? Countermeasures developed in response to sloppy cause analysis will at best address the symptoms of the nonconforming product. So recurrence is all but assured! And it’s quiet possible that these countermeasures may destabilize a process and increasing its variation leading to the creation of more nonconforming product.
Holding properly dispositioned nonconforming product hostage is not the right way to improve the performance of the nonconforming product handling process. Not only are you not adding value by doing that, your actions are costing the company. So please stop doing that. There are other better ways to improve the performance of quality system processes that support the company objectives.
I have identified 40 procedures required by 21 CFR 820: the Quality System Regulation
- § 820.22 for quality audits…
- § 820.25(b) for identifying training needs…
- § 820.30(a)(1) to control the design of the device in order to ensure that specified design requirements are met
- § 820.30(c) to ensure that the design requirements relating to a device are appropriate and address the intended use of the device, including the needs of the user and patient
- § 820.30(d) for defining and documenting design output in terms that allow an adequate evaluation of conformance to design input requirements
- § 820.30(e) to ensure that formal documented reviews of the design results are planned and conducted at appropriate stages of the device’s design development
- § 820.30(f) for verifying the device design
- § 820.30(g) for validating the device design
- § 820.30(h) to ensure that the device design is correctly translated into production specifications
- § 820.30(i) for the identification, documentation, validation or where appropriate verification, review, and approval of design changes before their implementation
- § 820.40 to control all documents that are required by this part
- § 820.50 to ensure that all purchased or otherwise received product and services conform to specified requirements
- § 820.60 for identifying product during all stages of receipt, production, distribution, and installation to prevent mixups
- § 820.65 for identifying with a control number each unit, lot, or batch of finished devices and where appropriate components
- § 820.70(a) that describe any process controls necessary to ensure conformance to specifications
- § 820.70(b) for changes to a specification, method, process, or procedure
- § 820.70(c) to adequately control these environmental conditions
- § 820.70(e) to prevent contamination of equipment or product by substances that could reasonably be expected to have an adverse effect on product quality
- § 820.70(h) for the use and removal of such manufacturing material to ensure that it is removed or limited to an amount that does not adversely affect the device’s quality
- § 820.72(a) to ensure that equipment is routinely calibrated, inspected, checked, and maintained
- § 820.75(b) for monitoring and control of process parameters for validated processes to ensure that the specified requirements continue to be met
- § 820.80(a) for acceptance activities
- § 820.80(b) for acceptance of incoming product
- § 820.80(c) to ensure that specified requirements for in-process product are met
- § 820.80(d) for finished device acceptance to ensure that each production run, lot, or batch of finished devices meets acceptance criteria
- § 820.90(a) to control product that does not conform to specified requirements
- § 820.90(b)(1) that define the responsibility for review and the authority for the disposition of nonconforming product
- § 820.90(b)(2) for rework, to include retesting and reevaluation of the nonconforming product after rework, to ensure that the product meets its current approved specifications
- § 820.100(a) for implementing corrective and preventive action
- § 820.120 to control labeling activities
- § 820.140 to ensure that mixups, damage, deterioration, contamination, or other adverse effects to product do not occur during handling
- § 820.150(a) for the control of storage areas and stock rooms for product to prevent mixups, damage, deterioration, contamination, or other adverse effects pending use or distribution and to ensure that no obsolete, rejected, or deteriorated product is used or distributed
- § 820.150(b) that describe the methods for authorizing receipt from and dispatch to storage areas and stock rooms
- § 820.160(a) for control and distribution of finished devices to ensure that only those devices approved for release are distributed and that purchase orders are reviewed to ensure that ambiguities and errors are resolved before devices are released for distribution
- § 820.170(a) for ensuring proper installation so that the device will perform as intended after installation
- § 820.184 to ensure that DHR’s for each batch, lot, or unit are maintained to demonstrate that the device is manufactured in accordance with the DMR and the requirements of this part
- § 820.198(a) for receiving, reviewing, and evaluating complaints by a formally designated unit
- § 820.200(a) for performing and verifying that the servicing meets the specified requirements
- § 820.250(a) for identifying valid statistical techniques required for establishing, controlling, and verifying the acceptability of process capability and product characteristics
- § 820.250(b) to ensure that sampling methods are adequate for their intended use and to ensure that when changes occur the sampling plans are reviewed
While I have worked with hundreds of people, I can think of only a few who cared about what they did. The care they exercised showed in the superior quality of their work product. Most, however, don’t seem to show any care for the quality of what they produce. Tasks are treated as hot potatoes, to be shoved off to someone else as quickly as possible.
Doing something with care requires you to pay attention to the task and to be mindful of its context. That, in my experience, is rare. What I do know is that when the right actions are done right the outcome is most assuredly a quality product. There is a certain beauty and elegance about it.
Some pay great attention to getting the details of a task right while being oblivious to whether the task is appropriate for the context. Others may do the task appropriate for the context, but fail to pay attention to getting it right. Both of these failures generate poor quality outcomes that cause tremendous frustrations for people on the receiving end.
Caring about your work doesn’t mean you love what you do. It has more to do with the sense of pride you derive from producing something of high quality. Your workmanship is an expression of your skillfulness; your mastery of a process or craft. There is a sense of joy felt in exercising your skill.
People may be well-intentioned. I’ve seen many display such quotes as “Amateurs Practice Until They Get It Right; Professionals Practice Until They Can’t Get It Wrong”. Few, however, work to build mastery of a skill or develop a sense of awareness of their environment. To do that you must practice performing a task while being mindful of your context. That is hard.
Even if we don’t get to choose what we do, we can do what needs to be done with uncommon grace.
Design verification is the process by which we check whether what we designed (design output) matches what we asked for (design input). The concept is simple to state (Figure 1), but it can be incredibly challenging in practice, partly because of the sheer volume of checks that need to be made.
Figure 1. The basic concept of design verification: Did we get what we want?
In my experience, the design effort results in a drawing of the product which can then be used to make a physical prototype (Figure 2). So in verifying the design, we are checking whether the drawing or the prototype meet all the requirements we specified. The processes used for verifying the design range from simple visual inspection to elaborate tests.
Figure 2. One possible product development process showing where design verification fits in.
Methods of Verification
Visual Inspection It is possible to confirm whether a design contains the required physical attributes through simple visual inspection. Physical attributes include:
- Aspect ratios
- Count of features
- Surface finish
- Surface coating
Table 1. The table shows one possible approach to design verification of physical characteristics. Design inputs are specified in the Engineering Requirements column. Design output in this example is the drawing titled B6-32.DWG. The evidence of the verification of a particular characteristic is given by the page number and grid location on the drawing.
Analysis Properties of the designed product may be calculated using physical laws (from Newtonian Mechanics)
- Weight may be calculated using the design’s volume and material density
- Deflection may be calculated using methods from Statics/Mechanics of Materials
- Stress under given loads may be calculated using methods from Statics/Mechanics of Materials
- Computational modeling such as finite element analysis (FEA) may be used, provided certain requirements are met (see the FDA’s guidance on Reporting of Computational Modeling Studies in Medical Device Submissions)
- Tolerance Stack Analyses (TSA) may be performed for sub-assemblies and assemblies to determine whether the design conforms to the size requirements
- TSA’s may similarly be used to determine whether the components of a sub-assembly or assembly fit together
Table 2. The table shows one possible approach to design verification of a property of the design. Design input is specified in the Engineering Requirements column. An engineering analysis report was created to show the calculation of the weight. The evidence of the verification is given by the page and section number of the report.
Testing Some aspects of the design will require testing. These include:
- Package integrity
Table 3. The table shows a second possible approach to design verification of a property of the design. Design input is specified in the Engineering Requirements column. A prototype was built. The test was performed on it, and a test report was created to show the results. The evidence of the verification is given by the page and section number of the report.
Design Verification by Assembly Hierarchy
Now let’s take a step back and look at design verification from a different perspective: that of design hierarchy (Figure 3). Product design can be any combination of components, sub-assemblies, or assemblies. Verification should occur at each hierarchy of the design.
Figure 3. Overall product design can be any combination of components and sub-assemblies.
For example, in the case of an assembly made up of a bolt, a nut, and a washer, my approach to verification would be the following:
- Perform a visual inspection of the bolt drawing to verify the bolt’s physical attributes e.g. size, material, color, etc.
- Do the engineering calculations using the bolt’s design to demonstrate the bolt’s functional attributes e.g. torsional strength, bending strength, etc.
- Perform a visual inspection of the nut drawing to verify the nut’s physical attributes e.g. size, material, color, etc.
- Do the engineering calculations using the nut’s design to demonstrate the nut’s functional attributes
- Perform a visual inspection of the washer drawing to verify the washer’s physical functional attributes
- Do the engineering calculations using the washer’s design to demonstrate the washer’s performance attributes
- Perform a visual inspection of the assembly drawing to verify the assembly’s physical attributes:
- Are the correct bolt, nut, and washers specified?
- Does the drawing specify the number of each component to be used in a single assembly?
- Does it show how these components are to be assembled?
- Perform a tolerance stack analysis of the components to show components will fit together, or build prototypes of each component and perform a test of assembly.
Different aspects of the design are verified at each level of the assembly hierarchy.