On the Purpose of Performing the Operational Qualification (OQ) for an Equipment
A company I worked for believes that the purpose of performing an operational qualification of an equipment is to demonstrate that the equipment functions at the settings expected to be used in production. As such, it does not feel it is necessary to validate the manufacturer’s claims about its product’s greater functional range. I wholly disagree with this limited perception.
Consider a simple hypothetical example wherein I intend to make bread. The recipe calls for the bread to be baked in an oven at 400 F for 25 minutes. Applying this company’s point of view, the operational qualification of my oven would require only demonstrating that it functions at 400 F for 25 minutes. There is no need to validate that the oven is capable of functioning to its manufacturer’s claim of 170 F to 550 F. I believe this model for conducting an equipment operational qualification to be shortsighted at best and a total misunderstanding of what an operational qualification is at worst.
While such an approach fulfills the immediate need to demonstrate that my oven is capable of generating 400 F, such a limited test does not take into account my future potential needs. Perhaps better bread might be made by baking it at 350 F for 35 minutes. Or, I might want to make pizza – that requires my oven to operate at 550 F, the hottest it gets. Or, I might want to keep food warm in the oven at 170 F. An operational qualification of this type precludes me from using my oven at temperatures that haven’t been tested. I would have to repeat the operational qualification at other temperatures before use.
But, I believe that this approach to performing an operational qualification is actually more than just shortsighted. It is wrong. It is an incorrect interpretation of what a proper operational qualification is intended to demonstrate: that the equipment is capable of performing to its manufacturer’s claims. A proper operational qualification should be done such that the functionality of the equipment is evaluated at a minimum at the high & low settings of all its critical input controls. The design of experiments provides a statistically robust and cost efficient method to do this.
In an abstract sense an operational qualification is part of the receiving inspection process, albeit a complex one. And just as in the typical receiving inspection process, it is better to find defects or deficiencies in the equipment at this stage (even those that don’t directly impact our immediate application) than to discover them during production. Defects, like cracks, tend to migrate into the production operating zone over time.
At its most basic level an operational qualification has very little, if anything, to do with using the equipment for any given application. Operational qualification of my oven has very little to do with me using the oven to bake bread, make pizza or keep food warm. Validating that the equipment will perform as expected for a given application is done as part of a performance qualification (PQ) of the equipment. So, when my former employer performed an operational qualification of a tool at an application specific setting, it was actually performing a performance qualification; bypassing operational qualification.
When a proper operational qualification has been performed for a piece of equipment, it does not need to be repeated until the equipment undergoes some sort of preventive or breakdown maintenance when consumable or broken parts are replaced. As such, a company that intends to extend the use of a piece of equipment to multiple applications needs to only perform a performance qualification of the equipment for a given application’s settings before releasing the tool to run that new application.
It is disturbing when quality managers do not take the time to understand the purpose of a particular assurance activity. As you can imagine, this leads to wrong tests being run, wrong data being collected and wrong conclusions being drawn. The problem is compounded by their willingness to take shortcuts. Is getting to the wrong place faster better? It doesn’t make any sense to me why a company would not perform a check of the manufacturer’s claims when the effort required isn’t any greater and the long term benefits are significant.
Book Review: “the other side of innovation: SOLVING THE EXECUTION CHALLENGE”
No one can climb a mountain for you. You have to do it for yourself.
The fundamental assumption that “the other side of innovation: SOLVING THE EXECUTION CHALLENGE” is based on is that your organization is attempting innovation initiatives beyond its current capabilities. Capabilities that you are not willing to develop internally. In other words you are attempting to climb Mt. Rainier, to use the authors’ opening example, when you are neither fit for the challenge nor possess the skills for it. Think about this for a second. Contemplate the likely outcome of such an attempt. If it doesn’t kill you it will most assuredly maim you leaving you worse off for having tried.
However, Vijay Govindarajan and Chris Trimble “see no reason why established organizations should be incapable of executing any innovation initiative”. So, what is the solution these authors dictate? After 10 years of field research at “[innovative companies] as diverse as Allstate, BMW, Harley-Davidson, IBM, Nucor, and Timberland”, they recommend that you “Build the Right Team” and “Run a Disciplined Experiment”. Let us understand something clearly: it doesn’t matter how many expert mountain guides you hire or how well you plan your expedition, if you are not fit, if you do not possess the necessary skills, it will fail… disastrously. So, shame on the authors for making a flawed assumption and then impelling organizations to attempt such challenges.
To be fair, they have made valid observations of several crucial shortcomings in organizations today:
- It is not an organization’s creativity and technology that falls short, it is its management’s capability: leaders just aren’t trained to drive innovation.
- Organizations vest most of their innovation efforts in defining opportunity and not much in executing it. This is tied to the previous point.
- As companies mature, they disengage from innovation efforts, relying on profit from increasing operational efficiency.
These are problems that organizations have to overcome internally through rigorous self reflection which leads to creating projects that (re)build the organization’s fitness and skills. They can thus expand their core competencies and cultivate deep domain knowledge necessary to address the total challenge of innovation. Not do it through hired outside experts or mergers and acquisitions.
Hugely successful innovation initiatives in recent memory such as Toyota’s Lexus or Scion, or Apple’s iPod, iPhone, iPad or iCloud have been internal to their respective companies. Neither Toyota nor Apple were first to market with these products. Each took its time to develop domain knowledge; stretching itself through multiple learning cycles before introducing class redefining products.
The authors underestimate and trivialize the value of such continuous improvement programs even when their own studies at Nucor and Deere & Company demonstrate successful outcomes. In fact they dismiss their own observations of successful innovation initiatives at many companies in favor of “What If?” scenarios. Never mind whether these hypothetical initiatives were ones these companies might have pursued but didn’t for lack of capability. They never say so.
The one example the authors cite where a transformative innovation initiative was required: the New York Times. But, this was necessary as the landscape of the entire newspaper industry was unexpectedly and fundamentally changed by an unaccounted force: the internet. An extremely rare event that no book or expert can help to plan for.
In any case, given that the book’s fundamental premise is faulty, the structure built upon it is rickety at best. Don’t get me wrong, there are some good ideas, but they are either poorly communicated or poorly reasoned or both. The authors assume an unjustified authoritative tone that often patronize the reader on his/her complaint (constructed in the authors’ imagination). There is a smugness in the manner they criticize their partner companies’ approach to innovation. I wonder if any of them will have these researchers back to work with them again. And, they fail to properly attribute their learning cycle – Plan-Do-Study-Act – to its developers and advocates: Walter Shewhart and W Edward Deming.
My suggestion is you pass on reading this book. It teaches the wrong lessons. Learn the proper way to execute innovation initiatives by benchmarking the best in class. Two excellent reads are “The Toyota Way” & “The Elegant Solution”. Develop your organization’s management foundations and knowledge building efforts with “Out of the Crisis”. There is a tough climb ahead, but not an insurmountable one provided you start preparing for it now.
Book Review: “Drive: The Surprising Truth About What Motivates Us”
As I read “Drive: The Surprising Truth About What Motivates Us” I found myself nodding my head in approval… a lot. Daniel Pink makes a compelling argument against the adequacy of commonly accepted ideas on what motivates us: ‘carrots and sticks’, and the effectiveness of management that continues to use those ideas today even after they have been debunked. He presents the findings of scientific studies conducted by thought leaders in the field of psychology, sociology, economics like Mihalyi Csikszentmihalyi, Edward Deci and Richard Ryan, Carol Dweck, Dan Ariely, and Teresa Amabile as he builds his own model to describe what drives us.
Pink’s model of motivation is elegant. Made up of three parts, it is simple yet captures key factors that drive motivation:
- People have a desire for autonomy with respect to task, time, team and technique,
- People want to gain mastery over what they do, and
- People crave to be a part of something bigger than themselves.
It holds an intuitive appeal for me. I suspect that might be a function of the type of work I do and my experience doing it. For as long as I have been working I have been dissatisfied with the management style used by each of my employers. Among other things, I found them lacking in the way they chose to motivate. The studies Pink disccusses in the book helped me understand why some of my job functions went from ‘play’ to ‘work’. They also explain why, from time to time, they reverted back from ‘work’ to ‘play’. His model crystallized what I am looking for as a knowledge worker: An opportunity to choose what I do, when I do it, with whom I do it and how I do it. An opportunity to do it well and get better at it. And, an opportunity to do it in the cause of something greater than myself.
“Drive: The Surprising Truth About What Motivates Us” is a well written book. Pink doesn’t make the reader work at it. He weaves stories that hook a reader and pull him along without being wiser to it. It takes lots of skill and effort to pull that off. Pink builds his case for his model of motivation logically, never losing the reader in the process. He doesn’t just cite studies, he explains their findings. Then he takes the additional step to explain the implication of those findings as he works to incorporate them into his model. So, the net result is an extremely reader friendly book that informs and educates. I place this book in the same space as Jonah Leher‘s “How We Decide“, Dan Ariely‘s “Predictably Irrational: The Hidden Forces That Shape Our Decisions“, Ori Brafman‘s “Sway: The Irresistible Pull of Irrational Behavior“.
Footnotes:
Fork in the Road
Excepting in the presence of active research in a pure science, the applications of the science tend to drop into a deadly rut of unthinking routine, incapable of progress beyond a limited range predetermined by accomplishments of pure science, and are in constant danger of falling into the hands of people who do not really understand the tools that they are working with and who are out of touch with those that do…
– Harold Hotelling, Memorandum to the Government of India, 24 February 1940
That is the predicament I found myself in.
I was hired to support the company’s efforts to validate their processes in preparation for registering their manufacturing facility with the FDA. The FDA defines process validation as:
…establishing by objective evidence that a process consistently produces a result or product meeting its predetermined specifications.
– 21CFR820.3(z)(1)
While the definition is succinct, process validation is not a trivial task. And, contrary to the belief of management that is not literate in the quality system regulations or the subject of quality assurance, it is certainly not something you can “whip out”. It requires an understanding of the process in question – its key inputs and the key attributes of its output – coupled with an understanding of statistical principles such as the design of experiments (DOE) and analysis of variance (ANOVA) necessary to generate the objective evidence that will establish for the company and the FDA that the “process consistently produces a result meeting its predetermined specifications”. And, it also helps to have a proper plan that allows management to identify and allocate the resources required to successfully meet its objectives. At a minimum, a basic project plan should include a detailed checklist of actions items with clearly defined owners and due dates.
But, when management does not understand or trivializes these requirements, it makes decisions that endanger the best interests of the company. Unnecessary risk is assumed. Resources are wasted. Workers are put in a chaotic situation that is the primary source of much of their frustration and fear. So it should not be a surprise when “new” action items “pop up” in crunch time; when there is confusion around the ownership of a task, or when deadlines are missed and missed again causing tensions to flare. And, while these gaffes in project management might be overcome through working harder (translated as management by proclamation – “because I said so” – and long hours), there is no hope to compensate for poorly designed experiments with insufficiently identified process parameters through brute force. Without the right data there is no way to show the process’s capability or even that it is in statistical control.
So there it is – a fork in the road. A choice that we are all confronted with more often than we would like: follow the whack-a-mole tactics of a management team without a strategy or as Dr. Hotelling put it “people who do not really understand the tools that they are working with and who are out of touch with those that do”, or make a swift exit to focus on developing your personal knowledge and skills while searching for a better opportunity. As scary as it seems, the latter choice will always lead to a better outcome. At least that has been my experience.
Deming Redux
Last Saturday, April 29, 2011, the Austin section of ASQ held its educational event for the 2010-2011 year. It was a daylong seminar titled “Dr. Deming Day” in reverence of management guru Dr. W. Edwards Deming. The objectives were to recognize the relevance of Dr. Deming’s teachings in today’s business environment; to better understand the difficult to understand, and to recast existing management data by understanding variation.
Now, there are plenty of ‘consultants’ that hold seminars on Dr. Deming’s philosophy. (Just do a search on the web and you’ll see what I mean.) But, I would just as soon read what the man wrote for myself than listen to someone else interpret his writings for me. So, about a month ago I had started reading his classic “Out of the Crisis”. (I hope to post a review of it when I am done.) And, as I worked my way through it chapter by chapter I noticed him crediting Bill Scherkenbach for contributing to various points he was making.
Who is Bill Scherkenbach? With an online search I found out that he studied under Dr. Deming at NYU and accompanied him on countless seminars. Dr. Deming had high praise for Bill, saying: “He was my student, and there’s none better in the world… It takes a little ingredient called profound knowledge, and he’s got it.” Lo and behold, the “Dr. Deming Day” seminar was being presented by none other than Bill Scherkenbach. While I would have loved to have had an opportunity to attend one of Dr. Deming’s talks, that wasn’t going to be possible. He passed away in 1993. However, I couldn’t pass up the next best thing: an opportunity to learn from someone who studied under him and he was proud of.
Bill started off by providing insights into who Dr. Deming was as a person and a teacher. He put the many quotes attributed to Dr. Deming into their context. You’re never going to get this type of behind-the-scenes access from a book. Context matters. It helps understanding. He followed up with a history and overview of Dr. Deming’s System of Profound Knowledge and the theories that are integral parts of it: Knowledge of variation; Knowledge of psychology; an Appreciation of a system, and the Theory of knowledge. He walked us through Dr. Deming’s famous Funnel and Red Bead experiments to demonstrate the concept of variation; explained how Fundamental Attribution Error affects our judgment; talked about the need for components to run sub-optimally so that the system may run optimally; and how management’s job – prediction – is only possible with knowledge built on theory.
In the course of listening to Bill, and trying to absorb his message, I caught a glimpse of the tremendous challenge we still face: transforming ourselves to meet our obligations to the rest. That’s the bad news. The good news is a framework and a roadmap to do this exists. The ideas talked about by Dr. Deming aren’t just for businesses. They are applicable to all aspects of life and form a philosophy. If understood and applied properly, they will totally transform society (see the rise of Japan post WWII). This I firmly believe. A one day seminar is not enough to internalize the message. I noted down many questions. Questions I hope to explore and understand in future blog posts.
I’m glad I went to the seminar. I would have sorely regretted missing the opportunity if I hadn’t.
Applied Materials Plant Tour w/ ASQ Austin
Yesterday evening members of ASQ Section 1414 were treated to a plant tour of the Applied Materials facility in Austin, TX. This was a rare opportunity to see the manufacturing operations of a top tier semiconductor equipment vendor up close and personal.
The event was kicked off by Steve Rogers, Managing Director of Silicon Sector Group Manufaturing Operation for Austin & Singapore, who presented an excellent overview of Applied Materials’ quality philosophy and how it permeates every aspect of its manufacturing operation. Quality tools such as Kaizen, 8D Problem Solving, lean and Six Sigma are used not just to address problems when they pop up, but to continuously improve processes.
The semiconductor equipment business is a made-to-order business. Every chip manufacturer that Applied Material supplies defines its own unique proprietary tool configuration – one that presumably gives it a competitive advantage over others in the marketplace. You cannot meet such mass-customization and exacting demands by working harder and faster. Applied Materials is doing it by creatively using quality tools to become nimble and efficient – being able to rapidly convert a manufacturing line from one tool type to another, and by moving from building and testing fully built tools to a modular approach.
The tour of the facility included 4 separate buildings that highlighted the company’s different product offerings – various versions of Producers & Enduras. Attendees were broken up into four groups. Each group was escorted by a member of the manufacturing team and one from the quality assurance team. (My thanks to Neelam & Ted for escorting Group 2.) While it was not possible to actually step onto the manufacturing floor which are clean rooms, tour members still got an excellent visual of tools in various stages of production. Tour guides were cognizant of the non-semiconductor background of the group and patiently explained what was being shown.
I was impressed how all the pieces fit together at Applied Materials. Semiconductor equipment are complex. And yet, by applying quality principles and tools in a disciplined manner Applied Materials routinely meets its customers’ expectations worldwide.
I’m especially grateful to Daphne Gilbert, quality engineer at Applied Materials, for making the tour possible. Many thanks!
Pursuit of Perfection
I came across “The Elegant Solution: Toyota’s Formula for Mastering Innovation” by Matthew E. May while scanning the business section at a Half Price Books store. I picked it up because the price was right, to be honest. It sat in a stack at home for months before I came across it again. I had just finished reading Daniel H. Pink’s “A Whole New Mind: Why Right-Brainers Will Rule the Future” and was pondering about creativity & innovation. So, it made sense to read on the approach Toyota took. I regret not reading it sooner.
May dispels the myth that innovation happens in flashes of brilliance within a select group of people possessing an aptitude for creativity. He presents the system Toyota has in place wherein innovation and creativity are the domain of every person. Everybody is inherently creative and we would do well tap into this vast resource. But before you roll your eyes, May gives a reality check and points out that the net impact of your innovation is relative to your base of responsibility, power and control. This grows the higher you move in the organization.
In the book he reveals the three principles that fuel the engine of innovation at Toyota even today: Ingenuity in craft, Pursuit of perfection, and Fit with society. He details 10 key practices – the toolbox – that make these principles operational: Let learning lead; Learn to see; Design for today; Think in pictures; Capture the intangible; Leverage the limits; Master the tension; Run the numbers; Make Kaizen mandatory, and Keep it lean. And, he demonstrates how these principles and practices come together with various examples & anecdotes that go beyond Toyota or even the automotive industry to addressing social problems.
May takes great pains to pepper the book with quotes from figures across the spectrum of human endeavors to show that Toyota or the East didn’t invent these concepts. But, Toyota’s innovative & disciplined use of them has made it “a double threat: the world’s finest manufacturer and a truly great innovator…”
The writing is very reader friendly. I was devouring the book with speed. The structure of the book reinforces the problem solving approach. Each chapter on the 10 practices defines the Problem, identifies the Cause and presents the Solution. Each chapter ends with a section for self-reflection (Hansei): questions that I am using to exercise my brain daily. I would recommend that everyone, not just professionals, read “The Elegant Solution” at least once.
There is a Better Way
I have been guilty, many times over, of making snarky comments about someone else’s ideas. Sometimes I have made them directly to the person. Other times I have said things behind their back. Now with Twitter, Facebook and the many other social media apps on the web it’s easier to pass judgment on ideas of people I don’t know and will probably never meet.
There isn’t anything inherently wrong with finding fault with and pointing out flaws in a person’s ideas. In fact, poking holes in them separates the wheat from the chaff. It helps to focus our efforts and resources on the best ideas. But, shooting down ideas without presenting alternatives is a purely destructive action. It is the way of an uncreative lazy person with nothing of value to add to the dialog. And, it has serious insidious effects that undermine the objective.
At best, I can compare and contrast two ideas in my head: one that someone else has proposed and the other is my own. This left-brain driven analysis makes me seek out differences between the two ideas and causes me to fixate on the flaws in the proposed idea. Flaws that I then use to dismiss the idea entirely as not worthwhile. This reductionist approach is second nature to us all. It is the way we were taught to solve problems in school and college and at work. It is easier to do than to find areas of agreement between the ideas to build upon.
Evaluating ideas in this fashion, though, has a chilling effect. It stops cold any thoughts that might have built upon the valuable aspects of the original idea. It throws out the baby with the bathwater. The idea gets dismissed entirely. Its valuable aspects lost for future consideration. It suppresses the expression of new ideas from others. People get the message that their thoughts are not sophisticated enough. They might feel unfairly dismissed, frustrated & resentful directly resulting in less engagement and less cooperation.
Instead, what is required is a structured process of generating ideas, collecting them, organizing them before evaluating them. Creativity is a fragile mechanism that brings a person’s lifetime of experience, memory and perspective together to synthesize an idea. Each one must be allowed expression without instant evaluation or judgment. Each one should be written down so that many more eyes can see it and many more minds can use it as a trigger for their own ideas. The large collection of ideas can then be sourced for the strongest and best to put into practice.
No one person has the wherewithal to understand, much less solve the problems we choose to tackle today. A collective effort is needed. But, when we kill ideas with snide snarky condescension we undermine our collective objective. Instead we must make every effort to promote idea generation. Be constructive in our commentary of others. No doubt, it takes greater effort to do this. It slows us down. But, we need slowing down so that our haste doesn’t make waste.
Outsourcing Life… Don’t
You are constantly confronted with deciding between doing something yourself or having someone else do it for you. Should I cook or go out to eat (order-in)? Should I clean my house or hire someone to come in and do it? Should I change the oil in my car or take it to a mechanic? With so many products and services available to cater to your every conceivable want, it hardly seems a difficult choice to make. But, I believe, you end up making a Faustian bargain in making the easy choice.
Last year I read “In Defense of Food: An Eater’s Manifesto” by Michael Pollan. It changed the way I look at food and drastically impacted my eating habits. For starters, I got closer to actual food instead of consuming processed food products. (The only time I felt this healthy was eating my mom’s cooking.) I have discovered a sense of joy, satisfaction and pride in making something with my own hands that I didn’t get from eating something pre-made. And, cooking has let me express my creativity, as it has let me build new dimensions to my friendships – exchanging recipes & pictures of my creations. (Check out my friend Katie’s blog on her experiments.)
I have also found a similar sense of joy, satisfaction and pride managing my own place. (It’s more than just cleaning.) I learned about 5S when I got started in the Quality field: Sort, Straighten, Shine, Standardize and Sustain. It has greatly helped me organize my life. Repeating the first four steps each week as part of the fifth step has kept me in touch with what is happening in my house. I have been successfully getting rid of things I don’t need or use instead of wasting time and energy caring for them. Putting things in their place has meant being able to find them when I need them without frenzy and panic. Cleaning everything on my own has let me see problems & fix them promptly before disaster struck.
Last weekend I changed the oil in my car by myself for the first time. It’s not anything complicated, but for someone that had never done it before it was a sense of accomplishment. I learned a new skill! And, it’s something that can’t be taken away easily. I feel more empowered for being able to maintain my own car. I feel more secure that the job was done properly. (I’ve never been a fan of mechanics. I’m always suspicious whether they do what they say they did.)
Doing things by yourself helps you grow. You’re not always going to get things right the first time around. Nor is it going to be cheap. But, it adds to your experiences that make for a great life story that you can share with friends and family. It’s your life. You should live it, not outsource it.
Can’t Affect What You Can’t Control
This rarely happens. So rarely that I don’t remember the last time, if ever, I had a clean drive to my destination. You know, the drive where you not only hit every single green light, but cars shift out of your lane so you don’t have to compensate for their slowness. Ah! It was most satisfying.
Putting that obviously aberrant experience aside, most times I drive I seem to hit every traffic light and inevitably get stuck behind morons that can’t drive. This is woefully true when I am running late for an appointment. “C’mon dude! Can’t be lettin’ everyone in!” I feel as if the Fates were mocking me and laughing harder with each curse word I would let loose. (You’ll have to trust me that I am a defensive driver. I promise.)
But, is it really so? Are there alway special causes? Would it all be okay if ‘this guy’ or ‘that girl’ would just drive better — like me? I decided to check. Everyday I’ve been logging my commute time to and from work. They are shown in the charts below.
Sure, the average (solid red line) commute to work (~35 minutes) is less than the average commute from work (~43 minutes). And, yes, the variation about the average commute to work is a lot less than that about the average commute from work. But, all points are within my control limits (dashed red lines)! That means, all the variation about the average is from common causes! There are no special causes.
If there were special causes, as would have been highlighted by points outside the limits, I could do something about them. If I ran out of gas during my trip which then added to my commute, I could start my future trips with a full tank of gas. If my tires ran flat because they were bald and delayed me, then before all future trips I would make sure my tires were okay. If there was an accident, then I have a legitimate excuse for being late.
Thing is I have no control over common causes for variation like traffic signals changing to red or getting stuck behind some random slow driver or rain or fog. Sometimes, all these things and others combine in a ‘perfect storm’ to make the commute unusually long… or unusually fast. What the charts tell me is, by and large, the time it takes me to get to work will be between 26 – 45 minutes, and the time it takes me to get home from work will be between 27 – 59 minutes. No amount of frustration is going to change that.
Instead, I should tune into NPR and learn about what is going on in the World around me.
