Design verification is the process by which we check whether what we designed (design output) matches what we asked for (design input). The concept is simple to state (Figure 1), but it can be incredibly challenging in practice, partly because of the sheer volume of checks that need to be made.
Figure 1. The basic concept of design verification: Did we get what we want?
In my experience, the design effort results in a drawing of the product which can then be used to make a physical prototype (Figure 2). So in verifying the design, we are checking whether the drawing or the prototype meet all the requirements we specified. The processes used for verifying the design range from simple visual inspection to elaborate tests.
Figure 2. One possible product development process showing where design verification fits in.
Methods of Verification
Visual Inspection It is possible to confirm whether a design contains the required physical attributes through simple visual inspection. Physical attributes include:
- Aspect ratios
- Count of features
- Surface finish
- Surface coating
Table 1. The table shows one possible approach to design verification of physical characteristics. Design inputs are specified in the Engineering Requirements column. Design output in this example is the drawing titled B6-32.DWG. The evidence of the verification of a particular characteristic is given by the page number and grid location on the drawing.
Analysis Properties of the designed product may be calculated using physical laws (from Newtonian Mechanics)
- Weight may be calculated using the design’s volume and material density
- Deflection may be calculated using methods from Statics/Mechanics of Materials
- Stress under given loads may be calculated using methods from Statics/Mechanics of Materials
- Computational modeling such as finite element analysis (FEA) may be used, provided certain requirements are met (see the FDA’s guidance on Reporting of Computational Modeling Studies in Medical Device Submissions)
- Tolerance Stack Analyses (TSA) may be performed for sub-assemblies and assemblies to determine whether the design conforms to the size requirements
- TSA’s may similarly be used to determine whether the components of a sub-assembly or assembly fit together
Table 2. The table shows one possible approach to design verification of a property of the design. Design input is specified in the Engineering Requirements column. An engineering analysis report was created to show the calculation of the weight. The evidence of the verification is given by the page and section number of the report.
Testing Some aspects of the design will require testing. These include:
- Package integrity
Table 3. The table shows a second possible approach to design verification of a property of the design. Design input is specified in the Engineering Requirements column. A prototype was built. The test was performed on it, and a test report was created to show the results. The evidence of the verification is given by the page and section number of the report.
Design Verification by Assembly Hierarchy
Now let’s take a step back and look at design verification from a different perspective: that of design hierarchy (Figure 3). Product design can be any combination of components, sub-assemblies, or assemblies. Verification should occur at each hierarchy of the design.
Figure 3. Overall product design can be any combination of components and sub-assemblies.
For example, in the case of an assembly made up of a bolt, a nut, and a washer, my approach to verification would be the following:
- Perform a visual inspection of the bolt drawing to verify the bolt’s physical attributes e.g. size, material, color, etc.
- Do the engineering calculations using the bolt’s design to demonstrate the bolt’s functional attributes e.g. torsional strength, bending strength, etc.
- Perform a visual inspection of the nut drawing to verify the nut’s physical attributes e.g. size, material, color, etc.
- Do the engineering calculations using the nut’s design to demonstrate the nut’s functional attributes
- Perform a visual inspection of the washer drawing to verify the washer’s physical functional attributes
- Do the engineering calculations using the washer’s design to demonstrate the washer’s performance attributes
- Perform a visual inspection of the assembly drawing to verify the assembly’s physical attributes:
- Are the correct bolt, nut, and washers specified?
- Does the drawing specify the number of each component to be used in a single assembly?
- Does it show how these components are to be assembled?
- Perform a tolerance stack analysis of the components to show components will fit together, or build prototypes of each component and perform a test of assembly.
Different aspects of the design are verified at each level of the assembly hierarchy.
Most of us have been taught to live our lives singularly focused on outcomes. We are a result oriented society. We worship winners and revile losers. With that as the context, is it any wonder that the majority of us are risk averse? Why do something if there is a chance we might not achieve our desired end? Who would want to be labeled a failure or a loser?
I remember an incident back in middle school. I had got a ‘D’ in one of my classes. It generated such anxiety that I doctored my report card. With a couple of well placed dots penciled in on the dot matrix printout I changed the grade from the ‘D’ to a ‘B’. I knew it was wrong, but the fear of failure was dominating. Even though my parents had never set any expectations for grades – they had always wanted me to just do my best – I had nevertheless internalized the stigma of failure from other social contexts.
The message we all, student or professional, get every waking moment is clear even if it is just implied: hit the target. It does not matter how. Anything less is not only worthless but it will bring negative consequences. What we fail to grasp is the fact that just as you cannot inspect quality into a product, you cannot test knowledge into a student. Trying to do it just fosters our present culture wherein we are all afraid to take chances and risk failure.
We must shift our focus upstream to the learning process. We must be taught how to learn beyond just what to learn. All of us have to understand how knowledge is built and then use that process to grow our own knowledge. That process, the learning cycle, is elegantly expressed by Dr. Shewhart and Dr. Deming: PDSA or Plan-Do-Study-Act. Its deceptive simplicity hides the profound impact it has had in growing human knowledge.
Make a guess at the solution to the problem you’re facing and plan a way to test your guess. Following the plan, actually do the test; carry it out. Study the result by comparing it against your guess. Do they match each other? How you act, depends on the answer. If the result matches your guess, you have confirmation of your guess. If not, you will need to modify your guess with the new found data and run through the cycle once more. Either way, you have increased your knowledge. There is no failure!
How many cycles it takes to understand the nature of the problem we face depends on how good our initial guess is. We must afford everyone the opportunity to run through as many cycles as they need without judging them as a success or a failure based on the outcome of any given iteration of the cycle. Doing so is a way to kill intrinsic curiosity and stop the learning process cold. If we want people to take risks, then we have to foster an environment of experimentation and cooperation.
Life is a journey, not a destination. — Ralph Waldo Emerson
The Toyota Production System was born out of necessity. These two videos show its evolution.
My friend Katie inspires me. It is because she creates. She is a maker of things. Katie makes greeting cards; she draws; she writes; she cooks; she takes pictures. Her joy shows in all of her creations. Her work has on more than one occasion sparked me to create and share my own things.
I feel that many of my generation have lost touch with our intrinsic drive to make things. Instead we have replaced it with habits of consuming things others have made. And while there is that initial excitement at having bought something, it wears off quickly. But the thrill of making things on your own is different. It stays with you. There is a sense of pride and ownership about having learned something or having figured something out.
This past weekend I watched “the next list“, a program on CNN hosted by Dr. Sanjay Gupta. It profiled Dale Dougherty and his creation – the Maker Faire. At a time when everyone seems panicked about America losing its edge in creativity and innovation, Dale is igniting that fire single handedly. He is providing a venue for do-it-yourself-ers to show off their creations. “the next list” spot lighted a couple of makers and their creations at one such fair. I was amazed with what I saw.
Somewhere on our way to the 21st century we started to believe that only geniuses can create; that failure was not an option. We stopped making making things and exploring ideas for their own sake. It is time to reconnect with our humanity; get our hands dirty once again and start making things. America doesn’t have to fear losing its leadership in innovation and creativity. But to maintain it we do need to replicate Maker Faire in all of our homes, schools and businesses.
The fundamental assumption that “the other side of innovation: SOLVING THE EXECUTION CHALLENGE” is based on is that your organization is attempting innovation initiatives beyond its current capabilities. Capabilities that you are not willing to develop internally. In other words you are attempting to climb Mt. Rainier, to use the authors’ opening example, when you are neither fit for the challenge nor possess the skills for it. Think about this for a second. Contemplate the likely outcome of such an attempt. If it doesn’t kill you it will most assuredly maim you leaving you worse off for having tried.
However, Vijay Govindarajan and Chris Trimble “see no reason why established organizations should be incapable of executing any innovation initiative”. So, what is the solution these authors dictate? After 10 years of field research at “[innovative companies] as diverse as Allstate, BMW, Harley-Davidson, IBM, Nucor, and Timberland”, they recommend that you “Build the Right Team” and “Run a Disciplined Experiment”. Let us understand something clearly: it doesn’t matter how many expert mountain guides you hire or how well you plan your expedition, if you are not fit, if you do not possess the necessary skills, it will fail… disastrously. So, shame on the authors for making a flawed assumption and then impelling organizations to attempt such challenges.
To be fair, they have made valid observations of several crucial shortcomings in organizations today:
- It is not an organization’s creativity and technology that falls short, it is its management’s capability: leaders just aren’t trained to drive innovation.
- Organizations vest most of their innovation efforts in defining opportunity and not much in executing it. This is tied to the previous point.
- As companies mature, they disengage from innovation efforts, relying on profit from increasing operational efficiency.
These are problems that organizations have to overcome internally through rigorous self reflection which leads to creating projects that (re)build the organization’s fitness and skills. They can thus expand their core competencies and cultivate deep domain knowledge necessary to address the total challenge of innovation. Not do it through hired outside experts or mergers and acquisitions.
Hugely successful innovation initiatives in recent memory such as Toyota’s Lexus or Scion, or Apple’s iPod, iPhone, iPad or iCloud have been internal to their respective companies. Neither Toyota nor Apple were first to market with these products. Each took its time to develop domain knowledge; stretching itself through multiple learning cycles before introducing class redefining products.
The authors underestimate and trivialize the value of such continuous improvement programs even when their own studies at Nucor and Deere & Company demonstrate successful outcomes. In fact they dismiss their own observations of successful innovation initiatives at many companies in favor of “What If?” scenarios. Never mind whether these hypothetical initiatives were ones these companies might have pursued but didn’t for lack of capability. They never say so.
The one example the authors cite where a transformative innovation initiative was required: the New York Times. But, this was necessary as the landscape of the entire newspaper industry was unexpectedly and fundamentally changed by an unaccounted force: the internet. An extremely rare event that no book or expert can help to plan for.
In any case, given that the book’s fundamental premise is faulty, the structure built upon it is rickety at best. Don’t get me wrong, there are some good ideas, but they are either poorly communicated or poorly reasoned or both. The authors assume an unjustified authoritative tone that often patronize the reader on his/her complaint (constructed in the authors’ imagination). There is a smugness in the manner they criticize their partner companies’ approach to innovation. I wonder if any of them will have these researchers back to work with them again. And, they fail to properly attribute their learning cycle – Plan-Do-Study-Act – to its developers and advocates: Walter Shewhart and W Edward Deming.
My suggestion is you pass on reading this book. It teaches the wrong lessons. Learn the proper way to execute innovation initiatives by benchmarking the best in class. Two excellent reads are “The Toyota Way” & “The Elegant Solution”. Develop your organization’s management foundations and knowledge building efforts with “Out of the Crisis”. There is a tough climb ahead, but not an insurmountable one provided you start preparing for it now.