A Tale of Two Tests

Hardware product development has two unique types of testing – each with their own distinct purpose. Much like software development, hardware tests are required to validate features work as specified, during the development phases. But only hardware products require testing of each individual manufactured unit during the production phases.

So, What’s The Difference?

Product Development Testing is used during the development stages of your product lifecycle to ensure the product meets the feature requirements. This is necessary as you progress towards a final design and to ensure existing features don’t regress as new development follows.

Product Design Testing is used during the manufacturing stages to ensure each individual instance of your product meets the functionality and quality requirements of the final released product design.

Most people understand the value of product development testing, but I get a lot of questions when it comes to hardware design and testing methods.

Manufacturing is not necessarily a precise process. Even once a product design is locked down there are still many variables introduced into the workflow that create risk and can culminate into problems with your product. These variables can be anything from incorrect parts being assembled, to parts missing or assembled incorrectly, to parts getting damaged somewhere along the workflow. All of these “worst case scenarios” need to be tested for at some point in the manufacturing assembly line. While there is a large amount of automation in manufacturing to minimize these risks, the automated processes and hardware testing tools are still managed by people and we all make mistakes.

It’s a Gaussian, Gaussian, Gaussian, Gaussian World

The more obscure variables to test for are the natural variances that occur in repetitive processes. We live in a Gaussian world where there is always some inconsistency between reoccurring events. This normal variance, when not accounted for, can cause subtle but potentially catastrophic problems. It’s critical to test for these inconsistencies to make sure the final assembled products each perform within the acceptable variance limits. It’s often not economical to spend the engineering effort to completely design out variance in your product, so it’s best to just accept it. However, you can put limits on exactly how much variance you can accept before an instance of your product shows unacceptable performance and must be rejected.

Show Me The Variance

Just about everything has variance, starting with the parts you ordered from your supplier. If you order 10,000 screws and measure the length and diameter of every one of them, you will see a Gaussian (or normal) distribution of measurements. The majority of instances should measure near exact to the specified dimensions in the supplier’s documentation for the screw, but a smaller percentage of them will measure above or below this median value. If the mating parts for this screw in your product are not designed to accept this variance, your product may show reduced performance or functionality.

Furthermore, the processes themselves, used to assemble the parts, have natural variance. For example, the solder paste screen printer may apply slightly more or less solder every time a bare PCB is inserted, resulting in potential electrical shorts or opens in your printed circuit board assemblies. These defects can be subtle and undetectable by a mere visual inspection.

Oh, What to Do?

So, to account for all of these manufacturing variables it’s imperative your product goes through one or more functional test stages prior to final packaging. There are many published strategies on how best to test your product reliably and efficiently while not incurring too much additional cost – which I’ll leave as an exercise for the reader. But, a basic best practice for product development testing to follow is to start with adding tests for the most critical features and then add tests for features that could have the most potential variance during production. Electronic circuits are top candidates for this.

An excellent strategy to add product design testing to your product is to start with a functional test fixture to test the PCBAs (Printed Circuit Board Assemblies) after they are fully assembled but prior to integration into any mechanical components. These test fixtures are commonly referred to as a “bed-of-nails” and the process is called FCT (Functional Circuit Testing). The details of how to properly design and operate a PCBA functional circuit test fixture are extremely important and worthy of their own blog post – which I will publish in the coming weeks.

For now, the key takeaway from this article is to make sure you don’t neglect or underestimate the importance of functional tests on your production line. Your contract manufacturer will ultimately be responsible for operating the tests, but it’s your job to design the hardware testing methods and monitor them. There’s too much at risk to leave these tasks to someone else.