Product Verification for Electronic Hardware

I recently wired my lab for ethernet and began automating my testing with Python scripts and the awesome Python(x,y) environment curated and maintained by Google. Freed from the burden of so much test setup this got me thinking about how to design and execute a test which fully and accurately characterized the unit under test. This is broken into the following objectives:

• Providing thorough test coverage in your test plan
• Creating a measurement that accurately reflects the circuit under test
• Testing enough to be representative of all the devices you'll ever manufacture

Let's start with a (very general) listing all the main stages in a product design and highlighting where this article focuses:

1. Product conception
2. Marketing research and product definition
3. Product specification
4. Product architecture
5. Pre-design simulation and FMEA
6. ID, product, circuit and software design
7. Post-design simulation
8. Prototype manufacturing
9. Product verification
10. Circle back at any stage if changes need to be made
11. Production line design and product ramp

Providing thorough test coverage requires wisdom and is a bit of an art. But this is a general guideline that should provide good coverage:

1. You need to test all the features found in the product definition
2. For each feature above you need to test all of the components in a system that directly provides that functionality
3. For each component you need to test all of the components in a system that support that component

It's a hierarchy. If you've listed out all of the features and all the components that provide them and you have functionality left over it doesn't belong or you've made a mistake.

For example you want to store movies in a set top box so the NVRAM in the system will need a thorough testing. But the power supplies and bus that provides the interconnect also needs validation as does the host processor.

BTW, when designing any IC I usually go through the following steps. Bring-up and verification are usually the same:

1. Wire up the power supplies
2. Wire up any clocks
3. Set and strapping that is required
4. If there is any logic (that is not a simple strap) then configure that as well
5. Connect any I/O buses
6. Attach input or output devices

Accurate measuring requires careful consideration of your test circuit and how it is affecting the system. That is really a separate article (or book) and is not the focus of this article, but generally speaking your test system should not load the circuit under test and should provide performance >= to the circuit.

Meaningful to me means a test result or measurement that accurately conveys the performance of ALL SYSTEMS not the particular system under test or the time it was tested. This means that a particular functionality needs to be characterized over all possible dimensions and be sampled enough to be statistically meaningful. For most consumer electronics this means testing over temperature, altitude, system configuration (e.g. CPU load, settings, etc), component/process and manufacturing/assembly variation.

Statistically meaningful means taking enough measurements from which an accurate sigma can be be determined. This allows you to calculate the 6 sigma values and know with certainty the performance of your system and whether or not it will fail or violate spec. Typically once I have a good setup I will use a pass/fail or mask on my test equipment (or in software) and take hundreds if not thousands of measurements. This is usually pretty easy on a modern scope that can trigger on a specific behavior, display a mask, count violations and display histograms or statistical information.

Testing a circuit on one system will provide a distribution (probably Gaussian) that describes your system but does not cover environmental and component/process variation. In order to test over environment you need to use a chamber to simulate the environmental extremes the product is designed to operate in. But the reason we test more than one unit is to provide coverage over component/process variation and manufacturing/assembly variation. And just like any test to be statistically significant you must test many, many units.

The sum of the distributions of all the units over environment, system configuration and process will provide you with accurate characterization of the design.

If you aren't violating spec you are ready to ramp.