Comparing Coefficients is Oversimplification

When you get your computer output back from analyzing DOE data, other people are going to encourage you to look at "effects" and rank them largest to smallest. They will imply that you can make decisions about what your company should do by looking at that list. Resist that line of thought; it's an oversimplification that is unnecessary.

A much smarter thing to do is the use "effects" in the model fitted to your data to interpolate predictions of product performance throughout the region of interest that is covered by your data. Then choose the place to operate your company's process as the one that has the best predicted performance: the "sweet spot."

The tendency to oversimplify by ranking effects got started very early: about 1925. Imagine a one-factor-at-a-time experiment:

                   x1    x2   Response Y
existing process:  22   110     8
change only x1:    26   110    12   effect of x1 is 4 units of Y
change only x2:    22   125    17   effect of x2 is 9 units of Y

It is quite natural to say that x2 has a bigger effect and so it is "more" important. It is more productive to look at both effects:

*changing only x1 helped 4 units (8 to 12)
*changing only x2 helped 9 units (8 to 17)

The tempting next conclusion is that if we change both at once, then Y ought to go up 4 + 9 = 13 units. So we check that out by changing both factor levels:

                   x1    x2   Response Y
change both:       26   125       6

Now we have done in total the four trials in a 2-factor, 1-level interaction design ("factorial"). Design of Experiments took off into important territory when it went past one-factor-at-a-time and looked at both-changed also.

Now we see a surprising "effect": changing both x1 and x2 has dropped the response from 8 to 6. This is surprising because x1 by itself helped 4 units and x2 by itself helped 9 units. Changing both, as those results suggest we should do, drops the response 2 units (8 to 6). This event in data is called "interaction."

Interactions are common and valuable in industry. Because of interactions it is self-deception to look only at lists of main effects and then make decisions about how to operate processes. Two factors can have near-zero effects on the average in your data, yet together they can be very powerful in helping you operate the process better because they interact. Or, they can cancel out their effects when both are changed.

Avoid the use of a Pareto chart which lists effects largest to smallest. It does not help you decide how to set the factors for best process performance.

Use a model for each response to predict performance for all combinations of factors within your region of interest. Then pick the place to operate that is the most profitable: the "sweet spot." The "effects" will be part of the model. The effects will work together in the model to help you find the sweet spot. Don't waste your time looking at them separately.

In the example above, the interaction shows that x1 and x2 should not both be increased at once; they are antagonistic. Their benefits tend to cancel.

In other cases, the reverse can occur: synergy. When increasing two factors shows more benefit than the sum of their individual effects, that situation is an interaction called synergy.

Ranking factors by their effects doesn't solve the problem of finding the sweet spot for best operation of the process.

Return to top of page