This an example that explains an experiment from identification of the need to an answer that solved the problem. This entire example can be followed and duplicated with pencil and paper, but notes in the text highlight the advantages of the application of appropriate technology.
A report on this R&D project was published in Adhesives Age in August 1988
Step One -- What is the Question
Step Two -- Identify the Responses to be Measured
Step Three -- Identify the Factors and their Limits
Step Seven -- Analyze the Data
Step Eight -- Compute the Sweet Spots
Step Nine -- Share Your Results
Retrospect -- Doing Even Better
The Management at Monsanto assigned this challenge to Peter Tkaczuk:
The Marketing Department at Monsanto wanted a pressure sensitive adhesive that would meet these goals for these four responses.
| Peel | 4.6 to 4.9 |
| Loop Tack | 4.0 to 4.3 |
| Shear | 10 to 25 |
| Cost | Under 14 |
Management's Question: Two factors, cross linker and monomer, can be adjusted when making the adhesive. How should these factors be set to meet the goals for the four responses above?
(Cross linker and monomer are ingredients in creating an adhesive. These could be the names of any two factors in any process.)
Your Project
Your management might not define the problem as clearly as Peter's did, but you must achieve this level of definition before you know what problem to solve.
Step One -- What is the Question
The first challenge in DOE is present the question in writing for management: "If I answer this question, will you use the answer?"
Management had already framed the original question. Peter simply needed to verify that the answer would be used. Management's Response: "Yes, we will use your settings for cross linker and monomer ratio if the adhesive meets our goals for Peel, Loop Tack, Shear, and Cost."
Your Project
What is the question to be answered by your research? Make sure that you put it in writing.
Focus on how the results will be used. Avoid such vague phrases such as "to investigate" or "learn about." Focus on the action that will be taken based on the results you obtain.
Take your question to management and ask: "Will you use the results if we answer this question?"
Step Two -- Identify the Responses to be Measured
If the question is clearly stated, the responses and the appropriate measurements will be easy to identify. Peter's necessary measurements were clearly Peel, Loop Tack, Shear, and Cost.
Your Project
Responses are outcomes which can be measured or observed: color, hardness, octane, strength, etc. Peter only was concerned with four responses, but you may include as many responses as you need and are willing and able to measure.
List your responses and proceed to Step Three.
Step Three -- Identify the Factors and their Limits
Factors are variables like temperature which can be changed in hope of improving the product. Peter varied two factors.
For each of these factors, note that the complete description of all of the possibilities is provided by the factor name with the upper and lower limit. Peter's factors are continuous factors; all levels between the lower and upper limits are candidates for sweet spots. Set the upper and lower limits as far apart as safety allows; too often, researchers discover after the experiment that they did not set their limits far enough apart to find the best sweet spot.
Your Project
A factor is an input which can be directly set at a specified level. What factors will you vary to achieve your objective?
Peter's example has only two factors. If you are following along with a project of your own, you may wish to consider only two of the continuous factors to make it easier to follow this introductory example.
Our example deals only with two continuous factors, but you situation may involve any of the following types of factors: Two, note that each kind of factor requires that you collect different information.
| Factor Type | Required Information |
| Continuous | Upper and Lower Limits |
| Discrete | The Number of Discrete Levels to be studied. |
| Restricted | Upper and Lower Limits for each permitted range |
You may also impose constraints on the relationships between the factors. Mixtures are the most common examples of this: if the upper and lower limits of the mixture components are defined in percentages, then all of the components in that mixture in any one trial must add up to 100%.
Peter chose an interaction model because it gave him linear interpolation between his data points and the ability to detect synergy.
We would recommend that Peter use the quadratic model if he can afford the extra runs. Our software has reduced the increase in cost to go to a quadratic model. Peter didn't have that luxury in 1988.
Your Project
Dollars and source materials that are available limit the choice of model. There are several choices in increasing order of expense and effectiveness: main effects models, interaction models, and quadratic models. (Note that more expensive means that more trials are required)
The main effects model which is the least expensive model provides the least information. Main effects models are typically used for screening large groups of factors (ten or more) factors to see which factors to follow up on in an experiment using a higher order model. Choose the main effects model only when the number of factors increases the required number of trials beyond your budget.
The interaction model is more expensive than the main effects model, but it is capable of detecting synergy between the factors. The interaction model fits the data well when the response measurements can be linearly interpolated.
The quadratic model is the model of choice when budget allows because it can fit the data even when the responses show curvature away from linear interpolation. It is considerably more flexible, and optimal design techniques have reduced the cost of quadratic models.
More complex models can be used. They add to the expense of the experiment. They might or might not repay that expense in the form of a better answer. The quadratic model provides an adequate fit for most industrial problems. We recommend that you call The Experiment Strategies Foundation (1-800-732-7381) if you are considering a higher order model.
The purpose of an experiment design is to find the best answer at the lowest possible cost. You could be certain of finding the best answer if you could run trials for all of the possible combinations of the levels of the factors. This is only possible and economical with small problems involving only discrete factors. Most often you are looking for an efficient and effective means of running a few trials and predicting the rest.
Peter used an economical classic interaction design. Note that these four trials represent all possible combinations of the low and high limits.
Trial |
X1 -- Cross Linker |
X2 -- Monomer Ratio |
1 |
0.2 |
0.0 |
2 |
1.0 |
0.0 |
3 |
0.2 |
1.0 |
4 |
1.0 |
1.0 |
This design is economical. The four trials above are the absolute minimum mathematically to allow computation of the four coefficients in the model. This design is called a factorial design because its inventors were studying factors (all designs study factors). Interaction design is a more meaningful name because it will measure the interaction which is synergy.
To get better precision, Peter did each run twice: two "replicates" of each "trial." Peter also ran the center point (X1 at 0.6 and X2 at 0.5) to test the quality of the model.
Your Project
If you are following along with your own experiment, you can create a similar design in the same way that Peter did. The beauty of this is that it requires no software, no reference book, and can be done with a pencil and paper.
Generating the most efficient design is a few hours work for an expert. The classic designs found in STRATEGY for Windows and similar products may be the most efficient means of answering your questions, but the more factors and special conditions that you have the more likely that only a custom design will meet your needs. As we will see later, custom designs can offer significant advantages even in a simple two-factor experiment like this.
Applying Software
The proper software will save you work at this stage, and software is absolutely essential if you need the most efficient design for a complex problem.
STRATEGY for Windows can generate a wide variety of the best classic designs, and Gosset can generate a wide variety of custom designs including I-Optimal (Hardin-Sloane) designs. For an incredibly modest fee, The Experiment Strategies Foundation will generate a custom design for you.
STRATEGY for Windows can analyze any well-designed experiment, and many of its capabilities have been fine-tuned for Gosset designs.
Peter scrambled the runs to protect the results from changes in the environment (such as humidity) which might damage the results.
Here is Peter's scrambled data list.
Run Order |
X1 |
X2 |
Y1 |
Y2 |
Y3 |
Y4 |
1 |
0.2 |
1.0 |
5.41 |
4.79 |
8.0 |
14.7 |
2 |
1.0 |
0.0 |
3.38 |
3.40 |
79.0 |
13.8 |
3 |
0.6 |
0.5 |
4.30 |
3.74 |
41.0 |
14.0 |
4 |
1.0 |
1.0 |
4.60 |
3.94 |
54.0 |
17.2 |
5 |
0.2 |
0.0 |
2.91 |
54.0 |
9.6 |
10.3 |
Each of the numbers above for Peel, Loop, Shear, and Cost is the average of a pair of replicates. Your chance of finding the best sweet spot improves when more replicates go into each average, but then the costs are higher. Budget dictates compromise.
Your Project
If you've been following along with pencil and paper, scramble your runs as Peter did and add as many columns as you need for responses.
Run the trials in scrambled run order and write down the measurements for each of the responses.
The selection of the model and the design determine the runs to make. Careful execution of each run and accurate recording of the response data provide the necessary basis for predicting the results of the trials that you chose not to run. Collecting the data usually consumes the bulk of the resources required by the experiment; an efficient design and a through analysis makes sure that those resources are put to effective use.
Applying Software
If you've been using STRATEGY for Windows, you can enter the data directly into STRATEGY, or you may print out the XY List.
When you enter the data, DO NOT average your runs as shown above. The regression engine needs the detail from each run to extract the most useful information from your data. The averaging shown above is done exclusively to demonstrate how a simple experiment can find an answer with no software or complex math.
Step Seven -- Analyze the Data
Here is Peter's data entered on square diagrams. The space within each square represents all possible combinations of levels of the two factors. Cost was computed from the cost of ingredients. Peel, Loop, and Shear were measured for each of the five adhesives made.
Goal: 4.6 to 4.9 |
Goal: 4.0 to 4.3 |
Goal: 10 to 25 |
Goal: Under 14 |
Look at Peter's four diagrams and the respective goals. Could you pick a sweet spot in the X1, X2 square at which all four goals are met?
Your Project
If you're following along with your own parallel example, use the following blank diagram for your own data. Make a copy for each response and fill them in the data as Peter did.

This is as far as you can go with just pencil and paper. Note that identifying the sweet spot by this method is imprecise. The sweet spot is even more difficult to find visually as soon as you have more than two factors.
Applying Software
If you have entered your design and data into STRATEGY for Windows, STRATEGY will find the sweet spot for you. A regression analysis will create the coefficients that are necessary to predict the trials that you did not run. Then GridSearch will use those coefficients to find one or more sweet spots.
Step Eight -- Compute the Sweet Spots
Peter used STRATEGY for DOS to generate a series of contour plots. He then shaded in all of the surface area where the response prediction did not meet the desired goals, leaving the target area white. He then overlaid the charts and held them up to a light source. The common white area let the light through, and the desired factor settings could simply be read off the chart.
Peter found a PREDICTION of a sweet spot. He had every reason to believe that it was a good prediction, but it was just a prediction. To see how accurate that prediction was, he compared his actual measurements at the center point with the predictions generated by the model.
Response Names |
Center Predictions |
Center Measurements |
Peel |
4.4 |
4.3 |
Loop |
3.8 |
3.7 |
Shear |
38 |
41 |
Peter could run another trial using the factor settings that his experiment predicted would produce the desired response values. If that run confirmed the prediction, Peter would also be ready to report his answer to management. This is ideal, but it can be inconvenient and expensive to run another trial. For example, some experiments require shutting down an entire plant to do a single run.
Management must now decide:
Are these predictions close enough to the observed responses?
Peter's management decided that the model predicts well enough for their purpose. What could Peter have done if management had found the accuracy of the predictions to be inadequate?
Peter found his answer quickly and inexpensively, and he did it with the tools available ten years ago.
Today, he would use GridSearch. He would give the goals to STRATEGY. The predictions from the regression analysis would be searched for points in the X1, X2 square that meet all the goals; sweet spots.
Your Project
You could repeat Peter's approach if you are following along with a two-factor problem, but the more factors the problem involves, the more difficult to find the sweet spot by visual inspection. Once you have run a regression analysis, it is easy to use GridSearch to find the best answer. Simply set your response goals as desired. Then use contour plots to display the answer to others.
Like Peter, you should always verify the sweet spot by running a trial that uses the indicated factor settings. Only after running this trial can you confidently answer questions like "Have I really included all of the critical factors?", "Did I use the right model?", or "Did the software create a useful and accurate prediction?" An untested sweet spot is just a theory -- a solid, well-supported theory but a theory nonetheless.
Applying Software
If you entered your data into STRATEGY for Windows and successfully ran a regression analysis, you are ready to run a GridSearch. Select Compute a Sweet Spot from the main menu, accept the default name for the file, and the GridSearch Definition Form should appear with the Goals tab showing.

The Goals tab lists each response in the model. Select whether each response should be minimized, maximized, or kept within bounds.
Step Nine -- Share Your Success
Peter knew he would have a receptive audience. Management came to him in the first place, and Peter had clarified and verified the question in step one. He found and verified the answer while management still cared -- timeliness is critical.
Simply finding the desired answer is not enough. You must also communicate that answer to the people who can benefit from it in a way that convinces those in charge to use it. If you followed step one as described, management should already be primed to use the answer as soon as you clearly identify the answer and show them how it meets their requirements.
For Peter, the tool he used to find the sweet spot was the tool that he used to show the results to management the overlaid contour plots. Even though contour plots aren't very useful for finding sweet spots in multi-factor experiments, they are still excellent for conveying the answer to others.
STRATEGY for Windows offers two and three-dimensional contour plots, and response overlay charts. These can be printed two, three, or four to a page, or cut and pasted to any other Windows application.
Retrospective -- Doing Even Better
Peter ran the interaction design (left below). He can add the "star points" to fit a better model.
By adding the five star points Peter can add X2 terms to the model.
Y= b2 + b1X1 +b2X2 + b12X1X2 + b11X12 + b22X22
He can use our software, STRATEGY, to compute the regression coefficients and contour plots. You can do this too.
If Peter had wanted to fit the quadratic model at the start then he could have used the 6 point optimal design shown below. It costs 3 runs less than the 9 trial interaction plus star design above.
| When you can afford 6 trials, instead of 4 as in the interaction model, then use this 6 trial design and fit the more flexible quadratic model. |