Using I-Optimal Designs for Narrower Confidence Limits
A Common Data Interchange Format for Experiment Designers
and Analyzers
Short Courses
Conference Notes
Jobs
-----------------------------
First, we would like to apologize to Dr. Selden B. Crary for leaving the byline off the
excellent article that he wrote last month comparing the efficiencies of classical designs
and I-optimal designs. Those that enjoyed this article may also enjoy a letter that
we got from Dr. Greg Piepel in response to Selden’s article.
http://www.processbuilder.com/doe/Newsletters/BackIssues/October_1999.htm
http://www.processbuilder.com/doe/Newsletters/Letters/piepel2.htm
Next, we would like to thank Bill Kappele for his excellent article below. Note that
this has a lot of tables and special formatting. If they do not show up in the
version you receive, read the article on-line at our web site. Note the announcement
at the end of the article on the free
teleclass on I-optimal designs .
-------
Industrial experiments demand high quality predictions of product performance from a small
amount of data. Scientists and engineers need experiment designs which provide
narrow confidence limits on predictions to meet this demand. The effect of a design
on the quality of predictions is seldom considered when selecting a design. This
paper shows the advantages of I-optimal designs over conventional designs used in
industry. Confidence intervals for predictions will show that I-optimal designs are
better.
Trials in designed experiments are "well spread out" from each other.
Classic designs spread the points out in space. I-optimal designs spread the points
out to provide the minimum average variance of predictions over the region of interest.
The variance is the square of the standard deviation of the prediction from the fitted
model.
I-optimal designs can be difficult to make. Dave Doehlert, Neil Sloane and Ron
Hardin have developed a means of making I-optimal designs relatively quickly. Their
designs are called "spherical code designs" or "Hardin-Sloane
designs." Doehlert, Hardin and Sloane have made it feasible for industrial
experimenters to use I-optimal designs in their work.
In this paper I will compare I-optimal designs with three designs provided to us by
experimenters in industry. Industrial experiments demand high quality predictions of
product performance, so the criteria for comparison should be the quality of the
predictions.
Predictions with narrower confidence limits are better quality. The experiment
design affects the width of the confidence limits on predictions. A superior design
will have a smaller average confidence limit width on its predictions.
I will compare a conventional design with a corresponding I-optimal design for three
different cases: case 1 is a simple process factor design; case 2 is a mixture
design; and case 3 is a process factor design with a constraint. For each case
I have selected a random point. You will see the confidence limits for the
prediction at this random point for each design. You will also see the average
confidence limits for predictions at 1000 different points in the region of interest.
To focus the comparisons only on the contribution of the design to the confidence limits,
I have standardized t to 2.57 and s to 1.00. These confidence limits will allow you
to compare the quality of the designs for industrial applications.
Another measure of the quality of a design is the "integrated variance" (IV).
This is the average variance for the entire region of interest. Smaller IV indicates
a better experiment design.
The conventional design for case 1 is listed in table 1 below. It is a 4 factor
design for the quadratic model with 2-factor interactions included. It has 25 runs:
20 trials, 5 of which have 2 replicates each.
|
Trial |
X1 |
X2 |
X3 |
X4 |
|
20 |
-1 |
0 |
-1 |
1 |
|
3 |
1 |
-1 |
1 |
-1 |
|
4 |
-1 |
1 |
1 |
-1 |
|
8 |
-1 |
1 |
1 |
1 |
|
9 |
1 |
1 |
1 |
0 |
|
1 |
-1 |
-1 |
-1 |
-1 |
|
14 |
1 |
0 |
0 |
1 |
|
1 |
-1 |
-1 |
-1 |
-1 |
|
2 |
1 |
1 |
-1 |
-1 |
|
11 |
-1 |
1 |
-1 |
0 |
|
2 |
1 |
1 |
-1 |
-1 |
|
15 |
0 |
1 |
0 |
1 |
|
4 |
-1 |
1 |
1 |
-1 |
|
10 |
1 |
-1 |
-1 |
0 |
|
12 |
-1 |
-1 |
1 |
0 |
|
19 |
-1 |
0 |
1 |
-1 |
|
3 |
1 |
-1 |
1 |
-1 |
|
5 |
-1 |
-1 |
-1 |
1 |
|
7 |
1 |
-1 |
1 |
1 |
|
13 |
0 |
0 |
1 |
1 |
|
17 |
0 |
1 |
-1 |
-1 |
|
18 |
1 |
-1 |
0 |
-1 |
|
6 |
1 |
1 |
-1 |
1 |
|
16 |
0 |
-1 |
-1 |
1 |
|
5 |
-1 |
-1 |
-1 |
1 |
Conventional Process Design
Table 1
The corresponding I-optimal design is listed in table 2 on the next page. Table 3
summarizes the comparison of the two designs.
|
Trial |
X1 |
X2 |
X3 |
X4 |
|
1 |
-1.00 |
-0.39 |
1.00 |
-0.45 |
|
2 |
-1.00 |
-0.39 |
1.00 |
-0.45 |
|
3 |
-0.01 |
-1.00 |
0.01 |
-0.10 |
|
4 |
-0.01 |
-1.00 |
0.01 |
-0.10 |
|
5 |
1.00 |
0.30 |
0.04 |
0.88 |
|
6 |
1.00 |
0.30 |
0.04 |
0.88 |
|
7 |
1.00 |
-1.00 |
-1.00 |
0.37 |
|
8 |
1.00 |
-1.00 |
-1.00 |
0.37 |
|
9 |
-0.06 |
0.12 |
-0.08 |
0.00 |
|
10 |
-0.06 |
0.12 |
-0.08 |
0.00 |
|
11 |
0.04 |
-0.01 |
1.00 |
0.43 |
|
12 |
0.00 |
-0.01 |
-1.00 |
-1.00 |
|
13 |
1.00 |
1.00 |
1.00 |
-0.03 |
|
14 |
-1.00 |
-1.00 |
0.19 |
1.00 |
|
15 |
1.00 |
-0.15 |
-0.05 |
-1.00 |
|
16 |
-1.00 |
1.00 |
-1.00 |
0.00 |
|
17 |
-1.00 |
-1.00 |
-1.00 |
-1.00 |
|
18 |
1.00 |
-1.00 |
1.00 |
-1.00 |
|
19 |
1.00 |
-1.00 |
1.00 |
1.00 |
|
20 |
-1.00 |
-0.25 |
-1.00 |
1.00 |
|
21 |
-1.00 |
1.00 |
0.04 |
-1.00 |
|
22 |
-0.03 |
1.00 |
1.00 |
-1.00 |
|
23 |
-1.00 |
1.00 |
1.00 |
1.00 |
|
24 |
1.00 |
1.00 |
1.00 |
-0.92 |
|
25 |
0.34 |
1.00 |
-1.00 |
1.00 |
I-optimal Process Design
Table 2
In table 3, notice that the confidence limits on the random point for the conventional
design are 60% wider than for the I-optimal design. Also notice that the average
confidence limit width for the conventional design is 54% wider than for the I-optimal
design. The integrated variance for the I-optimal design is also better. The
I-optimal design provides better precision of prediction in the same number of runs.
|
|
Conventional |
I-Optimal |
|
95% CL for random point |
+/- 2.96 |
+/- 1.86 |
|
Average 95% CL |
+/- 2.23 |
+/- 1.45 |
|
Integrated Variance (IV) |
0.77 |
0.32 |
Summary
Table 3
Table 4 has a listing of the conventional mixture design for 4 factors. It has 15
runs: 13 trials, 2 of which have 2 replicates each. The limits on the factors
are,
X1 = 0.740 - 0.890
X2 = 0.030 - 0.130
X3 = 0.055 - 0.160
X4 = 0.025 - 0.035.
The implied constraint for a mixture is X1 + X2 + X3 + X4 = 1.000. The model is a
three factor (X2, X3, X4) quadratic model with all 2 factor interactions. X1 is
implied by the constraint. The corresponding I-optimal design is listed in table 5.
Table 6 shows a comparison of the two designs.
|
Trial |
X1 |
X2 |
X3 |
X4 |
|
9 |
0.835 |
0.030 |
0.100 |
0.035 |
|
1 |
0.740 |
0.075 |
0.160 |
0.025 |
|
6 |
0.740 |
0.065 |
0.160 |
0.035 |
|
11 |
0.780 |
0.130 |
0.055 |
0.035 |
|
12 |
0.830 |
0.080 |
0.055 |
0.035 |
|
10 |
0.780 |
0.030 |
0.160 |
0.030 |
|
5 |
0.740 |
0.130 |
0.095 |
0.035 |
|
8 |
0.790 |
0.080 |
0.100 |
0.030 |
|
2 |
0.790 |
0.130 |
0.055 |
0.025 |
|
4 |
0.785 |
0.030 |
0.160 |
0.025 |
|
2 |
0.790 |
0.130 |
0.055 |
0.025 |
|
3 |
0.890 |
0.030 |
0.055 |
0.025 |
|
7 |
0.880 |
0.030 |
0.055 |
0.035 |
|
1 |
0.740 |
0.075 |
0.160 |
0.025 |
|
13 |
0.740 |
0.130 |
0.100 |
0.030 |
Conventional Mixture Design
Table 4
|
Trial |
X1 |
X2 |
X3 |
X4 |
|
1 |
0.802 |
0.074 |
0.095 |
0.030 |
|
2 |
0.802 |
0.074 |
0.095 |
0.030 |
|
3 |
0.780 |
0.030 |
0.160 |
0.030 |
|
4 |
0.780 |
0.030 |
0.160 |
0.030 |
|
5 |
0.740 |
0.084 |
0.141 |
0.035 |
|
6 |
0.790 |
0.130 |
0.055 |
0.025 |
|
7 |
0.834 |
0.081 |
0.055 |
0.030 |
|
8 |
0.825 |
0.030 |
0.120 |
0.025 |
|
9 |
0.818 |
0.034 |
0.113 |
0.035 |
|
10 |
0.890 |
0.030 |
0.055 |
0.025 |
|
11 |
0.880 |
0.030 |
0.055 |
0.035 |
|
12 |
0.740 |
0.130 |
0.100 |
0.030 |
|
13 |
0.780 |
0.130 |
0.055 |
0.035 |
|
14 |
0.740 |
0.085 |
0.150 |
0.025 |
|
15 |
0.802 |
0.074 |
0.095 |
0.030 |
I-optimal Mixture Design
Table 5
Please notice, in table 6, that the confidence limits on the random point are 41% wider
for the conventional design than for the I-optimal design. Also notice that the
average confidence limits for the conventional design are 16% wider than for the I-optimal
design. The I-optimal design also has a lower IV.
|
|
Conventional |
I-Optimal |
|
95% CL for random point |
+/- 2.58 |
+/- 1.83 |
|
Average 95% CL |
+/- 2.08 |
+/- 1.79 |
|
Integrated Variance (IV) |
0.69 |
0.55 |
Summary
Table 6
Case three has 4 continuous process factors. X4 is restricted to 2 levels.
The factor limits are,
X1 = 2 - 4
X2 = 0.0 - 3.0
X3 = 1.0 - 3.4
X4 = 4 or 6 (only 4 and 6 may be in the design, but the model must predict from 4 to 6).
The experimenter required the constraint, 10 < (2X1 + X4) <12. The
experimenter also required these two trials to be in the design:
X1 = 2, X2 = 0, X3 =
3.4, X4 = 6, and X1 = 4, X2 = 0, X3 = 3.4, X4 = 4.
The model is,
Y = b0 + b1X1 + b2X2 + b3X3 +b4X4 + b12X1X2 + b13X1X3 + b23X2X3 + b11X12 + b22X22 +
b33X32.
The conventional design is listed in table 7 and the corresponding I-optimal design is
listed in table 8. Table 9 summarizes the comparison of the two designs.
|
Trial |
X1 |
X2 |
X3 |
X4 |
|
1 |
2.00 |
0.00 |
3.40 |
6.00 |
|
2 |
4.00 |
0.00 |
3.40 |
4.00 |
|
3 |
2.00 |
0.00 |
1.00 |
6.00 |
|
4 |
2.00 |
1.50 |
2.20 |
6.00 |
|
5 |
2.00 |
3.00 |
1.00 |
6.00 |
|
6 |
2.00 |
3.00 |
3.40 |
6.00 |
|
7 |
3.00 |
0.00 |
1.00 |
6.00 |
|
8 |
3.00 |
0.00 |
2.20 |
4.00 |
|
9 |
3.00 |
1.50 |
3.40 |
4.00 |
|
10 |
3.00 |
1.50 |
3.40 |
6.00 |
|
11 |
3.00 |
3.00 |
1.00 |
4.00 |
|
12 |
3.00 |
3.00 |
2.20 |
6.00 |
|
13 |
4.00 |
0.00 |
1.00 |
4.00 |
|
14 |
4.00 |
0.00 |
2.20 |
4.00 |
|
15 |
4.00 |
1.50 |
1.00 |
4.00 |
|
16 |
4.00 |
3.00 |
1.00 |
4.00 |
|
17 |
4.00 |
3.00 |
3.40 |
4.00 |
Conventional Design with Constraint
Table 7
|
Trial |
X1 |
X2 |
X3 |
X4 |
|
1 |
2.00 |
0.00 |
3.40 |
6.00 |
|
2 |
4.00 |
0.00 |
3.40 |
4.00 |
|
3 |
3.00 |
1.55 |
2.27 |
6.00 |
|
4 |
4.00 |
3.00 |
3.40 |
4.00 |
|
5 |
3.00 |
1.43 |
3.40 |
6.00 |
|
6 |
3.00 |
1.32 |
3.40 |
4.00 |
|
7 |
2.00 |
3.00 |
3.40 |
6.00 |
|
8 |
4.00 |
0.00 |
2.08 |
4.00 |
|
9 |
2.00 |
0.00 |
2.07 |
6.00 |
|
10 |
4.00 |
1.81 |
1.00 |
4.00 |
|
11 |
3.00 |
1.55 |
2.27 |
6.00 |
|
12 |
3.00 |
0.00 |
1.00 |
4.00 |
|
13 |
3.00 |
0.00 |
1.00 |
6.00 |
|
14 |
3.00 |
3.00 |
2.04 |
4.00 |
|
15 |
3.00 |
3.00 |
1.00 |
6.00 |
|
16 |
3.00 |
1.55 |
2.23 |
4.00 |
|
17 |
2.00 |
1.82 |
1.00 |
6.00 |
I-optimal Design with Constraint
Table 8
Notice in table 9 that the confidence limits on the random point prediction for the
conventional design are 81% wider than for the I-optimal design. Also notice that
the average confidence limits on the predictions for the region of interest are 13% wider
than for the I-optimal design. The IV for the I-optimal design is also smaller.
It is interesting to note that the conventional design here is D-optimal. D-optimal
designs provide narrow confidence limits on the b coefficients, rather than on
predictions. Narrow confidence limits on predictions are more important in
industrial experiments.
|
|
Conventional |
I-Optimal |
|
95% CL for random point |
+/- 1.23 |
+/- 0.68 |
|
Average 95% CL |
+/- 1.83 |
+/- 1.62 |
|
Integrated Variance (IV) |
0.41 |
0.32 |
Summary
Table 9
This paper has shown I-optimal designs to be superior to conventional designs for
industrial experiments using three real cases. I-optimal designs are superior
because they give narrower confidence limits on predictions, on average, producing higher
quality predictions of product performance.
----
Math Options creates custom I-optimal designs for a fee, but better yet, sign up at http://www.mathoptions.com for the free teleclass
to learn how to create your own.
Computer
Generated Experiment Designs
|
|
You will learn what computer generated
experiment designs are, how they can
benefit you, and how you can generate your
own for free. |
|
WHEN? - Tuesday, November 23, 1999, 10 AM
PDT (West Coast). (1 PM East Coast). |
|
COST? - FREE! (but space is limited, so
please register early) |
|
This class will be taped. |
.You may contact Bill in the following ways:
4702 Camano Place, Anacortes, WA, 98221
1-888-764-3958
Bill@MathOptions.com
---------------------
The Internet has made it convenient for many industries to agree on a common data
interchange structure for certain chores. The development of XML (the eXtensible
Markup Language) has greatly simplified the construction and maintenance of such a
structure. That has encouraged us to create a prototype of such a structure for
experiment designs.
If you have opinions about the inputs and outputs of design engines, we would like to hear
about them.
http://www.processbuilder.com/doe/Software/designDTD.htm
****************
The last public classes of the year are going on as this is written. If you
couldn’t get into those, make plans now to go to one of the first sessions of the new
year.
These classes are sponsored and run by Math Options (http://www.mathoptions.com).
Check the Math Options web site for full details.
|
Jan. 18-20, 2000 |
San Francisco Bay Area, CA |
|
|
Feb. 1-3, 2000 |
San Diego, CA |
|
|
May 17-19, 2000 |
Anacortes, WA |
|
|
May 22, 23, 2000 |
Anacortes, WA |
****************
XIV Compstat
Utrecht, The Netherlands
21-25 August 2000
http://neon.vb.cbs.nl/rsm/compstat
--------
Central Michigan University --- Tenure-Track Statistics Position
The Mathematics Department is seeking qualified applicants for a tenure-track position at
the Assistant Professor level in Statistics starting in August 2000. Candidates should
have a recent Ph. D. in Statistics, show evidence of having conducted research in
Statistics, and have effective communication skills. The successful candidate will be
expected to teach graduate and undergraduate statistics and mathematics courses, to
conduct research in statistics, and to apply for external funding. We are especially
interested in individuals with research interests that overlap existing research of the
faculty.
The usual teaching load is nine semester hours. Salary is competitive and benefits include
university-paid retirement, medical, dental, disability, and group life insurance.
Please send a letter of application, curriculum vita, transcript, and three letters of
reference to:
Professor Sidney Graham, Chair
Department of Mathematics, Central Michigan University,
Mt. Pleasant, MI 48859.
Phone: 517.774.3596, fax: 517.774.2414,
e-mail: math@cmich.edu, web site: www.cst.cmich.edu/units/mth/.
Screening will begin on October 15, 1999, but applications will be
accepted until the position is filled.