Publisher’s Foreword
How to Institute DOE in Your Company
Short Course Notes
DOE and Statistics on the Net
Beyond the Formula Conference Report
* * * * *
* * *
The most common reason for DOE not working is still the
simple failure to apply DOE. We spend a lot of time arguing over
technical details, but we know that most of the people that we are arguing with
are getting far better results than any ad hoc one-factor-at-a-time (OFAT)
approach can possibly achieve.
Next month, we will go back to discussing the details.
Dr. Selden B. Crary has promised an article on his recent work comparing classic
designs and I-Optimal designs.
And the managers that I have said so many bad things about
below? Obviously, these are not the same managers that read DOE
Perspectives. The managers on this mailing list are all modern day
heroes, but every large organization has at least a few of the type lambasted
below.
* * * * *
* * *
You see a lot of articles around about how to IMPLEMENT
Design of Experiments (DOE), but that is the easy part. You need guidance
of a trained professional who will help you decide where and when to use it, you
need a training plan, and you need some software. It costs a little money,
but it is no harder (or easier) than integrating any new technology.
I don't mean to minimize that in any way, but I do want to
point out that it is rarely the quality of a new technology that determines
whether or not it will be adapted. DOE works, but I don't need to sell
that to the readers of this newsletter. But as certain as we all are, no
one can deny that design of experiments faces resistance even in environments
where it is a proven tool. Every research scientist or engineer who has
had a major success from DOE can tell you story after story of how management
still wanted problems solved one-factor-at-a-time.
Having the evidence is critical, but managers are human
beings and human beings are gifted at ignoring evidence. When your manager
is handed a problem, he or she really wants to believe that you know the answer.
Hearing that you need to sit down and make a plan, run trials, take measurements
means that the problem cannot be solved today. Management always wants to
hear that you have a short cut.
The answer is that DOE is the short cut. DOE's beauty
is that it allows you to find the best answer much more quickly and
inexpensively than ANY number of random stabs. That's the part that your
manager probably doesn't get. I call it the "let's jump off the cliff
because we don't have time to climb down" mindset.
It's not your manager's fault -- lots of corporate cultures
make this mindset a condition of inclusion into the ranks of middle management,
but you need to take responsibility for that attitude and change it. The
alternative is to be continually bullied into doing the job wrong.
This is a process, and like every process should start with
a written statement of the purpose. What do you intend to accomplish and
why? Here's one possible statement
"As a result of this process, when management presents
me with a problem to solve, the first question they should ask me is what budget
and other resources I need for my experiment. Management should come to me
to find the answers that no one knows rather than thinking of me as a mere
repository for previously discovered information."
I'm sure you can come up with a better way to say it.
What you want is your corporate culture to support a rational step-by-step
approach to product design, problem solving, and troubleshooting. You want
to avoid being asked to run around like a chicken with your head cut off.
You want them to treat you like a professional running a problem-solving system
not like a wizard expected to wave a wand and pull a solution out of thin air.
Note that you CANNOT win if you allow people to start
thinking that it is your job to know the answer. That perspective demeans
the real value that you bring to your organization.
The whole problem here is to change the behavior of other
people, your management. How can you do that? It's a nightmare for
most people to change themselves. Changing other people is almost
impossible, but you can't win in a corporate environment without the cooperation
of others.
The key is to change as little as possible, and make each
change feel like the next step along the intended path. How do you do
that? Determine what they are doing, and find out what they intend to do.
Start with the top, and work your way down to your local management.
You may understand your local management very well, but
this kind of project must be undertaken from the top down. (Deming
reportedly would not even talk to a company unless the president of the company
was willing to set aside a full day to talk to him.) The only real
alternative is to be a DOE guerilla. This may bring you lots of personal
satisfaction, but it means that you will always be swimming upstream and will
not get the credit you deserve.
Fortunately, top management is very transparent in their
goals. They want to please stockholders and customers, usually in that
order. Thus, if you want to get the president of your company to be a DOE
enthusiast, you have to show how DOE addresses stockholder and customer hot
buttons. Let's start with the stockholders (even though the customers ARE
more important).
Stockholders want to see the value of their stock go up and
dividends increase. From the perspective of the stockholders, this is the
job description for the chairman and the president of the company. You
need to sell the value of DOE in these terms. Fortunately, DOE is up to
the task. Here's a starter list.
Management wants to decrease costs and increase revenues
because that pleases stockholders. See how easily DOE maps to those goals.
And of course, the job of management is to manage. OFAT is simply not a
managed approached. DOE, by definition, uses a formal written plan that
has a definite beginning and a definite end.
Do not be reluctant to spell out the advantages in explicit
detail. Assume denseness on the part of either your manager or the people
to whom your manager needs to explain things. Past success with DOE is no
guarantee that it will be management's first thought.
If you are like the typical working scientist, there are
several layers of management between you and the president of your company.
What can DOE do for them? The simple answer is that middle managers also
ought to be focused on the needs of the customers and stockholders. In
real life, this means that they are really focused on the needs and desires of
their immediate management. This is why you need to build from the top
down. Get the top management to commit to backing DOE, and most middle
managers won't even think of bucking it.
The concerns can become considerably more personal by the
time it comes down to your immediate supervisor. Your manager does not
want to give the same answer to the same problem-status question twice, at least
not to upper management or customers. Your manager always wants to be able
to say that the problem will be fixed immediately and never occur again.
This can make your manager to make unreasonable promises on your behalf.
I have seen a manager refuse to approve a budget for
experiment that cost less than $10K and a time-frame of less than a week, only
to be forced to resort to the experiment after months of no progress using an
OFAT approach. When the experiment was finally run, the quality of the
process and the resulting product were significantly improved while saving
several dollars per unit. The company was selling several million units of
this product per year, and increased their market share as a result of
manufacturing the improved product. The manager's resistance literally
cost the company tens of millions of dollars.
What was astounding about this? The company already
had a long and spectacular history of success with DOE. It was a fairly
early adapter of DOE, and the successes were well-publicized within the company.
Moreover, the obstinate manager mentioned above had always been the manager of
the group where most of the successes had occurred, and the scientist was the
one who had made the most significant breakthroughs. The manager's
behavior seems so dysfunctional that you might wonder how he keeps his job, but
I have heard different variations of this story hundreds of times from different
people. (Actually, I even heard a similar report about the same manager
just a few months later.)
This is frustrating. There is nothing worse as an
employee than to feel like your manager is one of the obstacles to your success.
What could possibly change the unnamed manager in the above story? It brings us
back to upper management. When a problem is identified, upper management
needs to ask for a plan and a budget for solving it. A designed experiment
is a plan and a budget that works.
Upper management does not need to review every detail of
such plans, but it must insist that these plans be created and followed.
The alternative is that no guidance whatsoever will be in place. Deming's
greatest contribution was that he showed that defects were rarely the fault of
the individual; they were the fault of the process. The manager
cited above is not the problem. If the process required the development
and presentation of a step-by-step solution, no one would ever try to present
and defend an OFAT approach. Obviously, the process in this company keeps
asking managers to shoot from the hip, and they respond by asking their people
to shoot from the hip.
If your manager is being asked to provide a planned
approach and you have a planned approach that fills the bill, your job is easy.
If your manager is being asked to shoot from the hip and you want to follow a
planned approach, you have an uphill battle. To win your manager over, you
need to control the expectations that your manager is trying to live up to.
As long as your manager is being asked to find the answer
right now without taking the time to go through the process, you are going to be
asked for the same thing. The sad thing is that if you are a working
industrial researcher, your immediate manager was most likely a working
industrial researcher not that long ago. A manager in this position
probably understands that a designed experiment is the most efficient way to
solve the problem, but few managers act in accordance with that belief when
asked to solve a problem “right now”.
In fact, most of these problems go away when the manager
shows a little backbone. Unfortunately, you are among the lucky few if you
have the rare middle manager who will stand up to upper levels and say:
“This is what it is going to take."
Let’s assume your manager is not stupid (although
promotions are often brain-damaging). The typical manager is a gutless
wonder because that is the strategy that works in the multi-national corporation
of today. It’s not the way to the top of the company, but it is the way
to keep that cushy middle-management job that sends the kids to Stanford.
How can you undermine that comfort zone without making your
manager feel that you are not on the manager’s side? I think that one
way to approach this is like an insurance salesman. We all know that OFAT
and trying to pick an answer out of the air just doesn’t work just like we
know that we are going to die some day. The insurance salesperson says:
“Here comes a disaster. What can we do together to make sure that it
does not destroy everything you’ve worked for?”
In your manager’s case, the coming disasters are certain,
repeatable, and increasing in severity if the manager continues to make promises
that depend on your already knowing the right answer. Every manager knows
that upper management always wants to hear that the problem can be solved a cost
of little time or money. This where guts come in.
If your manager’s boss says: “I want an answer to this
XX problem by tomorrow,” and your manager says: “Can do,” that’s the
gutless answer if “can do” means “I’ve just bought two days” or
“there will be another emergency by next week so I’ll never get asked about
this again.” A manager with backbone and knowledge of the process steps
up and says: “No one knows the answer to this problem, but I’ll tell
you exactly what it is going to take by this time tomorrow.”
This is the critical moment in the process. The
manager that stands up for doing things the right way has put the company and
you on the most efficient path. Upper management may want the problem
solved by tomorrow, but what management, the company, and the stockholders NEED
is to reach the best possible solution in the shortest possible time. They
also need the resource cost to be predictable even when those costs are more
than they want. (Aren’t they always?)
If your manager caves in to the pressure for an immediate
answer at this time, only phenomenal good fortune stands between those promises
and having to report failure to keep those promises at the next meeting.
In any organization large enough to support a research group, it seems unlikely
that the meetings would be so haphazard that the problems are not followed up
on. More likely, your manager has to go to the next meeting and say that
despite all your best efforts you haven’t found the right answer yet.
Eventually the problem gets forgotten about or solved by
finally biting the bullet and taking a planned approach (DOE), but in the
meantime, your manager is selling you down the river. Each report of
failure is saying “I guess I don’t have the right person on this problem.
If I had the right person, I’d have the answer by now.” The manager
looks bad because his or her job is defined in this context as finding the
person that knows the answer, but the focus of negative attention is passed from
the manager to you.
Can you stop this? You have to, and the time to stop
it is right away. When Joe or Jill Manager comes to you and says:
“What do we need to change to solve this problem?”, you have to be the one
that say: “No one knows the answer to this problem, but I’ll tell you
exactly what it is going to take by this time tomorrow.” Otherwise, you
are setting yourself up for reporting failure and convincing your boss to find
someone that does know the answer.
OK, that sounds good, but we all know bosses that are never
going to buy into this. You could jump up and down until and argue until
you are blue in the face, and some bosses will still say: “We don’t
have time for that DOE nonsense. You’re the expert. Tell me what
you think the answer is.” What do you do in this situation?
Do an experiment design. Am I telling you to disobey
your boss? No! I am saying that you should do a design, then run one
of the trials from the design every time you are told to pick an answer.
Sure, there are risks with this approach, but this does allow a degree of sanity
to enter the process.
The worst risk is that you’ll settle for a less than
optimal result because of stumbling on an answer that solves the problem of the
moment. There are other risks. You may have to settle for a design
that is less than optimal because you don’t have the necessary resources to
come up with the optimal design. You may corrupt the run order by picking
the trial of the moment using your intuition rather than the run order
recommended in the experiment design. Time may introduce additional noise
into the experiment.
Obviously, you would like to run the experiment start to
finish with everything under control from the start, but with each additional
trial that is run, the costs of completing the experiment go down.
Eventually, you will have run enough trials to make some predictions. If
nothing has invalidated the data in any way, these predictions may be just what
you are looking for.
Never run a trial that unless the specifications for that
trial are taken from an experiment design.
Let's say that you understand that instituting DOE as the
standard way of doing things is going to be a challenge, but still want to take
it on. If you are determined enough, you will eventually get to run
designed experiments. That may seem like the hard part, but the key is to
capitalize on the success of those experiments.
What does that mean? It means that somebody needs to
tell upper management and the stockholders what this experiment means to them in
terms of their interests. They are interested in profit, market share,
etc.
It’s all easier after the fact. Before the money is
spent, every manager between you and the Board of Directors is motivated to
resist spending any money on the problem. After the money is spent, every
manager between you and the Board of Directors is motivated to prove that
spending that money was the smartest thing they could have done. All you
have to do then is help point out how smart they are.
It's all easier when you do have a technology that delivers
numbers and pictures. In the ultimate corporate picture, the sale needs to
be made to the stockholder, and numbers and pictures are the language that
stockholders understand. DOE is lucky here. Scientists are already
somewhat trained to deliver information to their management, and it doesn't take
much experience to realize that management needs pictures.
Use these pictures and graphs to show upper management and
stockholders how the experiment affected the bottom line. Put in as much
effort in reporting an experiment as you do into getting approval for an
experiment. That will make getting approval for the next experiment that
much simpler.
Scientists that get to do designed experiments time after
time are scientists that are good at making everybody in their chain of command
look good. When you prepare your report, prepare it so that your manager
will want to deliver to the next level of management and so they can report it
stockholders. Make it easy for people to tell your story. Make
people want to tell your story.
* * * * *
* * *
Note that our limited resources mean that these are the
ONLY public classes that we are offering through the end of the year.
Space is limited so sign up now. Note that completion of an approved basic
class is a pre-requisite for attending the advanced class.
These classes are sponsored and run by Math Options (http://www.mathoptions.com).
Check the Math Options web site for full details.
Objective Experiments for Mixtures and Discrete Factors
(AKA Advanced DOE Workshop)
Anacortes, WA
September 23 -- 24.
Basic Statistics for Industry
Las Vegas, NV
Nov. 1
Performing Objective Experiments
(AKA A Modern DOE Workshop)
Las Vegas, NV
Nov. 2-4
The Chance Database -- Home of the very interesting Chance
News
http://www.dartmouth.edu/~chance/
Data Vision's Links Page
http://www.datav.net/links.htm
Interesting without being Overwhelming
* * * *
* * * *
Robert Johnson reports that the Third Annual Beyond the
Formula conference held last month in upstate NY was a great success. The
Chance News reported similar impressions.
This is worthwhile conference supporting and promoting
statistics education. It’s unbelievably cheap, and upstate NY is
beautiful in August. Start thinking about next year.
* * * * *
* * *
Thanks for your attention.
Sincerely yours,
Al Corwin
President
Process Builder
http://www.processbuilder.com/doe/
mailto:abcorwin@processbuilder.com