We conduct experiements to improve process performance and product quality. The company’s default analysis method of choice has always been to use an ANOVA to pick the best setting among a small set of options. To be correct, this would be considered a one factor experiment however typically it is viewed as a way of choosing the best of a set of recipes where the recipe may actually change more than one factor at a time.
Since the beginning of the year when I came on board there has been a big push to use correct application of DOE especially when changing more than one factor at a time. Fundamentally this methodology invovles predicting a regression model that describes the response(s) as a function of the variable factors of choice.
There has been some resistance within the company, but we are excited to be moving forward and soon begin analyzing some of our first few experiments. Jinu Antony’s book Design of Experiments for Engineers and Scientists is a brief but excelent resource. Chapter 4 gives a great introduction to the methodology starting with the barriors to successful exeriment design. It continues with a breif, but sufficently comprehensive outline for practical experimetal methodology from start to finish. Chapter 8 is also a great chapter “Some useful and practical tips for making your industrial experiments successful.”
It is in secton 8.1.9 that this book introduces the question “How many exprimental runs are required to identify significant effect(s), given the current process variation?” This has been a question on my mind since we have started partly because we are limited by our process and because of the way things are done at the company. These constraints encourage us to limit the total count of all samples of all variables and levels in an experiment to 25 or if necessary multiples of 25.
Here at the company I have caused quite the debate about how to answer this question. I’ll present my solutions in detail later on. Unfortunately I have not found an all inclusive method to precisely determine the sample size because there are so many ways to design an experiment. Certianly your confidence (alpha and beta) come into play along with historic variation due to natural processes. The response and it’s detectability are also dependant on how small of a delta relative to the noise as well as the distribution of the response.
Furthermore, when we conduct experiments it would be ideal to identify critical responses that are expected to move or at least ones that we would like to determine if they moved or not. Alternatively the approach has typically been to run an experiment and then test ALL responses that we measure. The count of all these responses is on the order of 1000+. To my understanding, the main objective is to first determine which group best meets the needs of the experimenter’s goals and objectives then observe all other responses to ensure they don’t deviate beyond the current process variation. If there are any shifts or unexpected changes in any of the responses then their anticipated changes and trade offs can be understood.
Soon I’d also like to go into more detail about the challenges of DOE with multiple responses. I’ll also include solutions I’ve found so for for weighting and automatically selecting which responses are significant and worth investigating further. I’d also like to compare this process with experimental analysis using ANOVA. Hopefully I can establish a statistically sound method that is easy enough to follow for any of the company’s employees that have time constraints or who currently lack the depth of academic understanding.