The given article describes an experiment held with a random assignment of people to groups and statistical controls for confounding. The purpose of the research was to explore how people think about and act to manage future risks of product failure. Besides that, the researchers tried to investigate the key variables associated with risk management strategies.
Extraneous variables (those that may influence or affect the results of the treatment on the subject) were used. A constant variable was introduced through clearly specified outcome probabilities. Independent variables, which are variants of decisions the respondents had to make, were established by the researchers; however, dependent variables were not properly defined even though they may have made a significant impact on the results of the experiment. I may be mistaken, but based on this I’d say there is no strong evidence of design control. No matter that the design was based on random assignment and statistical controls, there are many factors that hadn’t been properly taken into consideration, such as people’s age, sex, past experience, education, occupation, social factor, etc. and their impact was not properly evaluated when interpreting the results.
Besides that, in my opinion, when analyzing and interpreting the collected data, the authors had to assess similarity of some particular (sub)groups (which, by the way, had not been singled out neither) and detect possible tampering with randomization process. Hence, if the groups were defined, it would be possible to conduct further factoral design in order to study different interventions on the same population and/or potential interactions between several populations. This would provide us with a wider range of statistical information and, hence, would allow the study to be more accurate, complete and extensive.
Anyway, the good thing about the given research is that it was carried out using randomizing between individuals, so factors that may influence outcome, are minimized or eliminated, notwithstanding the fact that all the responses are much influenced by individual personal experience. Williamson and Ranyard also underline that memory and prior experience play a significant role in risk decision-making and the suggested utility model is an inadequate description of choice process involving risk in the real world.
The bottom line here is that experimental design is intrusive and difficult to carry out in most real world contexts. And, because an experiment is often an intrusion, to some extent an artificial situation had been set up in the given case so that the researcher can assess the causal relationship with high internal validity. If so, then he is limiting the degree to which he can generalize the results to real contexts he hasn’t set up an experiment. That is, he has reduced the external validity in order to achieve greater internal validity.
Analysis of the design of experiments may be built on the foundation of the analysis of variance, a collection of models in which the observed variance is partitioned into components due to different factors which are estimated and/or tested.
In the end, if the situation is right, an experiment can be a very strong design to use. But it isn’t automatically so. My own guess is that randomized experiments are probably appropriate in no more than 10% of the social research studies.
Summary
Experimental methods are finding increasing use in manufacturing to optimize the production process. Specifically, the goal of these methods is to identify the optimum settings for the different factors that affect some particular process. In the discussion so far, the major classes of designs that are typically used in experimentation can been introduced: two-level, multi-factor designs, screening designs for large numbers of factors, three-level, multi-factor designs (mixed designs with 2 and 3 level factors are also supported), central composite (or response surface) designs, Latin square designs, Taguchi robust design analysis, mixture designs, and special procedures for constructing experiments in constrained experimental regions.
Interestingly, many of such experimental techniques have “made their way” from the production plant into management, and successful implementations have been reported in profit planning in business, cash-flow optimization in banking, etc.
References:
Brownlee, K.A.(1960). Statistical theory and methodology in science and engineering. New York: Wiley.
Campbell, D. and Stanley J. (1963). Experimental and quasi-experimental designs for research and teaching. In Gage (Ed.), Handbook on research on teaching. Chicago: Rand McNally & Co.
Cox, D.R. (1958). Planning of experiments. New York: Wiley.
Fisher, R.A. (1935). The design of experiments. (1st ed.) London: Oliver & Boyd.
Williamson, J., Ranyard, R. & Cuthbert, L. (2000). Risk management in everyday insurance decisions: Evidence from a process tracing study. In Risk, Decision and Policy, Vol.5, Number 1, p.19-38.
Winer, B.J. (1962). Statistical principles in experimental design. New York: McGraw-Hill.
Winer, B. J.(1962). Statistical principles in experimental design. New York: McGraw-Hill.
Winer, B. J.(1962) Statistical principles in experimental design. New York: McGraw-Hill.
Williamson, J., Ranyard, R. & Cuthbert, L. (2000). Risk management in everyday insurance decisions: Evidence from a process tracing study. In Risk, Decision and Policy, Vol.5, Number 1, p.1
Williamson, J., Ranyard, R. & Cuthbert, L. (2000). Risk management in everyday insurance decisions: Evidence from a process tracing study. In Risk, Decision and Policy, Vol.5, Number 1, p.20
Williamson, J., Ranyard, R. & Cuthbert, L. (2000). Risk management in everyday insurance decisions: Evidence from a process tracing study. In Risk, Decision and Policy, Vol.5, Number 1, p.23
Campbell, D. and Stanley J. (1963). Experimental and quasi-experimental designs for research and teaching. In Gage (Ed.), Handbook on research on teaching. Chicago: Rand McNally & Co.
Williamson, J., Ranyard, R. & Cuthbert, L. (2000). Risk management in everyday insurance decisions: Evidence from a process tracing study. In Risk, Decision and Policy, Vol.5, Number 1, p.34
Campbell, D. and Stanley J. (1963). Experimental and quasi-experimental designs for research and teaching. In Gage (Ed.), Handbook on research on teaching. Chicago: Rand McNally & Co
Cox, D.R. (1958). Planning of experiments. New York: Wiley.
Brownlee, K.A.(1960). Statistical theory and methodology in science and engineering. New York: Wiley.