Experimental Design
As can be seen that the action of test market occurred in actual e-commerce and the graphic design as one variable are manipulated while the effect to visitors’ purchases as dependent variable is observed. So in concept, it is called Field Experiments. The report circles the experimental design to assess the approaches used or to be used by the research.
Data Collection and Analysis
The research makes fully use of the computerization to automatically record the behaviours of the visitors who browse the two kinds of graphics within a month. At the same time, the system sorts the data into diverse classification. The research can interpret data into expedient information.
Sampling
All the visitors who visits website can be regarded as samples, are randomly selected and assigned into test and control group within one month.
IV Findings
Comment on research design
The basic research design can be defined as After-Only with Control Group Design in concept. Anyhow, there are some strengths and weakness raised up as below:
Strengths
- Majority of the extraneous effects are eliminated with the randomization, so the causality between the treatment, graphic design and observation result is highly valid.
- Since the experiment applies to all visitors to the web site in that month, no problems occurred on selection bias of choosing appropriate test group and control group.
- The forms of on-line experiment used for e-commerce business makes DVDs.com directly observe the true impact of graphic design while help them save cost and time.
- Little man-made errors because the records are tracked automatically by the computer.
- Easy to attain cooperation from individuals or organizations.
Weaknesses:
- Measure errors exist in the randomization .The treatment is likely repeatedly exposed to the same visitors when they visit the website within the month. Also since tracking system just identifies the terminals by IP address rather than the real customer, therefore, one IP address doesn’t stand for one customer.
- Lost of respondents will cause the problems in experimental mortality. some customers may not visit the website in that month, so the accuracy of outcomes will be effected on.
- One-month period is too short, it should be run long enough for an adequate number of repeat purchase cycles to provide a measure of the “staying power” of each treatment.
Assessment of difference
Differences
- Approximately 50.19% of all visitors visited the new graphic design which is a little more than the proportion of people who visit the old one within a month.
- In terms of the amount of the visitors exposed to each graphic, the percentage of made a purchase in the current one is around 33.40% which is 1.1% less than 34.5% of it in the new one.
- The purchase differences of two graphic design is shown as followed:
- The percentage of repeat purchasers, last 30 days, emerged that 22.40% was the purchasers who entered to the current graphic, which was higher rate than 18.50 per cent of those who signed in the new graphic.
Conclusion
- No apparent influences on the use of designs because of a little nuance in quantity of the visitors who visited the two graphic designs respectively.
- The percentage of who made a purchase and the average total purchase of visitors in new graphic groups is higher than that in old look.
- Average music and video purchase both increase by 9.8% and 10.6% respectively. It implies, at lease in the short term, choosing the new graphic is preferable.
- Overall purchase data represent the causation between the new graphic design and the merit on sales.
managerial perspective
From the above data analysis, it is obvious that the new graphic design is better than the current graphic design in attracting customers to make a purchase. However, the result is on the contrary in data of the percentage of repeat purchasers. What is the reason for the new graphic’s failure in attracting the customers to purchase again? Is it the problem of the graphic design or is the way we collect and analyze the data not valid enough to draw a conclusion? We are going to evaluate the differences between measures of the two graphic approaches from a managerial perspectives :
1. Though the amount of visitors exposed to two treatments(i.e. graphic design) is not exactly the same, the error caused by randomization is in a normal/acceptable level. Moreover, the distribution to the product purchased is nearly the same, i.e. the ratio of music purchase to video purchase is about 1:2 in both group. Consequently, all these can prove again the research method is of high valid and causal, because the treatment only have an effect on the purchase column rather than other changes (e.g. purchased product distribution).
2. The percentage of repeat purchasers, last 30 days, has been found a deviation, none the less it can prove nothing with the restricted time. It might attribute to the limited test period for customers have not yet familiar with the new graphic design. Furthermore the research recorded the data whether the visitors purchase or not in one month’s period, those visitors who visit at the end of the 30 days should be also observed in the following month. That is to say, though the research method is of high valid, the period of observation is not long enough, which will also mislead the decision making.
3. No apparent differences between the test results of the two measures though they are not the same. Therefore the unique research method is not enough to indicate the effectiveness of the two graphic designs. There is a need to consider other possible causal factors, such as the reasons why visitors make a purchase and why not. Considering the time and cost issue, attaching a mini online survey for collecting visitors’ opinions is preferable.
V Conclusions and Recommendations
As a whole, the projected experimental design is succeed in demonstrating concomitant variation and the appropriate time order of occurrence. However, there are still some errors in the process of the experiment:
- Errors of measurement timing occurred in the research design process. Therefore one month is not enough time to indicate the effect of the experiment treatment.
- Analysis the experiment only by the collected data will eliminate other possible causal factors. The popularity of the music and the content of the video are more decisive to the customers’ purchasing than others. Ignoring such effects, the data will be lack of reliability and accuracy.
- Error occurred in the randomization measurement. Either the current or new graphic randomly appears to the visitors, this means the customers might have possibly seen both of the two graphic designs before he made a purchase, therefore it is hard to tell which graphic lead to his purchase. Hence, any recorded data can not be accurate and valid.
In conclusion, the employed test can not fully indicate the effect of the experience treatment so that it is unlikely to be used for final decision. Thus, DVDs.com is suggested to adopt the following recommendations:
- Retaining the current graphic design and review the problems mentioned above.
- Carry out a mini on-line survey , which include no more than three questions in collecting information of customers’ attitudes, to assist the outcomes of the test .
- Defering the test period to 60 days to meet a full cycle of purchase as well.
VI Appendices
- The results of the test employed by the DVDs.com see page 280 of textbooks.
- The approach for testing
After-Only with Control Group Design
Basic Design:
Experimental Group: (R) X O1
Control Group: (R) O2
Sample: Random sample of visitors that visit the DVDs.com, are randomly assigned to the test and control group.
Treatment: (X) Placing the new graphic on website in the experimental group for a month.
Measurement (O1, O2): actual purchase data.