We as a group have decided to choose four methods of research, as we want to be able to collect as much accurate data as possible. This is because the information that is going to be collected has to be accurate and has to be detailed in respects of the objective of the tasks.
We have chosen the following research methods:
Questionnaire
The questionnaire is great way to obtain information and opinion, as this research method has a number of advantages and disadvantages when compared with other research methods. The key strengths and weaknesses of questionnaires are summarised in bullet points below. In general, questionnaires are effective research method for efficient collection of certain kinds of information. They are not, however, a comprehensive means of research and should be used to support and supplement other procedures for evaluating and improving aspeects of paticular tasks.
- Advantages of questionnaires
- They permit respondents time to consider their responses carefully without interference from, for example, an interviewer.
- Cost. It is possible to provide questionnaires to large numbers of people simultaneously.
- Uniformity. Each respondent receives the identical set of questions. With closed-form questions, responses are standardised, which can assist in interpreting from large numbers of respondents.
- Can address a large number of issues and questions of concern in a relatively efficient way, with the possibility of a high response rate.
- Often, questionnaires are designed so that answers to questions are scored and scores summed to obtain an overall measure of the attitudes and opinions of the respondent.
- They permit anonymity. It is usually argued that anonymity increases the rate of response and may increase the likelihood that responses reflect genuinely held opinions.
- Disadvantages of questionnaires
- It may be difficult to obtain a good response rate. Often there is no strong motivation for respondents to respond.
- They are complex instruments and, if badly designed, can be misleading.
- They are an unsuitable method of evaluation if probing is required – there is usually no real possibility for follow-up on answers.
- Quality of data is probably not as high as with alternative methods of data collection, such as personal interviewing.
- They can be misused – a mistake is to try to read too much into questionnaire results
- Open
- We have decided to have open-ended questions in the question as some of the information required has to be detailed and in this case it’s going to be detailed opinions on particular tasks. This will enable us to allocated the shortcomings of the website as well as why this particular aspect of the website is lacking in the opinion of the participant.
- Closed
- The main reasons of the closed-ended questions in the questionnaire are mainly due to the fact we want some quantitative information as well as the qualitative. The closed-ended questions allow getting to the exact point and not wasting valuable time and getting information that is not relevant.
Think Aloud
This qualitative research method refers to the verbalized account of a subject’s mental process; in particular, the sequence of cognitive events, or thought processes, between the introduction of a task or problem to the final product. Also known as concurrent verbal protocol and thought listing, it is predominantly used as a research tool in the field of cognitive psychology. It can be thought of as “self report” or “thinking aloud.” This allows a researcher to find out how a person approaches a problem or task and describes the problem solving techniques or interpretations he or she uses. Provides rich, qualitative information about how a person reasons his or her actions in specific situations.
Weaknesses of method, the observer may influence or affect how the person doing the “think-aloud” behaves. The need to give a verbal report of their mental process may change how a subject interacts with the task at hand and affect their decision process. A subject can only report what they are aware of and not the underlying unconscious processes that take place in addition to the verbal decision
“Talk-aloud protocol is based on the simple idea that to get at someone's thought processes while they solve a problem, you get that person to talk aloud what they are thinking, everything that comes into their head, as they solve a problem. This assumes that as someone solves a problem, they retain a small amount of information (7 plus or minus items) in a volatile memory store, their Short Term Memory (STM). The validity of protocol analysis rests on the assumption that if you can get at the contents of someone's STM as they solve a problem, you learn a significant amount about their problem solving process”.
Observation
Observation is very unique way of gathering information as you can see for yourself the problems that different people encounter, while they are carrying out the same tasks. You can see at first hand the problems being encountered and maybe understand why the problem has occurred as you can see why participant made the mistake.
There are essentially two observation methods you can use during usability testing. The first method is ‘unobtrusive’ observation, this where you focus on observing what the test user does and avoid doing as much as possible from influencing the participant by explaining the design or asking questions. The second method is ‘obtrusive’ observation, which means you are allowed to describe the design choice, ask questions, or communicate with the test user in a conversation.
We as a group have decided to use the ‘Unobtrusive’ observation as to ‘Obtrusive’ observation as we do not want to influence the research results whatsoever as we want accurate information from our research methods. The ‘Obtrusive’ observation requires the observer to interact with the user and may influence the results that are collected. Therefore we as a group have decided to choose ‘Unobtrusive’ observation to collect the information as we believe it is better to capture information without interacting with the participants as we do not want to influence the results in anyway.
- Unobtrusive Observation
- With unobtrusive observation you learn whether people can use the design in an easy and efficient way, and why this may not be the case. You learn how people behave and how people explain their behaviour, which amazingly are two different things. If you want to learn about behaviour, you have to study behaviour even thought that might not be what you want. If you want to learn whether users can use the design you have presented them with, you have to observe how they use it, not ask them what they think of it and this is where behaviour is monitored. Opinions are bad predictors of behaviour, as it has been known in the past that participants only score 1 out of 5 on basic tests but if asked about the system their opinion may not be what you expected if you were going on the findings of the results.
- Obtrusive Observation
- Unobtrusive methods will not tell you what the test users think of the design. This will involve interaction between the observer and participants where the participants are asked questions continually regarding the design of the system and the participant must explain their answers. If you want to learn more about the usefulness and the acceptance of the design, you will have to ask the test users. This method involves opinionated questioning as you will not ask “Do you understand what this means?” as you will be expected to ask “What do you think when you see this?” as you want the users opinion.
Time Measures
Time Measures is an effective way to see how a users interacts with a computer and how long it takes the users to carry out or and finish the task. It is clear the time taken to complete a task is some reflection of the amount of effort the user is making in order to complete or carries out the required task.
The group have chosen to use the questionnaire to collect the required information that is combined with open-ended and closed-ended questions. The think aloud aspect requires the users to say what they are thinking and this way the information gathered is more accurate as to information on a paper just answering questions. The observation method gives the observer the chance to see how the participant behaves during each task and why they are likely to make mistakes that they make. Finally we combine the time measure to our information collecting methods as this allows us to see how long it takes the users to carryout each task and compare novice, intermediate, experts at each task. To identify the main shortcomings of the website Pogo.com and this allows us to use this information to design a prototype that can iron out the mistakes making it more users satisfactory, enjoyable, efficient and effective.
Conclusion/ Recommendations
As a group we have evaluated an existing system using the ten heuristics to appoint the error that consist within the report. To test if these are affecting the user we used a method like questionnaire, talk aloud observation and time measure to test the system to capture the user requirements.
From the user requirement we were able to create a low fidelity prototype which fulfilled the usability of the system.
Improvements
We as a group think that the prototype could be improved by changing the colours and making the errors much clears and giving a popup error message which would stop the user from enter unless the message is triggered.
If we had more time we as a group would have created a high fidelity prototype improving it from the low fidelity prototype. Overall the group has contributed and carried out task well and on time.
Bibliography/reference
Websites
Lecture 3/ Usability/Evaluation Methods/Talk Aloud/Page 15
Books
User and task analysis for interface design
By: JoAnn T. Hackos
Janice C. Redish
Human computer interaction 3rd addition
By: Alan Dix, Janet Finlay, Gregory D. Abowd, and Russell Beale
Effective methodology for the study of HCI
By: J. Kirakowski and M. Corbett
