The Relationship Between Previous Psychology Knowledge, Confidence, and A Knowledge Test in Psychology.
Matt Ott
The Relationship Between Previous Psychology Knowledge, Confidence, and A
Knowledge Test in Psychology.
Abstract
This study sets out to ascertain whether psychology is simple common sense, by testing a relationship between previous psychology knowledge, performance in a knowledge test in the subject and the participants' confidence in their knowledge. 70 psychology students were presented with a multiple-choice test (a replication of the same used by Furnham (1996)) which set out to test their knowledge of a wide range of psychological theories, concepts and beliefs. Participants were to answer all 40 questions and then state whether or not they had done psychology A-level on the answer sheet. By giving an estimate of their score the students also exhibited their confidence in their knowledge. Findings showed that previous knowledge does lead to greater performance on the test, yet does not hold any relationship with confidence, which in turn has a good correlation with performance.
INTRODUCTION
In the modern day, psychology is constantly being criticised as "common sense" by the lay person. This consistent dispute has caused great dismissal of psychological findings as they are often seen by the lay person as simple common sense. Theories drawn by psychological research and findings are often discounted by the lay person as they are seen to be simple common knowledge. Indeed, many psychologists claim that our everyday or "folk" understanding of mental states constitutes a theory of mind. This is called "folk" psychology, which plays a central role in our ability to predict and explain the behaviour of ourselves and others. It should be noted however, that the nature and status of "folk" psychology remains controversial. If psychology is common sense, what would be the expected result? Surely, there would be no distinguishing between the knowledge of skilled psychologists and the lay person. Yet as many studies have found, this is not entirely the case (see Nixon (1925) reference later).
Psychology and common sense are not unrelated, in that psychologists draw upon, study and clarify common sense. Yet this does not mean that psychological theories and hypotheses are all simply common sense, but on the other hand, theories and hypotheses are often conceived via psychologists drawing upon the vast field of lay psychology. Therefore, it can be said that it is the common sense foundation that people use to predict, understand and explain the behaviour of others (Greenwood, 1999; Montgomery, 1987; Stich, 1983) .
It is argued that whether or not psychological findings are merely common sense, psychology is a study of (among other things) lay attitudes and beliefs (Furnham, 1983). Indeed, many psychological studies have been focused at common sense knowledge and studying and analysing them in a scientific environment. Such studies include phenomenological psychology (Schultz, 1964) and ethnomethodology (Garifnkle, 1967), in fact Kohler (1947) believed that common sense holds great importance as an object of study, and Joynson (1947) stated it importance as a source of theories and hypotheses.
Previous research has been done in this field, empirically; via questionnaires that determine how much people know/don't know about psychology, what non-psychologists know about the subject, and the misconceptions people have about it (Furnham, Callahan, Rawles, 2003).
A lot of research has been done in assessing the effectiveness of an introductory course in dispelling common myths about the nature of psychology(Furnham, 1992 ; Mckeachie, 1960 ; Vaughn, 1977). Also in the field of the knowledge, beliefs and superstitions students bring to social science courses, a certain study devised by Nixon (1925) has been replicated many times since, attempting to demonstrate that students arrived at his course with "unsubstantiated beliefs" about human behaviour, yet these change as a result of teaching. This belief would therefore show that without previous knowledge, students would not be able to do as well on knowledge tests in psychology. The results of his study showed a large level of ignorance and misconceptions about psychology. This study has been replicated many times since, including one done by Tupper and Williams(1986), which again showed levels of ignorance in those not knowledgeable in psychology.
It is interesting to note that studies have also been carried out involving tests of psychological knowledge usually given to students to establish their beliefs/knowledge of a broad area of psychology. The most widely cited misconceptions test is that of Vaughn (1977), yet in 1991 McCutcheon devised a test that did not use the true/false based test that had been previously used. Results indicated that there are alarming levels of misconceptions concerning psychology among introductory students. In the same way, Furnham(1992) tested 250 prospective psychology students who were still at school, and found their knowledge of psychology was very uneven, yet in general fewer then half knew the correct answers to the questions. In this test, he used items from a test devised by Colman (1988) that attempted to explain what psychology is. It can therefore be seen from previous studies that there is a relationship between previous psychology knowledge and performance on tests testing conceptions of psychology and theories, beliefs and attitudes associated with it.
Yet it is clear that prior knowledge doesn't seem to be enough, no matter how confident people might be in their knowledge.
Little study has been devoted to the knowledge and confidence in their knowledge of students who have deliberately chosen psychology as a subject of study. How would the relationship differ in this case? The students will have varying levels of prior knowledge and subsequent levels of confidence in their knowledge. Furnham (1992) showed there to be some interesting differences particularly in their misconceptions, but there were fewer differences than may be expected from chance alone. ...
This is a preview of the whole essay
Yet it is clear that prior knowledge doesn't seem to be enough, no matter how confident people might be in their knowledge.
Little study has been devoted to the knowledge and confidence in their knowledge of students who have deliberately chosen psychology as a subject of study. How would the relationship differ in this case? The students will have varying levels of prior knowledge and subsequent levels of confidence in their knowledge. Furnham (1992) showed there to be some interesting differences particularly in their misconceptions, but there were fewer differences than may be expected from chance alone.
This study sets out to try to document the relationship between previous knowledge of psychology, confidence in the knowledge and performance in a revised McCutcheon test. Psychology students in their first year at University have varying levels of knowledge and all have an interest in the subject, and so are possible fruitful areas of study. Therefore it hypothesised that-
1) There will be a correlational relationship between having done A-level psychology or not and performance on the multiple choice psychology test.
2) There will be a correlational relationship between judgments of confidence and performance on the multiple choice test.
3) There will be a correlational relationship between having done A-level psychology or not and the judgments of confidence in that knowledge.
4) There will be a difference in the scores from a psychology test between students who did psychology A-level and those who did not.
5) There will be a difference in the judgments of confidence between students who did psychology at A-level and those who did not.
METHODOLOGY
Participants-
70 first year psychology students were used in the experiment, all of whom were from the University of Bath with ages ranging from 18 to 40. The participants contained nine males and 61 females. 23 had no previous psychology knowledge, in that they did not do psychology at A-level, compared to the 47 who had done psychology A-level, yet all had done background research on the subject as prior-to-course reading.
Design and Materials-
A within participant design was used in this quasi-experimental study, as there was no control over whether or not participants had done psychology A-level. In order to test the relationship between previous psychology knowledge, confidence and performance in a psychology knowledge test, participants completed a 40 questioned multiple choice test (Appendix 1) based on one devised by McCutcheon (1986). In this study the test was a replication one used by Furnham (1996). It should be noted that certain questions previously in the original Furnham test were removed as they either were angled at an American audience, were ambiguous or simply out of date.
The 40 questions used covered a wide variety of psychology, in order to test fairly for psychological knowledge. The performance on the test was one of the dependant variables in the quasi-experimental hypotheses. The other dependant variable (in the quasi-experimental hypotheses) was the judgment of confidence by participants. It was devised that each participant was to write their expected score on the test paper, i.e. give a judgment about how many answers they thought they had got correct. By doing this, the confidence of each participant could be measured. In order to ascertain whether or not the participant had previous psychology knowledge, each student was required to note on the test paper whether or not he/she had done psychology A-level. This was the independent variable.
Statistic analysis (Pearsons testing for the correlational hypotheses and an independent T-test for the Quasi-Experimental hypotheses) was used to test the various correlations and relationships between previous knowledge, judgment of confidence and score on the psychology test.
Procedure-
Once briefed, participants were given the 40 item multiple choice test answer sheet on which to document their answer (there were a possible 4 answers which they had to choose from for each question). They were firstly instructed to write whether or not they had done psychology A-level on a blank section of the paper, by writing simply "did psychology A-level/ didn't do psychology A-level". The questions were then read out to the group, around twenty seconds was then given to the participants to think, after which the question was read out again. The participants then had around ten seconds to decide on an answer. They were to mark either a, b, c or d as their answer on the test sheet. Once all questions had been asked, the participants were asked to write their expected score on the paper before receiving their real mark. The scores were then self-marked as the correct answers were read out. The total mark was then written on the sheet before collection for data analysis. Pearsons testing was used to test hypotheses 1, 2 and 3, and an independent T-test was used to test the validity of hypotheses 4 and 5.
RESULTS
Table 1 (shown below) shows descriptive statistics for the entire group. The table presents the minimum and maximum estimated scores and real scores for the group, the mean of each score (estimated and real) and the standard deviation for each.
Table 1
Minimum
Maximum
Mean
Standard Deviation
Estimate
7
28
5.6
4.1
Score
0
28
6.9
4.0
In this form, the data does not hold much relevance, and so it was necessary to carry out certain statistical procedures to correlate the information. Firstly, a Pearsons test was used to see whether or not there would be a correlation between having done A-level psychology or not and performance on the test. The results of this test can be seen in Figure 1 (below)
Figure 1
r = .319 ; n = 70 ; p < .01
It is first necessary to clarify the different values. R is the Pearsons value (strength of correlation), n is the number of entries, and p is the probability of the results being due to chance.
As can be seen from the figure above, the correlational value between previous psychological knowledge and performance on the test was .319, the number of data entries being 70, and a significance figure of < .01. This is therefore a weak positive relationship, and is statistically significant.
A Pearsons test was also used to determine the correlation between judgments of confidence of performance in the test. Figure 2 (below) shows the processed data-
Figure 2
r = .600 ; n = 70 ; p < .001
This shows that the Pearsons value for this correlation was .600, the number of entries 70, and the statistical significance was .001. It can therefore be gathered that this is a strong relationship, which is statistically significant.
Through another Pearsons test between previous psychological knowledge and judgments of confidence, the following data was produced-
Figure 3
r = .095 ; n = 70 ; p < not significant
As can be seen from Figure 3, the correlational value was .095, and the statistical significance was in no way significant. It can therefore be stated that there is a next to no positive relationship between previous psychological knowledge and judgment of confidence, and is statistically not significant.
Table 2 (shown below) shows processed data for the quasi-experimental hypotheses. It shows the mean and standard deviation of the estimated and real scores for both non-A level and A-level participants.
Table 3
A-level?
Mean
Standard Deviation
Estimate
No
5.0
3.6
Yes
5.9
4.4
Score
No
5.0
3.4
Yes
7.8
4.0
This table shows that for those who had not done psychology at A-level, the estimates of score produced a mean of 15.0 and a standard deviation of 3.6, and the real scores had a mean of 15.0 and a standard deviation of 3.4. For those who had done psychology at A-level, the estimates had a higher mean than those who hadn't done A-level of 15.9, yet had more spread out estimates with a standard deviation of 4.4, showing greater dispersal around the mean. The real scores of participants who had done psychology A-level had a higher mean (than those who hadn't done psychology A-level) of 17.8 and a greater standard deviation of 4.0.
To test the difference in scores between A-level psychology students and non A-level students, a T-test was run, which produced the results shown in figure 4(below).
Figure 4
T(68) = - 2.777 ; p < .01
It can therefore be gathered that in this case, the difference between the means is significant.
A T-test was also used to test the differences in judgments of confidence between A-level psychology students and non A-level psychology students, the results from which can be seen in Figure 5 (shown below)
Figure 5
T(68) = -0.784 ; not significant
From these results, it can be said that the difference between the means is not significant.
DISCUSSION
The results gained from the study must now be explained in a non-statistical way, in order to understand the psychological implications of the findings. The first set of results which should be looked at are the overall statistics for the entire group (see table 1 in results section). From these, it can be gathered that the mean score was higher (16.9) than the mean estimate (15.6), this means that judgments of confidence were wrongly low. In other words, on average, participants scored higher than they thought they would, showing a lack of confidence. The standard deviations for both the estimated scores and the real scores were roughly the same, therefore no comment can be made about the varying accuracy of the mean scores.
Yet does this relate to the entire group or do confidence levels vary whether or not the participant did psychology A-level? By looking at table 2 in the results section, this question is easier to answer. Those who did not do A-level psychology, on average, estimated a score of 15.0, and the mean of their real scores was also 15.0. Standard deviation was low in each case, showing dispersal of results around the average was not greatly spread. Those participants who did psychology A-level had an average estimated score of 15.9, and an averaged real score of 17.8. The standard deviation for both estimated and real scores were higher than those of participants who didn't do A-level. This shows a greater level of dispersal of results around the mean, in other words, scores (estimated and real) of those who did psychology A-level were more uneven than those who did not do A-level. Those who did A-level also were less accurate in their estimates than those who did not do A-level. The mean estimate score was 15.9 (higher than the 15.0 average for those who did not do A-level), yet the mean real score was 17.8. This shows, on first impression, a less accurate judgment of confidence.
Figure 1 (see results) shows statistical analysis of results, determining the relationship between having done A-level psychology or not and performance on the test. It is first necessary to clarify the different values. R is the pearsons value (strength of correlation), n is the number of entries, and p is the probability of the results being due to chance. By looking at the correlational value (r = .319), it can be stated that the relationship is positive, yet weak. Thus meaning, that having done A-level psychology or not will have an impact on the performance on the test, more specifically, yet only to a certain extent as the correlation is weak. From looking at p < .01, it can be said that this correlation is statistically significant, which means that the correlation did not occur to chance.
Figure two shows the statistical correlation between judgments of confidence and performance on the test. There is a strong positive relationship between the two variables ( r = .600 ), meaning that the higher the estimated score, the higher the performance on the test cet. par. It can also be said that this correlation is statistically significant ( p < .001 ). Therefore the strong positive correlation did not occur due to chance and so is a significant relationship.
To investigate whether or not there was a correlation between having done A-level psychology or not, and judgments of confidence, a Pearsons test was again used. This gave the results shown in figure 3 in the results section. It can be gathered that this relationship does not hold any weight; with a measure of r = .095, there is next to no relationship between the two variables and not statistically significant.
In order to test the two remaining hypotheses (quasi-experimental), an independent T-test was carried out for each. Figure 4 (see results section) shows the T-test results from testing the difference in scores between A-level psychology students and non A-level students. This set out to see whether the difference between the means of A-level students and non A-level students was significant, or whether the difference occurred by chance. Indeed it can be stated that the difference between the means is significant.
By looking at figure 5, which shows the T-test for differences in judgments of confidence between A-level psychology students and non- psychology students, it can be ascertained that the difference between the means is not significant.
Are these results directly due to the tests carried out, or did certain design features of the test lead to distorted results? Firstly, McCutcheon's test, although testing a wide variety of topics in psychology, claims to, but does not cover all area of psychology. For example, clinical psychology was not represented on the test. The test also did not allow for varying levels of correctness, participants could ether be right or wrong. This does not allow for varying levels of knowledge in the topic. Therefore a test which could assess varying levels of knowledge would be useful, e.g. participants could be able to get marks for a near to correct answer, in other words able to receive credit for a near correct answer.
The dependant variable for the experiment was whether or not students had done psychology at A-level or not, hence dividing those who had "previous knowledge" and those who didn't. Yet there are many other factors to take into account concerning this; firstly, participants who had done psychology A-level received different teaching methods and different course outlines. This means that those who did do psychology A-level had varying levels of knowledge, yet there was no test for this in the design. Secondly, it was required for all students to do background reading on the subject before arriving on the course. This would mean that those who did not do psychology A-level and so had "no previous knowledge" in fact would have a certain level of knowledge. In the same way, all participants had completed a semester in psychology at the University before carrying out the test, so all would have a certain degree of knowledge. Perhaps selecting participants who had never studied, or read psychology would be a better way to select those who had no previous knowledge.
It would have also been easier and more productive for students to read the questions themselves at their own pace, as there is a possibility students would not have heard the question correctly. On the other hand, by having the questions read out, the time in which students had to answer each question was standardised.
The test was also very long, and there is potential for students sub-consciously switching off to the test as time continued and participants became bored. Feedback from the participants does in fact show that may students did get bored and un-interested by the test after a certain amount of time. This could lead to the answers being selected randomly and hence produce a distortion in the results.
It should also be stated that there were little controls in the design of the experiment. For example, the voice of the reader of the questions and the volume of the reader's voice. This could therefore lead to criticism of the findings. Also, not every participant's first language is English. This could mean that many scored lower than they should have done, due to misunderstanding of the questions, or inability to understand the reader.
So to summarise, it was found via Pearsons test that there was a strong significant correlation between students who did psychology A-level/did not do A-level, and performance on the test. In other words, those who did A-level psychology scored higher on average than those who didn't. This is also backed up by the T-test carried out on the differences in scores between A-level students and non A-level students, which proved to be significant. Prior studies by Nixon (1925) and Tupper and Williams (1986) also showed this to be the case, stating that there were high levels of misconceptions of psychology in those participants who had no psychology knowledge. Furnham (1992) also proved this to be the case in his study carried out on 250 students. Statistical analysis of the correlation between confidence and performance showed there to be a good correlation between these two factors. This conclusion is opposed by Morgan P.J and Cleave-Hogg D (1994), who stated that on statistical analysis of results researching whether or not breadth of clinical experience and student levels of confidence were indicators of competency on standardized simulator performance-based assessments, there was no correlation between level of confidence and clinical grades or written examination marks. Finally, both pearsons testing and T-testing showed that there is no relationship between having done A-level psychology or not and levels of confidence, once again the study carried out by Morgan P.J and Cleave-Hogg D(1994) states the opposite of these findings, stating that "Analysis of data showed good correlation between clinical experience and level of confidence.".
In conclusion, this study has made some interesting findings, some of which are backed up by previous research, some of which are opposed by other studies. Obviously, previous knowledge does lead to greater performance on knowledge tests, yet holds no influence on confidence. The study would therefore say that folk psychology holds less importance in modern day psychology, which is certainly not just common sense.
REFERENCES
Colman, A. (1988). What is Psychology? London: Hutchinson
Furnham, A. (1983). Social Psychology as Common Sense. Bulletin of the British Psychological Society, 36, 105-109.
Furnham, A. (1992). Prospective Psychology Students' Knowledge of Psychology.
Psychological Reports, 70, 375-382.
Garfinkle, H. (1967). Studies in Ethnomedology. Englewood Cliffs, NJ: Prentice Hall.
Greenwood, J.D. (Ed) (1999). The future of Folk Psychology. Cambridge: CUP.
Joynson, J. (1974). Psychology and Common Sense. London: Routledge & Kegan Paul.
Kohler, W. (1947). Gestalt Psychology. New York: Liverlight.
McCutcheon, L.E. (1986). A Test of Misconceptions about Psychology. Psychological
Reports, 68, 647-653.
McKeachie, W (1960) . Changes in Scores on the Northwestern Misconceptions Test in Six
Elementary Psychology Courses. Journal of Educational Psychology, 51, 240-244.
Montgomery, R. (1987). Psychologism, folk psychology and one's own case. Journal for
the Theory of Social Behaviour, 17, 195-218.
Morgan PJ, Cleave-Hogg D (1994). Comparison between medical students' experience,
confidence and competence. ISI Web of Knowledge
Nixon, H. (1925). Popular answers to some psychological questions. American Journal of
Psychology, 27, 91-98.
Schultz, A. (1964). The Stranger: An essay in social psychology. In A. Brodersen (ed.).
Studies in Social Theory. The Hague: Nijhoff.
Stich, S. (1983). From Folk Psychology to Cognitive Science. Cambridge, MA. MIT Press.
Tupper, V., & Williams, R. (1986). Unsubstantiated beliefs among beginning psychology
students: 1925, 1952, 1983. Psychological Reports, 58, 383-388.
Vaughan, E.D. (1977). Misconceptions about Psychology among Introductory Psychology Students. Teaching of Psychology, 4, 138-141.
APPENDIX 1
