Rosenberg points out that anthropologists have thus likened research to language learning. (Rosenberg: 52). Given the fact that subjective worlds are so unique and complex, and are not even perceived with clarity by many individuals, as well as the fact that these subjective worlds are hidden from immediate observation, I would argue that interpretation of these subjective worlds is unavoidable. It follows that the language learning approach, characterised by the anthropological enterprise, is the ideal way to uncover these conceptual worlds with the minimum of distortion.
However, The interpretation of significance of certain statements and actions respondents from a secondary perspective will never be the same as the respondent. It will never be 100% reliable or valid. Interpretative distortion will always occur. As Geertz says:
"To set forth symmetrical crystals of significance, purified of the material complexity in which they were located, and then attribute their existence to autogenous principles of order, universal properties of the human mind, or vast, a priori weltanschauungen, is to pretend a science that does not exist and imagine a reality that cannot be found." (Geertz: 20)
Unless the researcher can locate that specific sentence and can step into the conceptual framework of the specific individuals whom he is researching, data derived from the research process will lack accuracy and will be less useful in predicting future human behaviour with any degree of accuracy.
Specific problems of research involving large populations
This evidently creates problems for the researcher seeking to collect data on large groups by survey. The anthropologist has time to get to know the language, and time to write a largely descriptive report of how his subjects make sense of their world and to get to grips with how this conceptual framework relates to their values and beliefs. The survey researcher, on the other hand, is, by virtue of financial and time constraints, unable to learn the conceptual nuances of the experiential language of his respondents.
Following Rosenberg's idea of locating the very sentence that an individual would use to explain his own actions, beliefs and desires in order to reliably explain his actions would mean taking seriously Elm's suggestion and would involve dealing with individual psychological components or elements or thoughts. This would, however, be a daunting task where populations of millions of people are involved. The researcher is then, forced, I believe, to follow Elm's more serious suggestion of distinguishing between attitudes, beliefs and values, basic operationalisation which would be necessary in order for us to measure and compare attitudes and thus to improve the explanation and prediction of human behaviour.
As Elms says: "in order to study these matters quantitatively, as well as impressionistically - we need to become more specific and discrete." (Elms: 10) He goes on to argue for the case that we should perceive of attitudes as consisting of two fundamental aspects. BELEIFS are cognitive assumptions about the probability that an object exists, posses certain characteristics and relates to other objects in certain ways, and VALUES, which are what someone wants to be true, and will usually involve behavioural tendencies. He then goes on to define an attitude as:
"A feeling about a particular object in terms of its assumed relation to one's values." (Elms: 19)
Elm's theoretical breakdown of an attitude as consisting of conceptually distinct beliefs and values is a useful, and indeed necessary procedure for research purposes. But, as a result of this process, he has obviously come a long way from the confused experiential world of the subject, who does not readily distinguish between such concepts.
In breaking down the subject's experiential orientation to the external world into a useful conceptual framework created by the theorist, is the researcher reporting and explaining what the subject actually experiences as feelings, thoughts, and articulates as values and beliefs?
Explaining Behaviour: The purpose of attitude research
In order to minimise conceptual distortion, a whole host of research methods have been designed to measure people's attitudes, generally oriented around asking them what they think about particular people, issues or objects of attention in general. To a much greater degree than with the observation of behaviour and objective social phenomena such as age and income, the very act of designing research to tap people's attitudes and to try to get to their conceptual world creates problems. Along with the development of these new methodologies have come a whole host of problems of operationalisation, reliability, validity, and analysis, not least of which is the fact that, as verbal behaviour is also an action, we can't ever prove that saying yes is caused by an underlying attitude.
Ultimately, I would argue that the purpose of all attitude research, that is the whole point of trying to discover what people think and feel about particular aspects of their perceived external world is to enable us to better explain and predict human behaviour
That it is an important exercise to try to ascertain what people know and feel about, and how they evaluate particular issues is indicated by the proliferation of opinion polls and market research surveys that are funded both privately and publicly. The utility of such undertakings lies in their predictive value: that governmental departments or private enterprise can adapt or even abandon certain policies and strategies in the light of the fact that hostile opinion may well lead to hostile action if a particular policy is followed in a particular way.
Attitude research essentially became necessary because it was deemed that behavioural observation alone had proved inadequate for explaining patterns in human behaviour. I would thus disagree with Upshaw's conclusion that behaviour is a dimension of an attitudinal state, and would separate behaviour, which can be observed directly, from attitudes, which require careful theoretical and methodological preparation before research is carried out.
Behaviour does not come under the conceptual idea of an attitude, and it cannot by definitional necessity. When I consider attitudes below, I am not talking about behaviour, as attitudes, requiring the development and validation of indirect indicators, start where behaviour ends. Certain research problems that pertain to attitudes simply do not pertain to behaviour, and it follows that there are certain research problems that arise simply because attitudes, by the fact they are by definition, subjective, and thus hidden.
Below I will outline and criticise the various strategies that have been suggested to overcome the problems of interpretation associated with attitude research.
Methodological Problems of Attitude Research by Questionnaire
Research conducted by questionnaire consists of questions consisting of linguistic terms that are derived to measure theoretical constructs. According to Newell, the researcher should "begin with a hypothesis", and "each question must have a direct relevance to one of the variables of the hypothesis". (Newell, in Gilbert: 99). It follows that the variables that Newell is talking about are likely to be derived from theory, and will not located in the specific world of meaning which Geertz talks about. This means that the amount of interpretative distortion arising from survey research will inevitably be much greater than that arising from research of a more anthropological nature, which seeks to uncover the linguistic and conceptual structure of the world of the respondents.
What will be measured, post operationalisation, will never be the actual attitude as experienced by the individual. This, I believe, is what Proctor means when he says that: "an attitude is a hypothetical construct: no one has ever seen or touched one, and its existence or properties must be inferred indirectly." (Proctor, in Gilbert: 117)
Such problems arise partly from the fact that the researcher of contemporary industrial society has to deal with deadlines and will usually have a more specific purpose for conducting the research than the anthropologist. The survey researcher is a researcher with an agenda, and, I will argue below, that the very fact that he has goals other than describing the conceptual world of his respondents, creates problems beyond those associated with the unavoidable interpretation of attitudes. I will also suggest that the bigger and more theoretically rich the concepts and the larger and more diverse the survey population is, the more distortion will arise in the collection of attitudinal data.
Conceptual Distortion in Attitude Research
As I have already said, in order to measure attitudes over a large sample, it will probably be necessary, due to financial and time constraints, to construct a questionnaire. This will involve the delimitation of certain aspects of the respondents' subjective worlds. The beliefs, values and attitudes that a questionnaire is designed to tap will vary according to the object of attention that the attitude is held towards, as an attitude does not exist on its own, and is essentially a reaction to the external world.
For example, in The British Social Attitude Survey, running since 1983, and conducted by the Social and Community Planning Research Institute, designed to ascertain degrees of variance and changes in British attitudes over time, attitudes are defined rather vaguely in different concluding sections of the text as, for example, "levels of concern", or "degrees of pessimism". For research purposes, when dealing with a sample of 3000, taken to be representative of an entire population, an attitude has to be operationalised discretely, as something that is held by an individual towards something as concrete as public transport policy, the environment, or authoritarianism.
The decision of what to research is made by the researcher, or by those funding the research. This creates the problem that, as the conceptual world of the individual is complex and holistic, then any decision to research a particular isolated linguistic construct that is made separately of that individual will not reveal how that construct relates to the respondents' over all conceptual understanding of the world.
Because the subjective world is so complex, as soon as one sets out to measure a belief or value towards an isolated object, distortion will inevitably occur, because it is highly unlikely that each respondent will understand the concept in the same way as it has been operationalised. This could have serious implications for the use of attitudes in explaining and predicting human behaviour.
Such problems of conceptual confusion are demonstrated in Sharon Witherspoon's chapter of this report, designed to measure British attitudes towards the environment, entitled: The Greening of Britain: Romance or Rationality.
This begins with eleven questions designed to measure degrees of "romanticism" (defined as SCEPTICISM about the role of modern science, and a BELIEF that human intervention and economic growth will inevitably harm the environment), and "materialism" (a BELIEF that too much fuss is made about the environment and that scientific and economic imperatives come first). (Witherspoon, in BSA 11: 108-9)
Proctor warns specifically about not asking the respondent to make social scientific judgements. This is because if, to use Proctor's example, the respondent is asked how "politically radical" he is, then he is being asked to define politically radical. It follows that in such a case, the question containing this term can never be a standard measure of radicalism, as radicalism is defined differently by each respondent.
The proposed solution to this is to operationalise the term radical so that, essentially, the respondent is allowed less definitional flexibility of the linguistic terms that make up the question that is designed to measure how radical he is. Ultimately, the terms within the question must mean exactly the same thing to each respondent in order for it to be a 100% valid measurement of his level of radicalism. (Proctor, in Gilbert: 118)
Witherspoon, in line with Proctor, does not ask people how romantic or materialistic they are, and I assume the reason for operationalising these two constructs into eleven questions is to avoid the problems of definitional flexibility that would invalidate them by allowing them to be interpreted too broadly by respondents. I would argue, however, that some of the terms Witherspoon employs in her eleven Likert scales are so broad as to be asking the respondents to be making social scientific judgements in any case.
To take as an example, the question: "We believe too often in science, and not enough in feelings and faith" (Witherspoon, in BSA:110) includes three terms that are extremely abstract. On reading the term "science", the individual may be unconsciously reminded of his boring physics lessons, and may balk at the terms faith and feelings because he simply doesn't know what they mean, and so conclude that he doesn't agree with the statement. In this case, his response would be taken as a measure of his pro-romantic orientation to the world, when really the respondent was indicating that he thought physics lessons were boring. This is an extreme example, but it makes the point that those terms that are used as standard, objective measurements of a theoretical construct by the researcher will be interpreted subjectively by the respondents to the questions containing those terms. Such questions will therefor never be objective, standard measures of an attitude, however attitudes are defined.
To take an even more extreme example, three out of the eleven questions contain the term nature, which has been consistently regarded as one of the most complex words in the English language. The probability that this term would be interpreted in the same way by each of the 1500 respondents is, I think, fairly minimal.
Evidence for the fact that conceptual confusion has arisen lies in Witherpspoon's conclusion that " Far from being the least romantic about the environment, these strong materialists prove to be the most romantic" (Witherspoon, in BSA 11: 111) and that "there is an incoherency and inconsistency in public valuations of the environment, a situation commonly found when attitudes are only superficially held" (Witherspoon, in BSA, 111).
I would be more inclined to conclude that, because the linguistic terms used to construct the questions that measure the two concepts of materialism and radicalism are so abstract, there is no guarantee that they are acting as standard measures, and so are invalid indicators of the two constructs.
"Smith argues that question wording is a significant problem in survey research and suggests that there must be a shared vocabulary between researcher and respondent" (Newell, In Gilbert: p110). With concepts as big as science and faith, however, and with a sample of 3000, chosen to be representative of the entire British population, that there is little hope of ensuring that there is a shared vocabulary between all respondents and the researcher.
How, then, does the researcher ensure that the questions designed are valid and reliable indicators of the attitude to be measured?
The Limitations of Construct Validity
The problems thus far explored arise out of the nature of attitudes as subjective phenomena embedded in a complex and holistic world of experience. It follows that there are billions of perspectives and interpretations of billions of objects of attention.
One of the fundamental sociological questions is whether or not there is a high enough degree of similarity in the way that different individuals interpret and understand the objective physical and social world to make useful comparisons of these conceptions possible. Or, is everybody so unique as to be deserving of absolute attention. Attitude research by survey assumes that there is a high enough degree of shared meaning to make measurement and comparison possible through the employment of hypothetical constructs such as radicalism, romanticism, or degree of negative feeling towards public transport.
There are thus two dimensions involved with attitude research. Firstly, the problem of ascertaining the degree of distortion between the subject's and researcher's interpretation of the meaning of any indicators used, and secondly the degree to which the indicators measure the indicators as hypothetical constructs.
The questions asked about validity and reliability in Proctor's chapter in Gilbert centre around how to test that certain questions are valid and reliable indicators of hypothetical constructs under measurement.
Of this problem of how to ensure validity, Proctor says: "a conceptually convincing solution is offered by the idea of construct validity. This requires that the measure being evaluated should represent a hypothetical concept that is well embedded in theory, so that the nature of the relationships between it and other concepts are well understood. Then, by analysis of the statistical relationships between the various measures, and comparison of these relationships, the appropriateness of the relationships can be assessed." (Proctor, in Gilbert: 127).
This is all very well and good, but the problem with testing for validity of an indicator using degree of correlation with other indicators only proves that the measure is a valid indicator of the indicator as hypothetical construct. In other words, the fact that a strongly agree response to a statement such as "local rail services that do not pay for themselves should be closed down" correlates significantly with a strongly agree response to a second statement such as "local bus services that do not pay for themselves should be shut down" only proves that these two statements are measures of the hypothetical attitudinal construct of "anti-public transport policy."
This statistical analysis suggested by Proctor is no guarantee of the fact that all of the linguistic terms have been understood by each respondent according to the same framework of conceptual logic. It follows that the validity of these terms as standard, objective measures of the attitudinal construct "anti-public transport policy " still rests on the assumption that the train of the respondent's logic comes from and goes to the same linguistic destination, and picks up the same passengers en route, as the train of the researcher's thought.
It follows that because attitude measurement by survey makes no attempt to get to the interpretative framework, from which the respondents interpret their external world, that such research only measures hypothetical constructs. It cannot measure actual thoughts, feelings or individual psychological units, as these are all unique, and hidden.
This particular problem evolves out of the perceived need to measure attitudes in the first place, to compare one persons experience with that of another necessarily involves a third perspective of comparison which distorts both of the subjective worlds to be compared.
(Of course, it may also be the case that these hypothetical constructs do fit into the conceptual worlds of the individual respondents.)
The question is how does the researcher test that his indicators, operationalised in the form of questions, are valid and reliable indicators of the actual attitudes that he hopes to measure? How does he avoid the problems of conceptual confusion and distortion.
Avoiding the problem of conceptual confusion
One way of getting round this problem is to select a more discrete sample size, and to try and focus more closely on the holistic conceptual world of the respondents. This would involve getting to their worlds, and researching what is relevant to them. This would also involve, however, the necessary time and money to undertake research more usually associated with anthropolgy.
Let us assume, for now, by virtue of resource constraints, that the survey questionnaire is the only viable option for researching attitudes. How does one ensure the highest degree of validity and reliability given these far from ideal circumstances?
One way is to follow Smith, and to use tighter definitions that are more likely to mean the same thing to the same respondents. This has been done in the case of chapter two of the eleventh report: Where Next for Transport Policy?
In this chapter, Gordon Stokes and Bridget Taylor commence by constructing five Likert scales, designed to measure degrees of pro and anti-public transport attitudes.
It is much more likely that the key linguistic terms that make up the statement "local bus services that do not pay for themselves should be closed down" (Stokes and Taylor, in BSA 11, 18) are going to be interpreted in similar ways by more respondents than terms such as "faith" or "nature".
This, I would argue is true because the terms bus and train are more likely to be part of the day to day vocabulary of more people than nature and faith, and it is quite clear that a bus, contrasted with a car, signifies public versus private transport policy. It is not clear, however, that nature, contrasted with science signify romanticism and materialism, as the two are higher level abstractions than bus and car (as they do not signify concrete, tangible things), and so are not mutually exclusive.
Even so, it still remains that there is no objective proof that a particular answer to a particular question indicates a particular belief or value. The fact that what is subjective is hidden means that it will always be feasible that a respondent is answering a question positively or negatively for a different reason. This stems from the fact that an answer to a question is, in itself, an action, and the action of a response to a question must be interpreted.
Further Problems of Attitude Research
Another problem is that attitudes are notoriously unstable, and change over time (this is especially true of opinions). One can verify the stability of attitudes by checking for high test re-test correlation, but this correlation will be no measure of the relative influence of response effects. Such effects are pointed out by Sudman and Bradburn who suggest that over-reporting of certain behaviour may occur due to the fact that a respondent may wish to be seen, among other things, as "fulfilling moral and social responsibilities." (Newell, in Gilbert: 107).
If the purpose of attitude research is to explain and predict human behaviour, then research that provides little or no indication of how the object of attention fits into the holistic world view of the respondent will be less valid as a predictor of future human behaviour. Such research offers us no guarantee of under what circumstances they are likely to behave in accordance with their attitudes.
Failure to uncover the entire conceptual scheme of each respondent will reduce the predictive ability of the data collected, as limited data on specific objects may be of no relevance to respondents on a day to day basis. It is very easy, for example, to answer positively to a question that asks how concerned you are about a particular environmental problem, when that problem is the immediate and only object of attention. Is this level of concern greater, however, than the level of concern about missing an episode of Eastenders?, and if it isn't is somebody really concerned about such a problem?
An indication of a high level of concern is no indication that an individual will act on or know the most effective way to act in accordance with his beliefs.
The question that I am left with is as follows: What criteria, that are a posteriori in nature can be used to validate the hypothetical constructs correlation with the actual experiential awareness that is the unique quality of each of the respondents? My answer has been that first of all, one can isolate specific terms that have a high degree of shared meaning to all respondents, and are thus very unlikely to be interpreted in different ways. Secondly, one could test the validity of all or some of the items within the questionnaire with more in-depth research of an anthropological nature. Finally, one could use behaviour as a directly observable, objective measure of the validity of questionnaire items. Given that the first two still leave unsolved the specific problems associated with the subjective nature of attitudes, what hope does observing behaviour offer the researcher in search of verifying that the data he has collected is actually measuring attitudes that really exist within the respondents, as opposed to hypothetical constructs that have no relevance to them?
The Problem of Behaviour as an Attitudinal Indicator
The basic problem, with using behaviour as a validator of the existence of an attitude is, as Rosenberg says, that an action can never testify to the truth of that attitude because there may be a whole host of other reasons why that individual may have performed that action.
Proctor calls the same thing the attitude-behaviour problem, summing it up as follows "In short, a verbal statement is only a behavioural indicator of an attitude and the attitude-behaviour problem is really just one aspect of the more general one of imperfect relationships between different behaviours." (Proctor, in Gilbert: 117)
The problem, however, is that the only way to increase the predictive ability of questionnaires is to make even more conceptual distinctions at the most abstract level of attitudinal definition. An example of this is with the semantic differential, developed by Osgood in the 1950s.
The benefits of such conceptual distinctions are extrapolated in the semantic differential, developed by Osgood in the 1950s. The semantic differential measures the several different kinds of meaning that individuals use when conceptualising objects. A semantic differential usually includes three scales for each major judgmental rating
The crucial point here is that:
"The individuals beliefs about an objects potency, activity, and at other times less important dimensions of meaning, may also be crucial to his overall attitude, determining whether his behaviour toward an object is similar to or very different from the behaviour of other individuals whos evaluative ratings resemble his own". (Elms: 27)
But it still remains, however important such a distinction is for explanatory purposes, that it is already a distortion of the individual's own experiential orientation to his external world. This is demonstrated simply by the fact that what Elms, myself, or any other researcher who has dutifully carried out his operationalisation of the term attitude in order that it can be measured, would call a belief or a value, would not be labled so by all respondents.
It is important, I believe, to distinguish between beliefs and values, as someone can believe that something is true of a particular object, without it having any implications for his behaviour. I would even agree with Osgood, in distinguishing between questions that are designed to tap potency and activity dimensions of belief, and a third category, designed to tap affective aspects. This at least, offers us some explanation of why there may be discrepency between attitude and behaviour on the part of so many people.
But as far as I am concerned, this is not the whole problem. It only becomes a hypo-thetical construct if one operationalises it.
Thus far, I have been talking about the problems associated with attitudes that arises out of their nature. It has been concluded that, unless the researcher is prepared and able to undertake a more anthropological enterprise, and is prepared and willing to act as reporter, he will not be researching attitudes at all if the definition of an attitude is the way someone values an object given its percieved relation to their desires, as in doing so, the researcher must divorce the individuals under scrutiny from the right to fully explain their behaviour in terms of their subjective values and beliefs, and in order to measure and to increase predictability, the researcher must define an attitude more precisely as part belief and part value (or more), and in this process has already gone beyond the experiential world of unique experience.
(I would go further and argue that questionnaires do not measure attitude at all, as an attitude, as it exists in its subjective, experiential form, is unstable, and susceptible to change, to response effects. It follows that a question will bring forth an attitude rather than uncover a consistent and persistent world view that the respondent owns.
Bibliography:
Elms, Alan C: Attitudes (Prepared for Social Psychology Block Ten), 1976. The Open University Press
Geertz, Clifford: The Interpretation of Cultures, 1973. Hutchinson and Co.
Newell, Rosmarie: Questionnaires. In Gilbert, Nigel (ed.), 1993. Researching Social Life: London, Sage.
Proctor, Michael: Attitude Research. In Gilbert, Nigel (ed.), 1993. Researching Social Life: London, Sage.
Stokes, Gordon and Taylor, Bridget: Where Next For Transport Policy In: British Social Attitudes the Eleventh Report, 1994/5. Social and Community Planning Research Institute. Dartmouth Publishing Company
Rosenberg, Alexander: Philosophy of Social Science, 1995. Westview Press
Upshaw, Harry S,
Weber, Max:
Witherspoon, Sharon: The Greening of Britain: Romance or Rationality?. In British Social Attitudes the Eleventh Report, 1994/5. Social and Community Planning Research Institute. Dartmouth Publishing Company
