Aspects of qualitative research can effect certain results and this can effect the extent to which they are generalisable. For instance, two separate studies of a tribe called Nyiah formed very different impressions of the group. Slater found them to be open and friendly, whilst Gartrell, a lone woman driving a Landrover, found them to be hostile, and perhaps these different results can be attributed to the Nyiah peoples belief that women should play a submissive role in society. To use another example, relating to qualitative study of the mass media, Glasgow University Media Groups study "Bad News" was effected by events out of the groups hands. For example, their often hostile relationship with the newsrooms they were scrutinising effected their research, particularly in the sense that only observation and no interviews were permitted. Furthermore, logistical problems such as the fact that by June 1975, half way into the study it was clear that due to financial considerations and work overload problems the group were not going to be able to continue their work up to the end of the year. This may have had knock on effects to their final results and conclusions, and thus has ramifications as to the generality of their research findings. The nature of qualitative research perhaps makes it more vulnerable to such effects given the flexibilty and lack of structure inherent in the approach. On the whole qualitative researchers do not have a rigid research strategy. For instance, Ditton, writing about his etnographic study of "fiddling" in a bakery, claims his research "was not set up to answer any empirical questions". This approach increases the opportunities for coming across unexpected issues, and problems, both of which can significantly alter the course of study.
Another problem associated with the qualitative approach is that of interpretation. In his book "Quantity and quality in social research" Bryman claimed that "explicating the subjects interpretation of social reality is the mainstay of qualitative research". This implies that a huge diversity of perspective is required in order that the researcher can gain a multiplicity of world views, and there are inherent problems within this, especially if one takes the view that it is impossible to approach a piece of social research in a totally objective manner, devoid of any presuppositions.
This problem is exacerbated when one considers the difficulty in assessing the validity of the interpretations presented in qualitative research. Due to the nature of the approach, evidence presented as proof that a certain interpretation is valid is often fragmented, for instance in the case of brief conversations and clips from unstructured interviews, and this calls into question how representative these interpretations could be said to be. Bryman makes this point when he suggests that it is hard to decide whether a qualitative researchers interpretation of actions and events are congruent with their subjects understandings and world views.
Another problem, often cited in relation to qualitative analysis is the extent to which its findings could be said to be reliable, and this often ties in with what has already been said about the problems of interpretation and generalisation. In comparison to quantitative analysis, qualitative analysis is often seen as the poor relation in this respect. In general quantitative polls have proved to be reliable in the sense that if different polls are taken on the same questions similar results tend to emerge. The quantitative researcher uses the same explicit set of measurement operations on each subject. This contrasts sharply with the qualitative approach which does not use any standardized sampling and measurement procedures. This presents a clear problem in testing the reliability of qualitative data because given the lack of standardization involved in attaining the research findings it would be difficult to test the results of one qualitative study by carrying out a similar one. For instance, Lewis' restudy of a Mexican village seventeen years after Redfields came to very different conclusions. As mentioned previously, this could be attributable factors such as the difference in the researchers style, or the time elapsed between the two studies et cetera. This example highlights the problems inherent in testing the reliability of qualitative research, hence reliability has been seen as a big problem with the qualitative approach.
However, it could be said that the problems of reliability, interpretation and generalisation can be overcome, and the elimination of such problems inevitably strengthens the approach. For example, the idea that the results of qualitative research are not generalisable has been disputed. Robert Weiss claimed that the researchers themselves, "....believe, as they often reveal in the body of their report, that their work is of value because the results are generalisable". Sykes certainly felt this way about his study of prisons when he claimed that, "....prisons appear to form a group of social systems differing in detail but alike in their fundamental processes".
A number of suggestions have been put forward in order to make qualitative research more generalisable. For instance, the study of more than one case. One example of this, to refer back to the work of the Glasgow University Media Group occurred in their study "More Bad News". The principal case study concerned the T.V. news coverage of the British Leyland strike of January 1975. In February 1979 the group took an additional sample of coverage of another Leyland dispute, this time in Birmingham, to compare with the earlier results. In this sense the group were not attempting to produce an exact replica of their previous experiment from 1975. They were studying different forms of basically similar situations, and arguably, this provided a way to test the generalisations developed by the group.
Other suggestions were also put forward to combat the problem of generalisation. For instance, the examination of a number of cases by more than one researcher, enabling the overall investigation to assume the framework of "team research". Again the Glasgow University Media Group can be presented as an example here. The main principle underlying this idea is that team research can give greater breadth to a study, and allow for a variety of different facets of the research subject to be studied, and this is important given the complexity of the media.
The final suggestion involves seeking a case that is typical of a certain "cluster" of characteristics whilst other researchers study comparable cases which belong to other clusters of characteristics. This is a way of approaching a summation of case studies so that generalisations can be extracted as the evidence is accumulated.
Similar solutions have been suggested to combat the problem of intepretation mentioned previously. One such suggestion is that of "respondent validation", whereby the researcher submits a version of his/her findings to the subjects themselves to see how the researchers interpretations tally with the interpretations of those being studied. For example, Ball, who had conducted qualitative research in a secondary school, circulated copies of his research papers to informants, held two seminars at the school in question, and submitted a copy of his thesis to the headmaster. Whilst this method is by no means infallible, for instance, it can invite censorship and defensive actions if the results are incompatible with the respondants self image, it does go some way towards ensuring that the interpretations of those studied are not too far from the researchers view of those interpretations.
There is also a sense in which qualitative analysis, far from being an unreliable approach, is infact a good deal more reliable than other research methods. For example, it could be said that quantitative analysis is too over structured in that it reduces complex issues into yes/no answers. Qualitative analysis on the other hand allows for a great deal of detail to filter through. This is important in the study of the mass media where there is so much dialogue and visual imagery at work. Such complexity requires a more sophisticated research approach. The descriptive detail qualitative research provides is seen as allowing a backdrop upon which events and situations can be viewed within a social context. Features such as the unstructured interview can increase the descriptive detail of a study as well as bringing up important facts. Measor, for example, claimed that, "....inevitably the interviewee will "ramble" and move away from the designated areas in the researchers mind. "Rambling" is nevertheless important and needs some investigation. The interviewee in rambling is moving into areas which most interest him or her.".
There are plenty of examples of qualitative research attaining greater levels of detail, and thus, perhaps a more accurate picture of events, in the work of the Glasgow University Media Group. For example, in "Bad News" linguistic features were cited as examples of T.V. news bias in a way that would perhaps have been overlooked using any other research method. For example, in the section dealing with the placement of items in the news broadcast , on one day the Department of Employments announcement of increased unemployment figures was immediately followed by the phrase, "Meanwhile, the rise in share prices continues....". As the Glasgow researchers put it, "The verbal link implies an association which is denied by the substance of the item.". The inference being made here was that the close proximity between industrial and economic items implied a causal connection.
The work of the Glasgow researchers also paid close attention to the type of language used in T.V. news coverage in a way that would have been overlooked in quantitative analysis. For example, the differences between the words "dispute" and "strike" were noted during an analysis of the T.V. coverage of Harold Wilsons speech in January 1975 about the governments future investment in the car industry. During the speech Wilson used the term "dispute", and this was broadcast in an early evening B.B.C. broadcast. However, on the nine o'clock news the journalist referred to Wilsons speech as a speech about "strikes". The Glasgow University Media Group stated that they saw "strike" as a more emotive term than "dispute" because, whereas "dispute" implies some sort of a debate between two parties, "strike" implies one party acting upon another. This ability to highlight such implicit features of the subject under scrutiny is extremely important in the study of the mass media where the hidden agenda of the programme makers often works at such a subconscious level.
Qualitative analysis has a number of other important strengths. For example, as an approach it enables the researcher to look at the background of why certain conclusions appear. This can include investigating aspects of the researchers background and those (s)he interviews, thus enabling the reader to gain a better sense of why certain questions were asked, why certain answers were given, why the overall conclusions of the study were drawn.
In conclusion, it would appear that there is a sense in which the problems of reliability, generalisation and interpretation can undermine the utility of qualitative analysis in mass media research. For example, the way in which the small numbers used in qualitative research could be said to impair the reliability and generality of the research findings. However, there are also ways in which these problems can be, at least partially, overcome, and combined with various other strengths inherent in the qualitative approach, such as its ability to look at the background of its research conclusions, and its ability to attain detailed, descriptive accounts, it would seem that qualitative research is still of
some use in the study of the mass media.
