Today's healthcare environment dictates that management decisions are clinically sound, operationally efficient, financially responsible and supportive of current health service strategic initiatives and policies.
INTRODUCTION
Today's healthcare environment dictates that management decisions are clinically sound, operationally efficient, financially responsible and supportive of current health service strategic initiatives and policies. Furthermore, it is also recognised that clinical decisions should be based on the best possible evidence of effectiveness (National Health Service Executive, NHSE 1998).
However, there has been increasing recognition that many practices in health care are based on tradition or local circumstances. Some commentators assert that healthcare suffers from a lack of relevant, comparative information based on clinical outcome measures (Teisberg, Porter and Brown, 1994).
The National Health Service in the United Kingdom (UK) is being encouraged to ensure uniform provision of high quality health care (Department of Health, DoH 1997). According to Ellis (2000a), the concept of quality as it relates to health care is a complex, seldom defined and requires constant analysis and clarification. Notwithstanding this, the modern health service has extended the requirements for assured quality of care by introducing the notion of clinical governance (DoH 1998) which suggests that quality can be identified, evaluated and managed (NHSE 1998).
Ellis (1995) put forward the idea that clinical practice benchmarking might be one evaluative technique available to nurses that provides a quality assessment framework and a cycle of continuous quality improvement which supports the development of quality of care. With her colleagues from the University of Central Lancashire and Preston Acute Hospitals NHS Trust, Judith Ellis demonstrated the value of clinical benchmarking in paediatric services by developing a series of benchmarks to inform practice during the 1990s. The success of this work led to the development of benchmarking networks in other areas of nursing practice and health visiting. Culminating in the development of a set of national benchmarks and an accompanying toolkit which was published in 2001 (Essence of Care, DoH 2001).
The benchmarking toolkit 'The Essence of Care: Patient focussed benchmarking for health care practitioners' (DoH 2001) is recognised as the first nurse led quality assurance (QA) process aimed specifically at improving patient outcomes to be developed in over a decade. As such, it has received great attention from the nursing community since its release nearly two years ago and has been compared to both the nursing process and clinical audit as a quality improvement cycle.
However, despite the expression of enthusiasm for the principles of clinical benchmarking from the nursing community across the UK. It is recognised by many commentators (e.g Bland 2001) that the toolkit developed to assist nursing staff to evaluate and compare local practice is unnecessarily complex with terminology that is confusing to the uninitiated. Thus, the development of benchmarking has been thwarted in some health care settings by the lack of clarity in regard to the implementation process. White (2002) suggests that the philosophy of the benchmarking process, which originates from mass manufacturing and serves to weaken the notion of individualised care. In addition to this White (2002) asserts that the national toolkit concentrates on what should be done to improve patient outcomes and does not offer guidance on how to achieve this.
Furthermore, there is also a need to ensure that benchmarking is clearly situated within the organisation's governance framework and is aligned with other quality assurance techniques such as clinical audit, special cause variation analysis and capacity planning (Fitzgerald, 1998). This will ensure that nursing and business managers are informed of potential practice changes (e.g via trust audit and research forums, clinical governance meetings, clinical group reviews etc) as well as the clinical staff involved in the exercise. Involving health care management should be seen as an essential component of the change management process as additional resources are often key to the implementation of practice changes (Ellis, 1998).
My empirical experience of clinical benchmarking activity as the head of clinical effectiveness at a large district general hospital, is that little interface exists between modernisation initiatives, clinical audit programmes and the development and implementation of benchmarking processes. Thus, it is often difficult to identify benchmarking activity and support practice changes when it is not recognised corporately and the concept of benchmarking is not promoted centrally.
As Fitzgerald (1998) points out, benchmarking as a process of information gathering will not provide stringent directives for health care but it is an important activity because but it can be used to keep abreast of current trends and provide best practice justifications for decisions and local policymaking. It is clear from the experiences of our American colleagues that benchmarking can be a useful clinical and managerial tool but little is known about the impact of the introduction of benchmarking in the UK (Cole, 2000).
The problem lies in part, with the fact that benchmarking in nursing practice is a relatively new concept in the UK and structures have not yet been developed to support the implementation of the national benchmarks. A second challenge stems from the fact that unlike our medical colleagues, nurses are not afforded protected time for evaluating practice and as such the practical problem of convening meetings which all the key stakeholders can attend contributes to the lack of progress. Similarly, the lack of protected time for clinical practice in the nursing profession has also been cited by many commentators as a contributory factor for the failings of clinical audit as a change management process (e.g Crombie et al, 1995). Finally, the complexity of the toolkit itself has led to confusion and simplification of the steps required is necessary before the process is likely to be adopted nationally.
It is clear from the discussion so far, that the extent to which benchmarking has been adopted in health care organisations since the launch of the national toolkit in 2001 has not been quantified. Nor has a mechanism for sharing best practice experiences been established in the UK. If benchmarking is to become embedded in nursing practice it is important that a transparent process for implementing benchmarking is established and that the reflections of practitioners with experience of benchmarking are widely published for others to learn from.
In brief, this dissertation sets out to examine the current literature in relation to the implementation of benchmarking processes in order to determine what factors contribute to the successes and failures of the benchmarking exercises in regard to practice changes. The dissertation also offers a proposal for further research to examine the exposure and experiences of nurses in regard to benchmarking from a wide range of health care settings. In an attempt to gauge the extent to which benchmarking has been adopted across the UK and to learn from the experience of practitioners involved in evaluative exercises.
AIMS OF THE DISSERTATION
The main aims of the study are to provide a critical appraisal of the literature and to offer a research proposal to explore nurses exposure and experiences of the introduction of benchmarking.
The literature review will examine the evidence in regard to the development and implementation of benchmarking exercises in nursing practice since the introduction of the concept in the mid-nineties. The review will concentrate on examining the literature in relation to the extent to which the authors describe the challenges and successes of the evaluations and what factors play a part in contributing to the success or failure of the benchmarking exercises.
The literature review will also seek to describe the development of benchmarking as a quality assurance (QA) tool in health care and the accepted model's used in nursing practice today. Secondary aims of the literature review are to include a brief account of the development of QA in healthcare since the 1970s, with particular reference to the evaluation of nursing practice.
The purpose of the research proposal is to outline a multi-method study to explore nurse manager's exposure to and opinions on the introduction of benchmarking and the extent to which benchmarking activity is taking place in their health care organisation. A more detailed account of the research proposal can be found on page 27.
LITERATURE REVIEW
The development of quality assurance in health care and the development of clinical practice benchmarking as a quality assurance tool
The development of health economics as a discipline has contributed greatly to the atmosphere of questioning and scepticism about the effectiveness of medicine in particular and health care in general (Mooney 1992). Sackett et al (1998) suggest that this is because medical treatments were often chosen through deduction on the basis of knowledge of pathophysiology and knowledge of the immediate physiological effects of particular treatments. Systematic study of the outcomes of such treatments did not become a feature of medical management until the development of QA models in the 1970s.
The QA agenda was borne out of growing concern regarding the organisation and functioning of the NHS during the 1950s and 1960s (e.g several scandals in the mental health sector). It was during the 1970s that the assumption made at the creation of the NHS, that the professional competence of the medical profession was the best guarantee of an acceptable level of medical care began to be questioned. In addition to this, economic instability led to a financial crisis in the UK, which also affected public sector funding. This prompted health care providers, professionals and politicians to consider how the increasing level of expenditure on health care might be controlled and whether it would be feasible to develop mechanisms for ensuring that the money was spent appropriately (Morris and Bell 1995).
Edgell and Duke (1991) assert that the 'Thatcher government' was 'ideologically committed to the reduction of public expenditure as a goal in its own right' and 'value for money' was central to policymaking for the public sector. Morris and Bell (1995) describe how the introduction of general managers and 'managerialism' into the NHS in the 1980s was intended as a move toward more coherent planning and cost control by the Conservative government. The idea of 'managerialism' involved the development of performance indicators and the setting of standards as a means of quality measurement.
The introduction of the health care management, the notion that the medical profession needed 'managing' and the need to demonstrate that medical care was both clinically and cost effective inevitably led to conflict between health care managers and the medical profession (Morris and Bell 1995).
However, medicine was not the only profession subjected to managerial scrutiny. Nursing and other health care professions and the functioning of the entire service was subject to review and reforms. The reforms of the 1990 NHS and Community Care Act (DoH 1989) served to increase the pressure to deliver cost effective services by building that pressure into the system of the internal market. Furthermore, the 1990 Act included an explicit requirement for compulsory medical audit (DoH 1989) which provided an incentive for the development of more sophisticated methods of quality measurement.
In the context of QA, audit can be described as a system of monitoring and checking to establish if quality is being delivered against a set of agreed standards. The Department of Health (DoH 1993) describes it as 'the systematic, critical analysis of the quality of medical care, including the procedures used for diagnosis and treatment, the use of resources and the resulting outcome and quality of life for the patient'. The whole process can be represented in an audit cycle. Crombie et al (1995) describes the key steps necessary to complete the audit cycle. The steps include ...
This is a preview of the whole essay
In the context of QA, audit can be described as a system of monitoring and checking to establish if quality is being delivered against a set of agreed standards. The Department of Health (DoH 1993) describes it as 'the systematic, critical analysis of the quality of medical care, including the procedures used for diagnosis and treatment, the use of resources and the resulting outcome and quality of life for the patient'. The whole process can be represented in an audit cycle. Crombie et al (1995) describes the key steps necessary to complete the audit cycle. The steps include the evaluation of clinical practice against agreed standards, the identification of the gap between actual practice and best practice (as defined in the standards), the exploration of the underlying causes of sub-optimal practice or service provision, the formulation of an action plan and the implementation of practice or service changes.
The process of completing the cycle appears relatively straightforward, however despite large amounts of ring-fenced money being invested in clinical audit; its value has always been questioned. The successes and failings of clinical audit as a means of effecting successful changes in practice have been described by many authors, since the 1990 Act detailed medical audit as a required element of medical practice (e.g Warren, 1998 and Johnston et al. 2000).
Clinical audit was again endorsed as an expected part of the clinician's role when the government introduced the new NHS reforms in the white papers 'The New NHS: Modern, Dependable' (DoH 1997) and 'A First Class Service: Quality in the New NHS' (DoH 1998). The new structures set out systems and frameworks for delivering clinical effectiveness and well as cost effectiveness. The key proposals included the development of nationally agreed service frameworks (NSFs) to evaluate services against national indicators of success. The establishment of an institute responsible for the assessment of new technologies and producing guidelines for the NHS (i.e. the National Institute for Clinical Excellence). A monitoring body (the Commission for Health Improvement) to assess the quality of clinical services at local level and intervene if necessary to deal with problems. And the development of a new quality improvement system (i.e. clinical governance) aimed at all NHS health care settings to ensure that systems for guaranteeing clinical quality improvements are put in place. The reforms saw a shift in responsibility within health care organisations. For the first time, Chief Executives would become accountable for the quality of services as well as financial management (Baker, 2000).
The introduction of clinical governance is at the centre of recent NHS reforms. Though some critics have welcomed this concept as a way of addressing the existing wide variation in the standard of care provided between health care settings (e.g Black 1998) others such as Goodman (1998) and Charlton (2001) have dismissed it as just another of the government's 'big ideas' with no substantial meaning'.
What is quality assurance? Goodman (2001) suggests that QA does not have anything to do with assuring quality in the 'real world' sense of excellence. Rather, he describes QA as a technical managerial term for auditing systems and processes as opposed to outcomes. Furthermore, Goodman (2001) criticises the notion that clinical governance should be based around evaluating processes and structures in health care. Goodman (2001) asserts that when 'quality' has been defined in terms of proper systems and processes rather than proper outcomes this has the effect of liberating quality enhancement initiatives from the need to define desirable outcomes and best practice.
It is recognised that QA is beset by epistemological problems concerning the correct interpretation of the evidence. This problem was recognised by Donabedian (1980) who included structure and process as well as outcome in his system of quality assurance, on the basis that outcomes do not always reflect the quality of input of treatment and care given and that health is affected by a multiplicity of factors.
Unlike Goodman (2001), many critics suggest that there is a need in health care to consider best practice not only in attaining the desired clinical outcome, but also within the structures and processes that need to be in place (e.g Ellis 2000). As Ellis (2000a) points out, Florence Nightingale was well aware of the poor outcomes for patients treated in field hospitals during the Crimean War. Understanding patient outcomes is only useful if it is used to improve the structures and processes that are used to deliver health care.
Ellis (2000a) suggests that clinical benchmarking activity allows structured comparison of these practices (e.g system and process review). The benchmarking cycle for continuous quality improvement focuses on patient outcomes. Structures and processes are then identified as factors that would support the attainment of the outcome. Through consideration of all available evidence, the best possible practice within each factor is arrived at and accepted as best practice.
The term benchmarking first entered management literature in 1979 as a way to describe the process that the Xerox Corporation undertook to compare its costs of manufacturing to those of its competition. This practice has led to significant changes in the way companies manufacture goods and services.
Although benchmarking has been undertaken in the NHS in the past, it came to the forefront as a quality improvement tool after the launch of the national strategy for nursing and midwifery (DoH 1999). According to Ellis (2002), the new nursing policy drew attention to the perceived falling standards in 'fundamental and essential aspects of care' and cited benchmarking as a proven tool that could support nurses in raising the standards of care.
In order to facilitate collaboration, the sharing of practice examples and to promote the concept of benchmarking as a valid QA tool in nursing, a national toolkit was developed over a period of months by more than 2000 individuals representing professions, patients and consumers and launched in 2001.
'The Essence of Care: Patient focussed benchmarking for health care practitioners' (DoH 2001) focuses on eight areas that are integral to the quality of care according to patients experiences. These are nutrition, continence, tissue viability, documentation, privacy and dignity, hygiene, self-care and safety. Each element has been broken down into the components of practice that support patient outcomes. A scoring continuum is provided for each factor (E being worst practice and A representing best practice) to enable nurses to score their own practice. The toolkit outlines a seven-stage process identifying the phases to be undertaken during benchmarking. The suggested phases are as follows: agreeing an area of practice, establishing a comparison group, agreeing best practice, scoring and rescoring, comparison and sharing, action and the final phase of dissemination.
The strategy (DoH 1999) describes benchmarking as a process through which best practice is identified and continuous improvement pursued through comparison and sharing. According to Delise and Leasure (2001), benchmarking is a process in which practitioners compare and measure their own practices, philosophies, policies and performance against those of high performing, high quality areas.
Fitzgerald (1998) offers a more comprehensive description of the process and defines benchmarking as 'a management tool used to support and promote continuous quality improvement. It is an iterative process of identifying potential areas for performance improvement and collecting best practice information from similar organisations. Best practice is defined as those practices and processes that support and enhance the outcomes of the organisation and are sound from a clinical, operational and financial perspective' (page 24).
According to Fitzgerald (1998) the value of benchmarking is that the process helps health care staff to focus on areas where performance improvement is possible. Benchmarking establishes a point of reference against which to measure progress, highlighting areas for in depth qualitative and quantitative analysis.
Fitzgerald (1998) warns however, that 'customer satisfaction' in health care must be seen it quite different terms from that it industry. Typically, customer satisfaction in industry equates to producing an outstanding product, from an extensive selection at a reasonable price. Fitzgerald (1998) suggests that the product of health care is fundamentally human change, which is often complicated by economic, social, emotional and physical factors. Fitzgerald (1998) offers clinical practice benchmarking as a process for information gathering which can provide reliable data on which to base decisions and drive up standards of clinical practice and service provision.
The preceding paragraphs offer a brief insight into the origins of benchmarking in industry, the development of benchmarking principles in health care and an overview of the benchmarking process as described in the national benchmarking toolkit (Essence of Care, DoH 2001). It is important to note however, that benchmarking does not set out to establish targets, formulate standards or offer recommendations for change. Nor is its purpose to measure standards of care (e.g clinical audit methodology) or analyse workflow (e.g process mapping).
Instead, the purpose of benchmarking is to provide an assessment of an organisation's performance against a peer organisation (e.g across neighbouring units or wards within a division). To identify best practices and innovations that can augment the organisation's strategy for service delivery and to facilitate collaborative working across boundaries (e.g primary care and secondary care) to identify creative solutions to shared problems.
Creativity and innovation are widely acknowledged in business literature as essential components of both short and long term success. According to Peters (1997), when applied to the core areas of problem solving and decision making they are viewed as pivotal to the life cycle of an organisation. Fraser and Greenhalgh (2001) assert that effective problem solving in a complex environment such as health care involves cognitive processes associated with creative and innovative behaviour. These include the ability to objectively appraise a situation, to choose an appropriate approach and to be able to transfer learning from experiences that contain similarities. The notion of creative problem solving has been around for years and is encapsulated in Osborne's model (1979). The key characteristics of the model can be found in the complementary processes of divergent and convergent thinking. More recently, the concept of problem solving itself has been analysed in relation to the cognitive responses it elicits. For example, is a problem positive or negative? Appreciative inquiry presupposes that problems and challenges and opportunities and asks the question 'what is working?' and how can we apply learning from 'successes' to areas that need development.
It is evident from the literature describing the principles of clinical practice benchmarking that the identification of best practice whether based on scientific evidence or lower level evidence is clearly situated within the field of creative problem solving and is characteristic of appreciative inquiry. Fitzgerald (1998) describes benchmarking as 'a catalyst for change and leadership. It encourages people to act without scripting the moves' (page 25).
Not all commentators believe that the development of QA techniques in nursing is beneficial to the profession or to patient services. Scott (2001) agrees that standards of basic nursing care have been declining over the last ten years and that the national benchmarks provide a framework for the systematic evaluation of nursing practice. Thus enabling nurses to review practice and identify ways of improving the service offered to patients. However, Scott (2001) points out that it is the dichotomous thinking of the Royal College of Nursing and the Department of Health which has created a nursing profession that is focussed on taking up a wider range of clinical tasks. Hence, principles of care such as attending to the nutritional and hygiene needs of the patient have become devalued.
The recently published handbook 'Developing key roles for nurses and midwives: a managers handbook' (Department of Health, 2002) describes how the expanded role of nurses will play a key role in the modernisation of NHS and meeting targets set out in the NHS Plan (2000). This document emphasises the need for the nursing profession to branch out into medical domains such as prescribing; developing nurse led clinical assessment services and performing minor surgery. Scott (2001) urges the nursing profession to examine nurse education in relation to the fundamental areas of nursing and to inspect how it has allowed the government of the day to define nursing practice.
Similarly, commentators such as Carlisle (2001) and Cooke (2001) assert that the decline of basic standards of nursing care can be traced back to the changes made to nurse education. Carlisle (2001) suggests that the traditional learning styles of universities have not been sufficiently adapted so that they are compatible with the needs of education as the emphasis is on the theory of nursing concepts and not the application of knowledge and skills. Instead, Cooke (2001) and Carlisle (2001) suggest that the introduction of a nationally recognised set of clinical benchmarks is an attempt to reduce the essence of nursing to a user manual and undermines the status of the profession.
The critical appraisal, which follows attempts to evaluate the extent to which benchmarking processes are perceived by, nurses to impact on nursing care and patient outcomes. In particular, the critique will seek to draw out examples of successful practice changes, the factors contributing to successful change management and barriers to the change management process.
A critical appraisal of the literature in relation to perceived success factors and barriers to the implementation of clinical practice benchmarking
A search was conducted of the British Nursing Index, CINAHL and MEDLINE databases for the years 1995 to 2002 using a truncated form of the keywords 'benchmarking and nursing' 'evaluation of benchmarking in nursing', 'implementation of Benchmarking in nursing' and 'development of benchmarking in nursing'. Papers which address the empirical views of nurses and theoretical discussions concerning the implementation of benchmarking processes, were retrieved and synthesised for inclusion in the literature review. The indexes of the British Medical Journal and Ovid E Journals portal were hand searched for key articles.
Key points from each article were collected and classified into broad categories using a traditional narrative review approach. The approach adopted was largely qualitative and the aim was to identify common elements in the literature which when integrated, would lead to a greater understanding of the key issues surrounding nurses experiences of developing clinical benchmarking strategies. No attempt was made to evaluate data from the literature identified due to the lack of comparability of the subject areas under investigation as studies were identified from benchmarking exercises in a wide range of health care settings.
A total of forty-seven articles were identified using the search criteria. Of the forty-seven articles identified, three were excluded because they were duplicates, a further twenty were discounted as unsuitable for inclusion in the review because they were international publications (mainly from the United States of America, USA).
Whilst the USA's experience of developing benchmarking techniques contributes the overall body of knowledge in regard to our understanding of the usefulness of benchmarking as a quality measure. The majority of articles identified in American publications dealt with the concept of benchmarking solely in terms of cost containment and cost effectiveness. The ethos behind the development of benchmarking in the UK, it could be argued, is heavily influenced by Judith Ellis' aspiration for the development of a modern health service where the provision of nursing services are evidence-based, patient centred and focussed on the continuous improvement of the structures, processes and outcomes for health care delivery. Cost effectiveness whilst important, is to a certain extent, viewed as a by-product of the delivery of health care within a framework of evidence-based practice.
Four articles identified via the search strategy were unavailable at the time of writing and therefore had to also be discounted from the review. A list of the articles identified for inclusion in the review is presented in appendix two.
Twenty articles were included in the final review. The studies ranged from reflections on the introduction of benchmarking as a concept in nursing to case studies of nurse's experiences of using benchmarking techniques.
The literature review identified three key themes: the importance of nurse's perceptions of the benefits of benchmarking, barriers, which block success and facilitating factors which promote success.
Many of the studies reported that nurse's felt they had benefited from benchmarking exercises through improvements in communication with colleagues, increased satisfaction and knowledge (e.g Ellis 2002, Ellis 1995, Woodward et al 2002). As an example of this, Woodward et al (2002) comprehensively describe their experiences of developing clinical practice guidelines in neuroscience nursing using benchmarking as a process. They emphasise the importance of benchmarking in the development of evidence based guidelines to support nurse decision making and suggest that it is one way of improving and achieving a consistent quality of care. Ellis et al (2000b) cite an example of how a 24 hour specialist mental health practitioner role enhanced the service offered to mental health patients in an acute care setting and that of the registered general nurses who were able to access invaluable support and advice.
Batteson (1999) examined health and safety issues using the benchmarking process. Poor scoring concerning awareness of trust policies for loan workers prompted a community wide promotion of personal safety issues and training. Furthermore, the results were used as a lever to enable equipment to be purchased such as location boards and personal alarms.
Benefits to patient care and service delivery was found in eleven of the twenty articles. Many examples were cited as benefits to patient care such as the improvement of catering services following an examination of the nutrition benchmark (Ellis et al 2000c). Developing clinical practice guidelines (Woodward et al 2002) and producing algorithms for discharge management (Ellis et al 2000b). Stark et al (2002) emphasise the importance of making small practice changes in their case study of describing the introduction of the documentation benchmark into GP practice areas in St Helens. She iterates that the experience of examining practice has stimulated professional discussion about best practice and raised awareness of quality issues and NHS policy.
Of the remaining five articles, three did not describe a specific case study in relation to benchmarking and the remaining four articles were letters debating the provenance of benchmarking.
A small number of articles (i.e. Carlisle 2001, Cooke 2001 and White 2002) criticised the concept benchmarking on the grounds that the quality and suitability of nurse education was to blame for the falling standards in basic care. These critics asserted that without educational forms, standards of care will not improve. In particular, White (2002) suggested that there is tension between the idea of standardising nursing practice and delivering individualised patient care. Furthermore, it was also put forward that the 'essence of care' should be patently obvious to a registered nurse and that inappropriate skill mix and the lack of trained staff are two fundamental reasons why standards are perceived to have fallen over the last 10 years.
Professional threat is a common reaction to the introduction of quality improvement measures (Morris and Bell 1995). Studies exploring nurse's perceptions of the introduction of benchmarking were not identified and therefore it is not possible to discuss the extent to which nurses attribute negative comments to their experiences of benchmarking.
The main barriers to the successful implementation of benchmarking and practice changes include the lack of resources, relationship difficulties and organisational impediments. Ellis et al (2000a) explain that the factors which scored lowest in the benchmarking of discharge planning were those with financial implications. Such as the delay in social services accepting referrals and the subsequent delays to discharge. In addition to this, the lack of mutual understanding between nursing and social services staff was cited as an important factor contributing to the low scores. Ellis (2000) suggests that in some cases multidisciplinary working can also be a barrier to the improvement of the standards of care. They describe how the evaluation of theatre lists which involved nursing, medical and managerial staff resulted in a lower post benchmarking score. Ellis et al (2000a) attribute this to the failure of medical and managerial staff to implement the suggested action plan.
Both Stark et al (2002) and Ellis et al (2000b) highlight the importance of using the scoring system positively to motivate staff rather than adopting a competitive stance. Even more importantly, it was considered that the scoring system should not be used to 'name and shame' clinical areas or individuals or be used by managers to impose sanctions. Stark et al (2002) suggest practitioners participating in benchmarking activities should establish ground rules before embarking on data gathering or sharing.
Many factors were identified in the literature which promote the successful implementation of benchmarking. Ellis (2000) acknowledges the importance of clinical ownership of the process and the data gathering to the success of developing practice benchmarks. She explains that benchmarking activity is not only about auditing practice to ensure that it is achieving the required measurable outcomes. Benchmarking supports open comparison and sharing to allow continuous improvement and development.
The availability of practical support (e.g access to evidence, critical appraisal skills and time) to facilitate the development of the benchmarking process was also considered a factor contributing to the success of the exercise (Ellis 2000).
Stark et al (2002) offer several insights into the factors which contribute to successful implementation of best practice. Firstly, they suggest that access to a trained Benchmarking Co-ordinator contributed to the successful implementation of the documentation benchmark in GP practices across Merseyside. Secondly, they emphasise the importance of identifying 'champions' with a good practical understanding of benchmarking who can motivate other staff. Thirdly, they suggest adapting the Essence of Care (DoH 2001) toolkit to suit the needs of your organisation and to not be afraid to tackle areas outside of the remit of the national benchmarks if this is more applicable to your area of practice.
Finally, Fitzgerald (1998) suggests that the success of change management via benchmarking is based on a few critical steps and the two most important questions are 'where the organisation wants to be and what the organisation wants to be' (page 29).
Only one of the articles identified in the review offered a description of the relationship between the benchmarking process and other evaluative exercises (e.g clinical audit or capacity planning). However, many of the studies acknowledged the fact that the benchmarking groups were multidisciplinary and managers were co-opted as and when required. Furthermore, most of the studies highlighted the process for disseminating the results of the benchmarking exercises and most author's implied that the work was recognised centrally (e.g via senior nurse networks etc). Notwithstanding this, the lack of transparency in regard to the where benchmarking sits within clinical effectiveness is an important issue which needs to be addressed by future researchers.
The majority of examples of benchmarking activity were identified as originating in paediatric services. This is unsurprising given that Judith Ellis pioneered the development of the benchmarking tools used in nursing today. Again, it is important that nurses explore the potential benefits of clinical practice benchmarking, as described in depth by Ellis in numerous case studies published in the last seven years (e.g 1995, 1997, and 2000).
Finally, it is clear from the literature that benchmarking is a practical tool aimed at identifying examples of best practice in nursing both internally (local) and externally (from other health care organisations). Therefore, it is unsurprising that the majority of evidence to support or refute the usefulness of the concept of benchmarking is of a lower classification of research/evidence and rarely seen in the more scholarly nursing journals.
Clearly, there is a need for more robust evaluation of the impact of benchmarking in terms of the perceived benefits to the structures, processes and out comes for patients and health care organisations.
Furthermore, it is nearly two years since the publication of the national benchmarking toolkit yet little literature is available in regard to the experiences of nurses implementing benchmarking processes from any of the major specialities. This could in part be explained by the time taken to get articles into print. However, a more likely explanation is that nurses are working hard to review practice and reflection on the experience of being involved has not been sought by the health care organisation they work for because the nursing profession has not called for its implementation to be monitored.
It is important that the nursing profession recognises that if nurses are to develop clinical skills to support modern service delivery as described in the 'Ten Key Roles (DoH 2001) then they should also be afforded the permission to monitor and evaluate the service that they deliver. The importance of monitoring care should be brought to the forefront now whilst the new roles are being designed so that a culture of evaluation can be developed. Thus, avoiding many of the conflicts that beset the medical profession when clinical audit was introduced in the early 1990s.
The research proposal which follows provides an outline of a research study using survey methods to elicit the opinions of nurses from a wide range of health care settings.
RESEARCH PROPOSAL
Introduction
The government has stressed the important role nurses have to play in improving the quality of health care through their contribution to the achievement of the six quality parameters set out in The Performance Assessment Framework (NHS Executive, 1999). This has been set out in the white paper, Making a Difference: Strengthening the nursing, midwifery and health visiting contribution to healthcare (Department of Health, 1999) and highlights a number of issues which health care organisations need to address to enable nurses to play their role in clinical governance. These issues include providing nurses with adequate access to clinical information, guidelines, research appraisal skills and forums for sharing best practice.
Ward and Radcliffe (2001) suggest that clinical benchmarking is a mechanism, which can be used by nurses to improve the quality of the service, offered to patients. They emphasise that nurses should embrace clinical governance as a means of improving patient care and working conditions. In 2001, the Department of Health published a national benchmarking toolkit aimed at facilitating the evaluation of care from the patient's perspective.
There is a paucity of literature available describing different quality measuring tools (e.g clinical audit) and models of quality assurance. However, despite the national recognition of the importance of clinical benchmarking in nursing few studies have been published which examine the experience of setting up a benchmarking comparison group or the successes and failures associated with this exercise.
The purpose of this research proposal is to outline a research study using the sample survey method to elicit the views of nurse managers in relation to the implementation of benchmarking in their organisation and their opinions on what factors promote or hinder practice changes.
According to Fowler (1995) most surveys are designed to measure objectives states and ask respondents about their perceptions or feelings about themselves or others.
Fetterman (1998) describes the etic perspective as the external, social scientific perspective on reality. Survey research is an arm of quantitative research which seeks to obtain information from populations regarding the prevalence, distribution and interrelations of variables. When surveys use samples of individuals they are often referred to as sample surveys.
According to Polit and Hungler (1999) the greatest advantage to survey research is its flexibility and broadness of scope. It can be applied to many populations, focus on a wide range of topics and the information can be used for many purposes. However, the information obtained can be relatively superficial and therefore survey research is better suited to extensive rather than intensive analysis.
Nurse researchers have used the survey research approach to study a wide range of phenomena (e.g barriers to recruiting nurses to rural areas, mother's attitudes to child rearing etc).
Surveys are useful for eliciting the opinions of large numbers of respondents (if required). Survey research can also be a relatively quick and cost effective method or research. As such the research method selected for this proposal will be based on the quantitative research method of sample surveying.
The research question proposed therefore is:
'What do nurse managers understand about clinical benchmarking and what are their experiences of implementing benchmarking in nursing services'.
Aims of the research proposal
. To undertake a quantitative study using survey techniques to elicit the opinions and experiences of nurse managers in relation to the implementation of the national benchmarks for nursing practice.
2. To inform other health care professionals of the opinions of nurses in regard to their experience of clinical practice benchmarking. In order to contribute to the wider body of knowledge concerning evidence-based practice and benchmarking.
Research Procedure
A credible researcher will be sought to undertake the project. Ideally the researcher should be a health care professional with an understanding of the concepts of clinical practice benchmarking, but not currently employed by any of institutions participating in the study.
The researcher will then be responsible for developing all aspects of the survey. This will include recruiting study sites, selecting samples, assisting in the development of an 'attitude scale', data collection, data analysis and report writing.
According to Polit and Hungler (1999), it is important to ascertain the credibility of the researcher before data collection is commenced, in order to establish that the researcher has the relevant training and experience to undertake the study. Furthermore, Polit and Hungler (1999) assert that it is also important that the researcher reveals any personal or professional information about themselves, in order that the data collection is not affected by personal experience (e.g. if the researcher was employed by a participating Trust, then there may be power issues involved that influence the way the data is collected).
Sampling Strategy
The researcher will identify the number of senior nursing staff employed in the north west region by contacting the human resource (HR) departments at the various healthcare settings. Senior nurses for the purpose of the study have been defined as post holders of nursing and midwifery grade H or above. Staffing details are subject to the same strict confidentiality and data protection requirements as patient details and so the researcher will seek permission from the Director of HR and the trust Caldicott Guardian (usually the Director of Nursing) to send out confidential surveys to selected staff via the HR department.
The researcher will use the anonymous staffing details to select a random sample of 4 senior managers from each of the health care settings willing to participate in the study. The researcher will seek to obtain a sample size of around 150 respondents.
The researcher will write to the selected respondents via the respective HR departments in order to seek their co-operation and participation in the study. Once the confirmation that they wish to participate in the study has been given, the researcher will arrange for the participant to complete the survey when they attend for their next outpatient appointment.
If more than 30% of the sample do not wish to participate in the study, then further respondents will be sought from the participating hospital to bring the samples sizes back up to at least 70% of the original target sample.
Crombie (1996) explains that in order to draw a representative sample (i.e. a final sample where generalisations can be made all the way back to the wider group of interest) a four-step process needs to be followed. The wider group of interest needs to be defined (e.g. all nurse managers), possible study participants (e.g. all nurse managers working in NHS health care settings in the north west region), the sample chosen (e.g. a random sample of 150 nurse managers from the said hospitals) and the actual sample on whom data was obtained (e.g. the number of nurse managers who agreed to participate in the study).
Crombie (1996) warns however, that there are a number of biases in sampling strategies that need to be minimised when undertaking a survey. Firstly, is the issue of non-coverage. This means that members of the target population were never included in the sample drawn. This can be an important flaw, if the non-coverage of a target population is so significant that it does not represent the wider group of interest. An example to illustrate this could be the selection of senior nurses from a specific health care setting (e.g mental health). Clearly, the experiences of senior nurses in a general hospital would be systematically different from those who work in a specialist centre in terms of the structure and possibly the process of care delivery.
Similarly, Fetterman (1998) suggests that using a highly randomised approach to study design without a basic understanding of the people under study may cause the researcher to narrow the focus prematurely. Furthermore, Fetterman (1998) asserts that 'such a misdirected study may yield high reliability but extremely low validity, undermining an entire research study' (page 33).
Secondly, is the issue of non-response. The potential for bias here depends on the number and the nature of non-responders. According to Parahoo (1997), patient surveys tend to have a higher response rate than population surveys. Some surveys by the nature of the target population have a low response rate because the population is so difficult to contact (e.g. the homeless or schizophrenics). Cormack (1996) suggests that one way to improve a poor response rate is to send out personal invitations to participants, which outline the purpose of the study, in question.
Cormack (1996) also states, that whilst there is no real basis for this figure, survey results can be meaningfully interpreted if the response rate is at least 70%.
Crombie (1996) cites another potential problem with surveys and that is the play of chance with which samples are obtained. That is to say, a sample could by chance alone contain more respondents with a negative opinion about a given statement, than respondents with a positive opinion. However, the larger number of respondents with a negative opinion might not reflect the wider group of interest. Crombie (1996) suggests that a 95% confidence interval should be calculated to ascertain the range within which the true value lies.
According to Polit and Hungler (1999) the use of multiple sites allows the study to test the consistency of the data and reduce the possibility of bias at any one particular intervention site. For example, if participants from only one hospital were included in the study, then it is possible that this group of staff might hold opinions or have had experiences that are not typical of the larger population of nurses involved in benchmarking activity.
Furthermore, Polit and Hungler (1999) also suggest that the use of multiple sites may also reduce the possibility of response set biases (e.g. nurses at any one site may have had a particularly positive or negative experience during of benchmarking. This could lead to extreme responses, which again, may not be representative of the general population).
Data Collection
The researcher will draw together an expert panel of nurses involved in benchmarking activity in order to inform the content and style on the questions and attitudinal scales.
This expert panel will advise on the appropriate core themes that should be addressed by the research study and to ensure those key areas of the nurse's experience and understanding of benchmarking explored. The researcher, working with the expert panel will then formulate a list of closed questions to which the participants will be asked to respond.
The researcher will post a survey to each willing participant in the study. The questions will take the format of an attitudinal scale (e.g Likert type scale) to collect the responses to the closed questions asked. Particular attention will be paid to the wording, the length and the sequence of the questions, in an attempt to make the survey easy to complete. The researcher will then code the responses and enter the data into a statistical software package (such as SPSS) in order that the data can be analysed.
The involvement of an expert panel will be useful when deciding which questions should be asked and whether they are considered appropriate for answering the research question effectively. However, as Cormack (1996) points out, the process of item sampling can sometimes introduce errors as selecting or deselecting themes can affect the data and ultimately, the study findings. Thus, great care needs to be taken when deciding which questions should be included or excluded so as to avoid bias.
As Polit and Hungler (1999) point out, the instrument format can influence the characteristics of a data collection tool and the results obtained. For instance open-ended questions will produce different responses and findings than closed questions. It is with these reasons in mind, that combinations of strict alternative and checklist questions have been chosen for this survey. Parahoo (1997) suggests that there are a number of drawbacks to using open questions. One such drawback is the fact that data analysis can be very tricky, because the responses given to the same questions by different participants might not be comparable.
Cormack (1996) explains that Likert scales are used to measure attitude by asking the respondent to assign a level of agreement to a particular statement. This study would seek to elicit responses to statements such as "I have access to protected time in order to participate in comparison group meetings'. The respondent would then indicate on a six point scale how strongly they agree or disagree with this statement.
Likert scales usually have an equal number of positively and negatively phrased statements. This system is adopted so as to avoid leading the respondent to answer in a particular way. For instance, if all the questions were orientated towards a negative response, then it is likely that the respondent would provide a negative answer. A survey consisting of a balance of positively and negatively phrased statements should reveal that the respondent would provide roughly the same answers throughout.
Cormack (1996) also notes that the questions in a survey should be short, precise and use plain English. He suggests that the language used should reflect the lowest possible educational level of the target population so that all respondents will understand the questions. One idea should be expressed per statement and ambiguous terms such as "often" should be avoided when formulating the questions.
Parahoo (1997) does concede however, that there are a number of disadvantages to using a self-reporting questionnaire format. In particular, the sample could be prejudiced by participants allowing friends or colleagues to assist in the completion of the survey or the respondent does not understand the meaning of the question being asked. Or quite simply, the respondent might refuse to participate.
Data Analysis & Transformation of the Findings
A combination of categorical and quantitative data will be collected. The data will then be analysed to produce results by nursing speciality. These results will then be presented as summary statistics in the form of graphs and tables.
The research findings will then be written up as a report and submitted for publication. It is hoped that the responses of nurses in regard to their experience of benchmarking and the successes and challenges encountered during this process will be a useful contribution to the body of knowledge that currently exists concerning the evaluation of clinical practice benchmarking.
Other Considerations
Research studies can be costly, especially in terms of the amount of time required to undertake a project. The researcher will have to spend a considerable amount of time arranging the dispatch and follow up of questionnaires. Input the data from the returned questionnaires, analyse the data and produce a report of the findings. However, the cost of the research can be justified in that it will contribute the evaluation of NHS policy and may in the long term, influence the funding of benchmarking or the development of future quality measurement tools.
There is also the issue of ethical considerations; Cormack (1996) states that research, which involves human subjects, should adequately protect them from harm and that ethical behaviour requires the researcher to seek informed consent from the study participants.
How Should the Research Findings be used to Inform Practice?
The participation of institutions in such a study should promote collaboration and information sharing, where it was not already established. Furthermore, the wider issues of how to embed quality improvement strategies into nursing practice should be addressed at a multidisciplinary level. To include clinical audit, effectiveness and governance personnel, as well as clinical and business managers. Using the opinions of patients as to how a service should be provided and what they think of the existing service is a strategy, which is welcomed by the current government.
The NHS Plan (Department of Health, 2000) outlines the importance of service users and providers opinions to the development of high quality health care services. And how these recommendations will be used to inform the change in structure to the future NHS. Research of this nature clearly fulfils the need to include service users in the decisions made when commissioning services and evaluation of current practice.