PERCEIVED FAIRNESS OF AND SATISFACTION WITH EMPLOYEE PERFORMANCE APPRAISAL

Authors Avatar

PERCEIVED FAIRNESS OF AND SATISFACTION WITH EMPLOYEE PERFORMANCE APPRAISAL

A Dissertation

Submitted to the Graduate Faculty of the Louisiana State University and Agricultural and Mechanical College

in partial fulfillment of the requirements for the degree of Doctor of Philosophy

in

The School of Human Resource Education and Workforce Development

by

Marie Burns Walsh

B.S., Louisiana State University, 1981

Master of Engineering, Louisiana State University, 1985

December, 2003


ACKNOWLEDGEMENTS

My sincere thanks are given to each of my committee members for their participation in my academic efforts.  Through the years I have learned from them and appreciated the opportunity they have afforded me in pursuing this degree.  I am particularly thankful to Dr. Michael Burnett for his unfailing patience and guidance during the research and preparation of this dissertation.  He shared his love of knowledge and research with me along with his time to help ensure successful completion of my academic journey.

My family has offered me unconditional love and support through the years that it has taken me to complete this degree.  My children, Catharine, Elizabeth and Matthew, have endured late meals, countless frozen pizzas and learned the fine arts of housekeeping and laundry while I attended night classes and worked on my research.  I have been blessed with my husband, Michael, who kept the home-fires burning, drove carpool and attended endless school activities and sporting events, often without me. I truly cannot express my appreciation for his love and the complete support that he has unselfishly given to me.

My family and friends have also proved time and again their willingness to lend a hand when needed.  I thank my parents for their constant love and initial encouragement of my academic pursuits.  My sister-in-law, Maud Walsh, also deserves special thanks. She never failed to volunteer her time to help us out and didn’t flinch when asked to use her editorial review skills.  Finally, I thank our life-long friends, the Frosts.  A continuation of our family, I could not have hoped for better friends during these many years. My heartfelt gratitude to all of you.  I look forward to beginning the next step in the journey.


TABLE OF CONTENTS

page

ACKNOWLEDGMENTS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . .. . . . . . .         ii LIST OF TABLES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        iv LIST OF FIGURES  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .         ix

ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        x

VITA  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        197


LIST OF TABLES



23        Summary of Items Representing Perceptions of Seeking

Appeals of Employees of Selected State Agencies Employing

Civil Service Workers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        98

24        Factor Loading for Items Representing Perceptions of

Clarifying Performance Expectations of Employees of

Selected State Agencies Employing Civil Service Workers . . . .        100

25        Summary of Perceptions of Clarifying Performance

Expectations of Employees of Selected State Funded

Agencies Employing Civil Service Workers . . . . . . . . . . . . . . . .        101

26        Factor Loading for Items Representing Perceptions of

Providing Feedback of Employees of Selected State

Agencies Employing Civil Service Workers. . . . . . . . . . . . . . . . .        103

27        Summary of Perceptions of Providing Feedback of

Employees of Selected State Funded Agencies Employing

Civil Service Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        104

28        Factor Loading for Items Representing Perceptions of

Explaining Rating Decisions of Employees of Selected State

Agencies Employing Civil Service Workers . . . . . . . . . . . . . . . . .        105

29        Summary of Perceptions of Explaining Rating Decisions

of Employees of Selected State Agencies Employing Civil

Service Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   106

30        Factor Loading for Items Representing Perceptions of Accuracy of Ratings of Employees of Selected State Agencies Employing

Civil Service Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  107

31        Summary of Perceptions of Accuracy of Ratings of Employees

of  Selected State Agencies Employing Civil Service Workers . .  . 108

32        Factor Loading for Items Representing Perceptions of Concern

Over Ratings of Employees of Selected State Agencies

Employing Civil Service Workers . . . . . . . . . . . . . . . . . . . . . . . . . .  110

33        Summary of Perceptions of Concern Over Ratings of Employees

of Selected State Agencies Employing Civil Service Workers . . . . 111 vi


34        Factor Loading for Items Representing Perceptions of Respect in Supervision of Employees of Selected State Agencies

Employing Civil Service Workers . . . . . . . . . . . . . . . . . . . . . . . . . .  112

35        Summary of  Perceptions of Respect in Supervision of Employees

of Selected State Agencies Employing Civil Service Workers . . . .  113

36        Factor Loading for Items Representing Perceptions of Sensitivity in Supervision of Employees of Selected State Agencies

Employing Civil Service Workers .  . . . . . . . . . . . . . . . . . . . . . . . .  114

37        Summary of Perceptions of Sensitivity in Supervision of Employees of Selected State Agencies Employing Civil Service Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  115

38        Summary For Perceptions of Fairness Scales as Applied to

Performance Appraisal by Employees of Selected State

Funded Organizations   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   116

39        Correlation Coefficients For Perceptions of Fairness in

Performance Appraisal and the Independent Variable Age . . . . . .   117

40        t-Test of Means for Respondent’s Perceptions of Fairness of

Performance Appraisal by Gender. . . . . . . . . . . . . . . . . . . . . . . . .        119

41        t-Test of means for Respondent’s Perceptions of Fairness of

Performance Appraisal by Race  . . . . . . . . . . . . . . . . . . . . . . . . . . .   120

42        Correlation Coefficients of Perceptions of Fairness of

Performance Appraisal with Years in the Department  . . . . . . . . . .  121

43        Correlation Coefficient of Perception of Fairness of

Performance Appraisal with Years in the Current Job . . . . . . . . . . . 122

44        Correlation Coefficient of Perception of Fairness of

Performance Appraisal with Educational Level  . . . . . . . . . . . . . . . . 123

45        Analysis of Variance for Overall Means of Respondent's

Perceptions of Providing Feedback by Educational Level. . . . . . . . . 123

46        Group Mean Comparisons of the Perceptions of Fairness of Performance Appraisal As Measured by “Providing Feedback” Scale Responses by Educational Level of Employees of Publicly

Funded Organizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   124 vii


47        Analysis of Variance for Overall Means of Respondent's Perceptions

of Respect In Supervision by Educational Level. . . . . . . . . . . . . . .   125

48        Group Mean Comparisons of the Perceptions of Fairness of Performance Appraisal As Measured by “Respect in Supervision” Scale Responses by Educational Level of Employees of Publicly

Funded Organizations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  125

49        Comparison of the Perceptions of Fairness of the Performance Appraisal System by the Subscales Representing Justice by Job Classifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        126

50        ANOVA for Subscale Means of Respondent’s Perceptions of

Respect in Supervision By Job Classification. . . . . . . . . . . . . . . . .   127

51        Group Mean Comparisons of the Perceptions of Fairness of Performance Appraisal As Measured by “Respect in Supervision” Scale Responses by Job Classification of Employees of Publicly

Funded Organizations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        127

52        ANOVA for Subscale Means of Respondent’s Perceptions of

Providing Feedback by Job Classification . . . . . . . . . . . . . . . . . .        128

53        Group Mean Comparisons of the Perceptions of Fairness of Performance Appraisal As Measured by “Providing in Feedback” Scale Responses by Job Classification of Employees

of  Publicly Funded Organizations . . . . . . . . . . . . . . . . . . . . . . . .        128

54        t-Tests for Means of Respondent’s Perceptions of Fairness of

Performance Appraisal by Supervisor Responsibility. . . . . . . . . .        129

55        Fit Indices for Competing Models of Justice As Applied to

Perceptions of Fairness of Performance Appraisal. . . . . . . . . . . . . .   131

56        Comparison of Nested Models of Alternative Factor Structure

For Justice Perceptions Applied to Performance Appraisal. . . . . . .   136

57        B-Values for Structural, Informational and Interpersonal

Model Representing Factor Structure of Perceptions of Fairness

of Performance Appraisal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   137

viii


LIST OF FIGURES

ix


ABSTRACT

Employee performance appraisal is one of the most commonly used management tools in the United States. Over 90 percent of large organizations including 75 percent of state employment systems require some type of annual performance appraisal (Seldon, Ingraham & Jacobson, 2001).  Performance appraisal is one of the most widely researched areas in industrial/organizational psychology (Murphy & Cleveland, 1993).   However, the traditional research agenda has done little to improve the usefulness of performance appraisal as a managerial tool.

Recent research has moved away from studies of rater accuracy and psychometric measures to themes of employee reactions towards performance appraisal as indicators of system satisfaction and efficacy.  Employee perception of fairness of performance appraisal has been studied as a significant factor in employee acceptance and satisfaction of performance appraisal.

This study investigated employee reactions to fairness of and satisfaction with an existing performance appraisal system utilizing a hypothesized four-factor model (Greenberg, 1993) of organizational justice as the theoretical basis.  The underlying hypothesis was that the conceptualized four-factor model, which differentiated between the constructs of interactional and procedural justice, would best represent the underlying factor structure of the data.

Data were obtained via a survey questionnaire from 440 participants from two organizations that were part of a large public employment system.  Ten multi-item scales

x


representing four factors of organizational justice and performance appraisal fairness and three scales indicating satisfaction were included.

The findings of the study indicated that respondents perceived the performance appraisal system was to be fair as indicated by their agreement with 9 of the 10 scales used to measure reactions to fairness.  The respondents also indicated their relative satisfaction with their most recent performance appraisal rating and with their supervisor.  Less satisfaction (although not dissatisfaction) was indicated with the performance appraisal system overall.

The conceptualized four-factor model was not found to represent the underlying factor structure substantially better than alternative plausible three-factor models.  The best fit three–factor model, however, provided some support for the differentiation between procedural and interactional organizational justice factors, which is a distinction that has been debated in the organizational justice literature.

xi


CHAPTER 1

STATEMENT OF PROBLEM Introduction

Employee performance appraisal, whereby a superior evaluates and judges the work performance of subordinates, is one of the most common management practices utilized in organizations in the United States.  Over 90 percent of large organizations employ some performance appraisal system and over 75 percent of state employment systems require annual performance appraisal (Locker & Teel, 1988; Murphy & Cleveland, 1991; Seldon, Ingraham & Jacobson, 2001).   The widespread use of performance appraisal can be attributed to the belief by many managers and human resource professionals that performance appraisal is a critically needed tool for effective human resource management and performance improvement (Longenecker & Goff, 1992).  The assumption appears to be that an effectively designed, implemented, and administered performance appraisal system can provide the organization, the manager, and the employee with a plethora of benefits (Cascio, 1987; Coens & Jenkins, 2000).

In spite of its widespread use, or perhaps because of it, the practice of formal performance appraisal continues to come under considerable scrutiny and criticism. Performance appraisal is one of the most widely researched areas in industrial/organizational psychology (Murphy & Cleveland, 1991.)  Researchers have developed and practitioners have implemented various changes to the evaluation criteria, rating instruments, and

appraisal procedures in an effort to improve the accuracy and perceived fairness of the

process (Banks & Murphy, 1985).  However, in spite of the attention and resources applied to

1


the practice, dissatisfaction with the process still abounds and systems are often viewed by employees as inaccurate and unfair (Church, 1985).

Evaluation of Performance Appraisal Efficacy

Widespread frustration and dissatisfaction with performance appraisal has challenged researchers and practitioners in both the private and public sectors to evaluate the effectiveness of performance appraisal systems. Evaluation of the success of a performance appraisal system is recommended as part of the system implementation and management process.  However, comprehensive research of the evaluation of performance appraisal system in a field setting is scarce. Murphy and Cleveland (1991) advise that problems with current methods for evaluating performance appraisal systems represent some of the most practical problems facing practitioners. Traditional approaches to evaluating performance appraisal systems have not adequately considered the complex personal, interpersonal, and organizational factors that affect the efficacy of performance appraisal in the organization setting (Mohrman & Lawler, 1983; Murphy & Cleveland, 1991).  A significant amount of performance appraisal research has focused on the rater and evaluation of rating accuracy, which is often studied in an isolated context, generally in a laboratory setting. Extensive research has concentrated on the cognitive processes of the rater and psychometric measurements of performance appraisal.  This research agenda has done little to improve the

usefulness of performance appraisal as a managerial decision-making tool (Banks & Murphy,

1985; Landy & Farr, 1980; Napier & Latham, 1986).

The traditional research themes of rater accuracy, psychometric measures, and technical considerations have recently been expanded to include organizational acceptance, employee attitudes toward the organization, and the performance appraisal system and


employee satisfaction as key indicators of performance appraisal efficacy (Cleveland & Murphy, 1992; Murphy & Cleveland, 1991; Tziner, Murphy & Cleveland, 2001).  Murphy and Cleveland (1995) suggested that employee reaction to appraisals is a class of neglected criteria that should be considered in evaluating the success of a system.  Bernardin and Beatty (1984) also suggested that employee reactions to a performance appraisal system are usually better indicators of the overall viability of a system than the more narrow psychometric indices.   A performance appraisal system can be psychometrically sound in design and construction but still wholly ineffective in practice due to resistance or lack of acceptance on the part of users. Thus, the effectiveness of a system is particularly contingent on the attitudes of the system users, both raters and ratees (Roberts, 1990).

The literature indicates that there are many factors to consider in the evaluation of performance appraisal including employee attitudes towards variables such as perceptions of fairness.  Bretz, Milkovich and Read (1992) indicate that the most important performance appraisal issue faced by organizations is the perceived fairness of the performance review and the performance appraisal system.  Their findings suggested that most employees

perceive their performance appraisal system as neither accurate nor fair.  Skarlicki and Folger

(1997) suggest that the appraisal process can become a source of extreme dissatisfaction when employees believe the system is biased, political, or irrelevant. In general, research indicates that perceptions of fairness arise from consideration of the outcomes received (outcome fairness); the procedures used to determine those outcomes (procedural fairness); and the way in which the decision-making procedures were implemented and explained (interpersonal fairness) (Smither, 1998).  This description of the components of fairness draws heavily on the research and literature in the area of organizational justice.


Fairness in organizations has been studied extensively by researchers in the field of organizational justice.  Organizational justice theory has been applied to many organizational systems and provides a theoretical basis to explore the complexities of performance appraisal more thoroughly.

Organizational Justice Theory

Organizational justice may be defined as the study of fairness at work (Byrne & Cropanzano, 2001).  Organizational justice researchers have reached general agreement that fairness can be divided into two primary types with a third, less clearly defined type often proposed.  The first commonly accepted type of justice is referred to as "distributive" justice. Distributive justice considers the fairness of the outcomes of a particular decision. "Procedural" justice, the second type, is generally defined as the fairness of the processed

that lead to the outcome.  These two areas form the foundation for the majority of research conducted in the field in the last twenty years (Byrne & Cropanzano, 2001).  Research indicates that people will accept a certain amount of unfairness in distribution if they perceive that the process by which the distribution decisions were made is fair.  A third type of justice is often referred to as "interactional" justice.  Bies and Moag (1986) defined interactional justice as the fairness of the interpersonal treatment that one receives at the hands of an authority figure during enactment of organizational processes and distribution of outcomes. The interactional justice concept has been included as an interpersonal aspect of procedural justice and also as a distinct construct along with procedural and distributive justice (Skarlicki & Folger, 1997).

Greenberg (1993b) emphasized the need to more fully consider the social

determinants of fairness that were not recognized by the prevailing emphasis on the structural


aspects of outcome of distributions and procedures.  He proposed a taxonomy of justice classes formed by cross-cutting the two commonly accepted categories of justice, procedural, and distributive, with two focal determinants, social and structural.  The distinction between social and structural determinants is based on the immediate focus of the just action (Greenberg, 1993).  Structural determinants reflect the situation whereby justice is sought by focusing on the environmental context in which the event occurs and ensures fairness by structuring a decision-making context.  The social determinants of justice focus on the treatment of individuals and help ensure fairness by focusing on the interpersonal treatment one receives.  Greenberg's four proposed classes of justice include: systemic (structural- procedural); configural (structural-distributive); informational (social-procedural); and, interpersonal (social-distributive).

The concepts of procedural and distributive justice are relatively well accepted in the study of organizational justice.  However, researchers have not agreed on the integration of the social, interactional, or interpersonal aspects of justice into a commonly accepted model of organizational justice.  Researchers have proposed a variety of models ranging from the two-factor distributive and procedural factor model excluding interactional type justice to

two and three factor models incorporating interactional justice as part of procedural justice or as a stand alone component.  Greenberg's four-factor model is an additional proposition

which may help researchers and practitioners in sorting through the complex issues of performance appraisal.

Organizational Justice Theory and Performance Appraisal

Greenberg (1986a) was one of the first to apply organizational justice theory to performance evaluation.  His basic research question focused on what makes a performance


appraisal appear to be fair.  He investigated if it was what one receives (rating or other outcome) or how it is decided that makes an appraisal seem fair. Greenberg's (1986) work supported earlier research by Landy, Barnes, and Murphy (1978) which showed that employees were more likely to accept an appraisal system and believe that their performance was rated fairly under certain conditions. Landy and Farr (1980) generalized that a fair evaluation is one that contains certain procedural elements regardless of the outcomes of the evaluations themselves.

Folger, Konovsky and Cropanzano (1992) used a "due process" metaphor to extend the application of justice to performance appraisal.  Three essential factors including adequate notice, fair hearing, and judgment based on evidence were used to describe a procedurally fair system.  Subsequent work by Taylor, Tracy, Renard, Harrison, & Carroll

(1995) showed that the due process model is consistent with the procedural justice theoretical model.

Other justice research related to performance appraisal has found relationships between interactional justice and organizational citizenship (Moorman, 1991) and satisfaction and acceptance of performance appraisal (Roberts & Reed, 1996).

Recent research has attempted to clarify the organizational justice literature and integrate the various factors related to performance appraisal to more fully explain employees' perceptions of fairness concerning performance appraisal. Greenberg's (1993) proposed four-factor model as applied to performance appraisal may be a way to further evaluate the complex phenomena of performance appraisal.  Each of the four categories of the taxonomy can be used to address a specific aspect of an organization's performance

appraisal system. There is limited research indicating that the four-factor model can represent


the perceptions of employees regarding the fairness of performance appraisal (Thurston

2001).

Statement of Problem

Performance appraisal in American organizations remains a widespread and common practice despite documented criticism of the process by practitioners and researchers alike. Exhaustive research has been conducted on a range of related topics with limited advances in the understanding and practice of performance appraisal.

Lack of efficient ways to evaluate performance appraisal systems within organizations has discouraged advances in theory related to performance appraisal as an organizational phenomenon.  However, studying individual variables has proved so inadequate at explaining the intricacies of performance appraisal that researchers are attempting more comprehensive evaluation techniques.  Attitudes and perceptions of performance appraisal by participants within the context of the organization in which the process operates are now being conducted.

The literature suggests relative agreement regarding the structural and procedural components of a “well-designed” performance appraisal system.  Many organizations have implemented systems which are based on accepted practices and procedures only to have them rejected by the users.  Clearly there is more to an effective performance appraisal system than a technically sound rating format and well defined policies and procedures.

There is however, no commonly accepted method or efficient approach to evaluating the effectiveness or success of a performance appraisal system based on a set of well-defined variables.  Identifying and organizing the most important variables in performance appraisal has proved to be a challenging task to researchers and practitioners.  Fairness however, is one


variable that has been indicated to be a key component in the ultimate success of performance appraisal systems.  Evaluating appraisal systems using a theoretical foundation drawn from organizational justice offers researchers the opportunity to examine how the fairness of different aspects of performance appraisal may affect the ultimate success of such organizational systems.

Purpose

The purpose of this study was to measure perceptions of fairness of and satisfaction with performance appraisal using Greenberg's hypothesized four-factor of organizational justice as a theoretical framework. Better understanding of the perceptions of the fairness based on the concepts of systemic, configural, informational, and interpersonal justice of performance appraisal and related employee reactions to such systems should provide decision makers with more specific information needed to improve the effectiveness of the system in achieving organizational goals. Multi-item scales based on the research of performance appraisal effectiveness and fairness were utilized to measure individuals' perceptions of the extent to which fair processes and interactions are manifested in an organization's performance appraisal system.

The second goal of the study was to test the theoretical structure of Greenberg’s four- factor model of justice using the scales designed to measure perceptions of fair appraisal practices.  The scales were allocated across Greenberg's (1993) taxonomy of justice perceptions that has been proposed to be a theoretical model that best integrates the various justice factors into a single model (Thurston, 2001).

The study was conducted in a large state government employment system located in the southern United States in the spring of 2003. Two separate agencies were selected for the


study. Both agencies have used the same performance appraisal process since it was introduced as a mandatory requirement by the state civil service management division in

1997.  The agencies have significantly different missions and work processes and employ different classifications of employees; one is predominantly technical and scientific with many professional and clerical positions while the other is a health care provider.  The oversight for the statewide performance appraisal system is maintained by the central civil service management department but each individual organization is responsible for implementing the system.  The only measurement of the performance appraisal system to this point has been the determination of the rate of usage of the system.

Research Objectives

The following research objectives will be explored in guiding this researcher in addressing the research problem:

1.        Describe employees of selected publicly funded organizations that utilize a state civil service employment system on the following selected personal demographic characteristics:

•        Age

•        Gender

•        Ethnic Group

•        Job classification defined by the EEOC Codes

•        Length or tenure in the present position (or with the present organization)

•        Highest level of education completed


•        Whether or not the employee has supervisory responsibility and functions as a rater in the performance appraisal system.

2.        Determine the satisfaction with the performance appraisal system currently being used as perceived by the employees of selected public funded organizations that utilize a state civil service system as measured by the reactions to the system, to the most recent rating and to the rater using the following scales: “Reactions to the PPR”; “Reactions Toward Your Last PPR Performance Rating” and “Reaction Toward Your Supervisor.”

3.        Determine the fairness and justice of the performance appraisal system currently being used, as perceived by the employees of selected public funded organizations that utilize a state civil service employment system, as measured by ten scales representing factors of organizational justice which were based on Greenberg's four- factor taxonomy of justice (Thurston, 2001).

4.        Determine if a relationship exists between the fairness and justice of the performance appraisal system currently being used as perceived by the employees of selected publicly funded organizations that utilize a state civil service employment system

and the following selected personal demographic characteristics:

•        Age

•        Gender

•        Ethnic Group

•        Job classification defined according to the EEO codes

•        Length or tenure in the present position (or with the present organization)


•        Highest level of education completed

5.        Compare the fairness and justice of the performance appraisal system currently being used as perceived by the employees of selected publicly funded organizations that utilize a state civil service system as measured by the scales organizational justice based on Greenberg's (1993) four-factor taxonomy of justice by whether or not the employees report that they have supervisory responsibilities.

The following objectives of the study were established as hypotheses based on the available performance appraisal and organizational justice literature and Greenberg's 1993 four-factor taxonomy of organizational justice.

1.        The ten scales of organizational justice as applied to performance appraisal will form four distinct constructs which conform to Greenberg's (1993) four factor taxonomy of organizational justice with data collected from the employees of selected public funded organizations that utilize a state civil service employment system and a standardized performance appraisal system.

2.        A positive relationship will exist between the scales measuring configural justice (structural-distributive form) and satisfaction with the performance appraisal system currently being used as perceived by employees of selected public funded organizations that utilize a state civil service employment system as measured by the scales “Reactions Toward the Most Recent PPR Performance Rating” and “Reactions to the PPR”.

3.        A positive relationship will exist between the scales measuring interpersonal justice (social-distributive) and satisfaction with the performance appraisal system currently being used as perceived by the employees of selected public funded organizations that


utilize a state civil service employment system as measured by the scale “Reactions

Toward Your Supervisor”.

4.        A positive relationship will exist between the scales measuring informational justice (social-procedural) and satisfaction with the performance appraisal system currently being used as perceived by the employees of selected publicly funded organizations that utilize a state civil service employment system as measured by the scale “Reactions Toward Your Supervisor.”

5.        A positive relationship will exist between the scales measuring systemic justice (structural-procedural form) and satisfaction with the performance appraisal system currently being used as perceived by the employees of selected publicly funded organizations that utilize a state civil service employment system as measured by the scale “Reaction to the PPR”.


CHAPTER 2

REVIEW OF RELATED LITERATURE Introduction

This chapter served as the foundation for the development of this study.  An overview of the extensive historical research related to performance appraisal is presented.

Application of the concepts of organizational justice as a way to understand the dynamics of performance appraisal and to aid in the evaluation of performance appraisal systems is addressed.

Definition and Description of Performance Appraisal

Performance appraisal is a process by which a superior evaluates and judges the work performance of a subordinate.  Performance appraisal systems include the processes and procedures involved in implementing, managing, and communicating the events involved in performance appraisal.   In many cases it is a formal process and is a part of the personnel management policy.

Numerous organizations employ a formal or informal assessment system that measures employee performance and contribution (Carroll & Schneier, 1982).  Coens and Jenkins (2000) suggest that performance appraisal is a mandated process in which, for a specified period of time, all or a group of an employee's work behaviors or traits are individually rated, judged, or described by a rater and the results are kept by the organization. Karol (1996) considered performance appraisal to include a communication event scheduled between a manager and an employee expressly for the purposes of evaluating that employee's past job performance and discussing relevant areas for future job performance.  DeNisi, Cafferty, and Meglino (1984) indicated that performance appraisal is an exercise in social


perception and cognition embedded in an organizational context requiring both formal and implicit judgment.

A variety of components may be included in the performance appraisal process. Landy and Farr (1980) presented a model of performance appraisal that included 13 interacting factors:  position characteristics, organization characteristics, the purpose of the rating, the rating process, scale development, the rating instrument, rater and ratee

characteristics, the observation and storage of performance data, the retrieval and judgment of that performance, analysis of this information, performance description and in the end, personnel action.  According to Mohrman, Resnick-West and Lawler (1989) there are four activities in the performance appraisal cycle in organizations:  1) defining what performance is or should be; 2) measuring and evaluating performance; 3) feeding information about that performance back to the individual; and 4) providing information to other organizational systems that use it.  Latham and Wexley (1981) listed similar requisite components but added a review of legal requirements, development of an appraisal instrument, selection and training of observers, and praise or reward for performance.

Regardless of the definition or the specific components included, performance appraisal in most organizations is formal, structured, and required.  The process is generally defined to include an interview between the rater and the ratee as well as performance documentation required by the formal evaluation system.  One descriptor left out of most definitions is that performance appraisal is often dreaded by participants. Folger and Lewis (1993) suggest that performance appraisals typically engender the same degree of enthusiasm as paying taxes.


Performance Appraisal in American Organizations

The importance of the performance appraisal process or system is underscored by the sheer number of U.S. organizations utilizing the process in one form or another.  The number of businesses conducting formal performance appraisal has steadily increased throughout this century.  Surveys indicate that between 74 to 89% of firms conduct formal performance appraisals (Murphy & Cleveland, 1991).  A 1987 survey of more than 300 organizations belonging to the Personnel and Industrial Relations Association of Southern California examined appraisal trends in private industry and compared the results with those of a similar survey conducted in 1977.  Results showed that 94 percent of organizations had formal appraisal systems, as compared with 89 percent in 1977.  In both years appraisals were most often used to make salary decisions, to improve individual performance, and to provide feedback to employees (Locker & Teel, 1988).

This percentage is similar for public organizations as well.  A recent survey of human resource professionals in state governments indicated that over 75% of the state employment systems required an annual formal appraisal.  Eleven states actually required supervisors to evaluate their staff twice a year and several utilized a process, which includes a series of planned meetings (Roberts, 1995).  Only Rhode Island reported no required performance appraisal system (Seldon, Ingraham, & Jacobson, 2001).

Another study by England and Pearle (1987) of non-managerial performance appraisal systems in the municipal public sectors found that 86 percent of 142 municipal governments appraised their employee's performance on an annual basis.

The results of a major survey distributed in 1998 to members of the International

Personnel Management Association (IPMA) and American Society for Public Administration


suggested that the current and future importance of performance appraisal in the public sector will not diminish. The intent of the survey, conducted in 1998, was to gauge the respondent's perceptions on the relative importance of various personnel techniques, activities and values. Respondents predicted that the widespread use of performance appraisal will continue, ranking it first in importance among human resource management issues at the time of the survey and in future years (Hays and Kearny, 2001).

Dissatisfaction with Performance Appraisal

In spite of the current ubiquitous use of performance appraisal systems and its perceived importance in the future there is considerable contention over its efficacy and usefulness.  Surveys through the years have indicated relative lack of satisfaction towards the effectiveness of performance appraisal systems in both private and public organizations. Bricker (1992) reported survey results indicating that just 20 percent of American companies were very satisfied with their performance review process.  A 1990 Industry Week survey of readers indicated that only 18 percent responding that their reviews were very effective.  This was down from 20 percent in 1987.  Thirty-one percent of the respondents found reviews to be not very effective or a waste of time (Verespej, 1990).  A Wyatt Company survey of 900 companies found that only ten percent of companies indicated satisfaction with their employee evaluation programs (Small Business Report, 1993).  Thirty percent were dissatisfied and 60 percent were not convinced one way or another.  A 1997 nationwide survey of human resource professionals by the Society for Human Resource Management found that only five percent of the respondents were very satisfied with their organization’s performance evaluation system and that 42 percent were dissatisfied to some extent (Barrier,

1998).


It should be noted that most of these figures were obtained from surveys often completed by human resource professionals and other organizational managers and do not reflect any rigorous evaluation of performance appraisal processes or systems.

Overview of Past Research and Literature

The amount of research regarding performance appraisal is vast.  However, the limitation of much of this research to the advancement of the understanding or practice of performance appraisal is generally acknowledged (Latham & Lee, 1986; Murphy & Cleveland, 1991).  Prior to the early 1980's the majority of theoretical and empirical studies focused on improving the psychometric characteristics of the rating instrument in an effort to reduce the subjectivity inherent in performance ratings (Feldman, 1981).  Due in part to the emphasis on psychometric aspects, the development of a “better” rating scale format that was valid and reliable received a great deal of attention (Woehr & Miller, 1997).  Research focusing on rating scale format and development peaked in the 1960’s and 1970’s with the development of several new formats including the Behavioral Observation Scale (BOS), the Behaviorally Anchored Rating Scale (BARS) and the Mixed Standard Scale.  Other popular and related research topics included training raters to reduce rating errors and improve observational skills and developing performance appraisal practices.  Research examining the efficacy of the different rating scales format generally indicated that ratings were not affected by changes in the rating scale format (Woehr & Miller, 1997).  According to Arvey and Murphy’s (1998) review of the research, there were literally hundreds of studies between

1950 and 1980 on the different types of rating scales; of rating versus ranking; and ways of achieving ratings that were objective measures of performance.


Landy and Farr (1980) published a highly critical and influential review evaluation of the performance appraisal research.  In this review they called for a moratorium on rating format research and attempted to redirect research to other areas such as understanding the rater and the process in an organizational context.  Landy and Farr (1980) characterized the abundance of studies in the following categories:  “roles” or characteristics of the rater and rate; the “vehicle” or rating format and form; the context of the rating including its use; and, and the rating process which dealt with data analysis and rater training.

The influence of Landy and Farr (1980) and Feldman (1981) resulted in a change of focus away from the rating scale format and rater training to understanding the rater as a decision maker who processes information and social cues. Research in the 1980’s and early

1990’s focused on raters and the accuracy of ratings and judgments and the application of knowledge about the judgment process in the development of performance appraisal systems.

According to Ilgen, Barnes-Farrell and McKellin (1993) the “rater process perspective” includes three critical sets of operations 1) acquisition of information about those to be evaluated; 2) organization and storage of this information in memory; and 3) retrieval and integration of the information in a fashion that leads to the recording of an evaluation of the person being appraised.  Researchers borrowed heavily from basic psychological research in cognitive psychology and social cognition to address the three process domains described above and to develop theories of the performance appraisal process (Denisi, 1984; Feldman, 1981, Ilgen & Feldman, 1983).

Other research included ratee and rater characteristics such as race, gender and likeability.  Rater attributes including race, cognitive style and knowledge of the job to be rated were examined.  Rating scale accuracy continued to be studied and the characteristics


of the setting in which appraisal occurs such as the purpose of appraisal, rater training and other factors were investigated (Ilgen, Barnes-Farrell and McKellin, 1993).

Research on performance rating accuracy and the development of accuracy criteria was common in the 1980’s.  Research focused on common psychometric biases, called rating errors, such as leniency, central tendency and halo, with the assumption that these implied a lack of accuracy (Ilgen, Barnes-Farrell, & McKellin, 1993).  It was assumed that decreasing the biases increased accuracy. Researchers argued that this assumption was not necessarily true in that bias-free ratings were not necessarily more accurate (Hulin, 1982; Murphy & Balzer, 1989; Roch, 1997).  Research on accuracy shifted from rater errors to discrepancy between ratings and some standard of performance.

Research of the performance appraisal process during the 1980’s contributed a number of key ideas to the literature including a heightened awareness of the importance of observation in the appraisal process and how knowledge obtained by raters is utilized.  The research of the 1980’s also helped to clarify or correct some assumptions about performance appraisal such as the belief that rating errors as commonly defined were evidence for rating errors when in fact the research indicated that there may not be resulting inaccuracies (Murphy & Balzer, 1989; Smither & Reilly, 1987).  Another contribution related to the use of performance appraisal ratings.  The context in which the ratings were obtained and the

beliefs about the use of such rating were found to influence the results.  Researchers further argued that ratings should only be used for those purposes understood by the raters at the time of the rating (Murphy, Balzer, Kellem & Armstrong, 1984, Zedeck & Cascio, 1982).

Ilgen, Barnes-Farrell and McKellin (1993) acknowledged these contributions but stated that the overall impact to the improvement of performance appraisal practice had been


limited.  Like Landy and Farr in 1980, they called for a redirection of research efforts away from demonstrations of cognitive effects towards the investigation the content of cognitive variables, the identification of factors that influence these variables and the design of appraisal systems that incorporate cognitive principles.

More recent research into performance appraisal has emphasized process and structural characteristics that influence the attitudes and affective reactions of system participants in addition to psychometric characteristics.  Murphy and Cleveland (1991; Murphy & Cleveland, 1995) suggest that researchers should consider the rating context before attempting to analyze or evaluate the effectiveness of ratings or rating systems. Research has included measures of employee attitudes toward performance appraisal and system acceptance and rater and ratee satisfaction in the appraisal process (Roberts, 1990). Bernardin and Beatty (1984) suggested that relative measures of the attitudinal kind may ultimately prove to be better measure and predictors of rating validity than such traditional psychometric variables as leniency, halo, and discriminability.  A performance appraisal system can be psychometrically sound in design and construction but still wholly ineffective

in practice due to resistance or lack of acceptance on the part of users. Thus, the effectiveness of a system is particularly contingent on the attitudes of the system users, both raters and ratees (Roberts, 1990).

Bretz, Mikovich, and Read (1992) generalized that research in the late 1980's and early 1990's was heavily weighted toward cognitive process issues.  Ratee and rater personal characteristics and rating errors and accuracy were also researched. The source of appraisal, appraisal feedback mechanisms, rater training, and performance appraisal format were found to be studied frequently.


Other empirical research has focused on the performance appraisal process and the factors that influence the communication and behaviors exhibited by the raters and ratees during evaluation.  Most of these studies are in experimental settings and not in the field (Karol, 1996).

While theoretical research on performance appraisal continues to evolve, practical literature has focused primarily on improving the performance appraisal process, making the review interview more positive, improving the contribution of the employee to the process, emphasizing goal-setting and applying procedural improvements (Karol, 1996).  Professional journals are filled with articles discussing performance appraisal practices in various organizations under varying conditions.  Case studies and "how to" articles are common.

Arvey and Murphy (1998) indicate that the literature indicates a substantial gap between research and practice in performance appraisal.  According to these researchers the gap between research and practice was apparent in the 1989’s when many studies were conducted in the laboratory and focused on discrete variables of cognitive processing in appraisal and evaluation.

Much of the past research has focused on the individual as related to the act of performance appraisal as opposed to performance appraisal as a system within the larger context of an organization.  More recent research has investigated performance appraisal in a more comprehensive and organizational context.

Approaches to Evaluating Performance Appraisal

Extensive systematic research has not been conducted on the evaluation of the success or efficacy of new or existing performance appraisal systems in an organizational context. Evaluation of the success of a performance appraisal system is recommended as part


of the system implementation and management process.  However, comprehensive research of the evaluation of performance appraisal system in a field setting is scarce. This may be due in part to the complex nature of the systems involved and in selecting proper evaluation criteria.  Murphy and Cleveland (1991) advise that the effectiveness of all human resource systems including performance appraisal need to be evaluated.  They indicate that problems with currently available methods for evaluating performance appraisal systems represent some of the most pressing problems facing practitioners.  Bernardin, Hagan, Kane and Villanova (1998) also suggest that the practice of evaluating performance is inadequate.

Researchers have identified components that suggest a greater likelihood successful performance appraisal system than if these same components were absent.  Mohrman, Resnick-West and Lawler (1989) state that the following key items are part of an appraisal system: Appraisal tools and methods; the degree of fit between other features of the organization and the appraisal system; the system design; the proper introduction of the system; and, training of individual system users. The authors state the performance appraisal process must be designed to match the organization's goals and the type of work that is performed.  They believe that one of the most critical factors in effective performance appraisal is clearly defining the purpose of the appraisal system. Possibilities include monetary compensation, career planning, documentation of staffing changes, work load evaluation, counseling and development and training.

In their description of a complete appraisal system, Mohrman, Resnick-West and Lawler (1989) include the following components:  1) two performance appraisal cycles that deal with immediate feedback and long-term career issues; 2) a decision about who defines performance; 3) how performance will be measured; 4) who will measure performance; and


5) what method will be used to gather performance information; and 6) effective feedback that is timely correctly and delivered by the appropriate person.  Appraisals should be timed so that they coincide with job characteristics and avoid peak periods of activity.  The performance appraisal system needs support from top management to generate the requisite commitment from middle managers.  An appeal process for employees to question or challenge their evaluation results lends credibility to the appraisal system.

Summary of the scholarship (Bernardin & Beatty, 1984; Landy & Farr, 1983; Latham & Wexley, 1981; Lawler, Mohrman, & Resnick, 1984; Murphy & Cleveland, 1995) points towards the following five areas as measures of an efficacious performance appraisal system:

•        Determines pay; explains and communicates pay decisions.

•        Provides the subordinate with development information and support.

•        Fosters mutual task definition and planning of future work goals.

•        Documents and recognizes subordinate's performance

•        Allows the subordinate to provide feedback about feelings, supervision and definition of work.

Other variables that may influence performance system effectiveness include the type of performance standards employed (Bernardin & Beatty, 1984; Landy & Farr, 1983; Latham

& Wexley, 1981; Roberts, 1990), the frequency of evaluation (Bernardin & Beatty, 1984); the presence of written administrative procedures; and existence of an appeals process (Cascio & Bernardin, 1981; Greenberg & Tyler, 1987).

Murphy and Cleveland (1991) state that when the following criteria are met, performance appraisals are most likely to be perceived by employees as accurate and fair:


1)        Appraisals are conducted frequently;

2)        There is a formal system of appraisal;

3)        Supervisors have a high degree of job knowledge;

4)        Ratees have an opportunity to appeal ratings;

5)        Performance dimensions are seen to be highly relevant;

6)        Action plans are formed for dealing with present weaknesses; and

7)        The organizational climate is cooperative rather than competitive.

Martin and Bartol (1998) discuss the need to monitor a performance appraisal system to keep it responsive to the needs of the organization.  The major actions required to maintain a performance appraisal system include three major categories: controlling the system; monitoring the system; and furnishing feedback to those who use the system. Control of the system includes the more technical aspects of the system such as rating techniques, rating periods, rater training, and development of performance standards.  Monitoring the system can include a review of the quality of performance standards; evaluation of the actual

conduct of the appraisal process and interview; and, analysis of the intended, perceived and actual use of the system.  Other factors in monitoring the system include review of the actual quality of ratings to check for rater biases, inconsistencies, rating inflation and investigation for any adverse impact as a result of the system. The third primary area to monitor is that of the amount and quality of feedback generated as part of the performance appraisal process.

Murphy and Cleveland (1991) maintain that the psychometric indices and rater error measures most often used to evaluate ratings are not adequate criteria for evaluation of performance appraisal system.  As an alternative they suggest: 1) developing information on employees from a variety of sources and maintaining adequate performance documentation


for review; 2) developing methods for measuring the accuracy of ratings 3) determine the aspects of accuracy that are most relevant to the various uses of performance appraisal; 4) determining the conditions under which so-called rater errors are beneficial versus harmful;

5) developing practical methods for establishing indifference curves among qualitatively different outcomes of appraisal; 6) developing methods for monitoring changes in the value of an organization's members that will necessitate changes in the appraisal system;  7) determining whether reaction criteria are important and 8)  determining whether utility estimates provide useful and credible information.

In the book Performance Appraisal on the Line, DeVries, Morrison, Schullman and Gerlach (1984) make the case for evaluating performance appraisal systems based on a ratio of cost to potential outcomes.  Costs include that of system development, system introduction and system maintenance.  Major outcomes include meeting the intended goals of the system and achieving organizational acceptance.  One of the most difficult aspects of assessing (or creating) a performance appraisal system is to identify a finite set of appropriate goals for the system.  They also state the need for informed participants at all levels throughout the organization to know why and how they are to do performance appraisal.

Mohrman and Lawler (1983) suggest that researchers should concentrate on how performance appraisal systems are perceived by organizational members to improve performance appraisal accuracy.  Further, Mohrman and Lawler (1983) suggest that organizations examine the uses of performance appraisal information to determine if the uses and functions are conducive to accurate performance appraisal.

Giles and Mossholder (1990) argue that while the context in which appraisal occurs has been designated as a source of considerable influence in the appraisal process, relatively


little research has been conducted on the environmental issues.  The researchers attempted to extend the development of measures that assess contextual aspects of performance appraisal; to investigate relationships between system contextual variables and employee reactions to performance appraisal; and, to assess the extent to which system contextual variables were related to employee satisfaction.  The study confirmed that commonly used reaction scales of fairness, satisfaction, perceived utility and perceived accuracy did indeed represent appraisal reactions.

Murphy and Cleveland (1995) referred to employee reaction to appraisals as one class of neglected criteria that might be considered in evaluating performance appraisal systems. Bernardin and Beatty (1984) suggested that employee reactions to performance appraisal systems are usually better indicators of the overall viability of a system than the more narrow psychometric indices such as leniency and halo.

According to Keeping and Levy (2000) employee reactions toward performance appraisal may be considered important for a number of reasons.  First, reactions are of great interest to practitioners.  Second, while reactions have been theoretically linked to determinants of performance appraisal success and acceptance they have been overlooked in the research. These issues are both within the context of the gap between research and practice that has been noted in the performance appraisal literature by a number of researchers including Banks and Murphy (1985); Bretz, Mikovich and Read (1992); Ilgen, Barnes-Farrell and McKellin (1993); and Smither (1998).

Researchers have suggested that reaction to performance appraisal is critical to the acceptance and use of a performance appraisal system (Bernardin & Beatty, 1984; Cardy & Dobbins, 1994; Murphy & Cleveland, 1995).  Reactions may even contribute to the validity


of a system (Ostroff, 1993).  Cardy and Dobbins (1994) suggest that “with dissatisfaction and feelings of unfairness in process and inequity in evaluations, any performance appraisal system will be doomed to failure” (p. 54).  Murphy and Cleveland (1995) stated that

“reaction criteria are almost always relevant, and an unfavorable reaction may doom the carefully constructed appraisal system” (p. 314).

Tziner, Prince and Murphy (1997) measured political considerations in performance appraisal to determine the extent to which distortions in ratings were present.  Their study investigated evidence that rating inaccuracy has more to do with deliberate volitional distortion of ratings than lack of training or ability.  Deliberate distortion of ratings includes raters’ conscious efforts to produce ratings that will achieve personal goals such as avoiding negative consequences; avoiding confrontations or bad feelings with employees; or portraying the image of a caring boss.

Tziner and Murphy (1999) studied the attitudes of managers towards performance appraisal and their organizations.  Raters who showed low levels of confidence with the system were more likely to rate employees unusually high and to fail to discriminate well among ratees.  Raters who showed higher levels of attitudinal commitment or who perceived more risks associated with distorting ratings tended to give lower ratings and to discriminate more between raters and/or dimensions.

Keeping and Levy (2000) examined the measurement of performance appraisal reactions.  They investigated how well commonly used reaction scales, representative of those used in the field, measured the substantial constructs of satisfaction.  They found that these scales did a “favorable” job of measuring appraisal reactions. In addition, they found that the data also fit a higher order appraisal reactions model.  Among the reactions


investigated were satisfaction (with the system and session), fairness (procedural and distributive justice) perceived utility and perceived accuracy.

Tziner, Murphy and Cleveland (2001) reported that attitudes and beliefs toward the organization and about the appraisal system affect how ratings are done and how feedback is handled.  These attitudes and beliefs have an influence on the accuracy and usefulness of ratings.  Their finding showed that beliefs about the performance appraisal system and rater orientation toward the system explained tendencies to give higher versus lower ratings and to discriminate between ratees and rating dimensions.

Thomas and Bretz (1994) conclude that performance appraisal continues to be a vexing human resource challenge that the academic research world has not adequately addressed.  The focus of academic research on appraisal accuracy, rating errors, or an understanding of the cognitive processes used in the appraisal process are not considered by practicing managers to be major organizational concerns.  Thomas and Bretz (1994) called for a transition from laboratory studies into the organizational world but realized the lack of access to organizational settings continues to hamper research.  According to Bretz, Mikovich and Read (1992) research is only beginning to address how context affects employees, perception of appraisal, reactions to appraisal, outcomes and how appraisal purposes (administrative vs. developmental) moderate these relationships. The limited

research considering organizational context has focused on system design and characteristics, system management, and other important performance appraisal issues including fairness and justice issues.


Fairness in Performance Appraisal

A significant direction of research regarding performance appraisal efficacy and approaches for evaluation has concentrated on employee satisfaction and perceptions of the process.  This direction has lead researchers and practitioners to take a more comprehensive view of performance appraisal system efficacy and evaluation of systems which include these factors. One common theme of recent research is that attitudes of the system's users toward the process determine to a large degree the ultimate effectiveness of a performance appraisal system (Roberts, 1990).

Employee perceptions of fairness of performance appraisal have been shown to be linked to satisfaction with the system.  Fairness of performance appraisal has been studied by a number of researchers over time.  In their review of performance appraisal research Bretz, Mikovich and Read (1992) indicated that the most important performance appraisal issue faced by organizations is the perceived fairness of the performance review and the performance appraisal system.  Their findings suggested that most employees perceive their performance appraisal system as neither accurate nor fair.  Skarlicki and Folger (1997) suggest that the appraisal process can become a source of extreme dissatisfaction when employees believe the system is biased, political or irrelevant.  A major problem for organizational leaders is that the performance appraisal process and the performance evaluation system are often perceived as both inaccurate and unfair (Latham & Wexley,

Join now!

1981).

Landy, Barnes, and Murphy (1978) studied employee perceptions of the fairness and accuracy of a performance appraisal system.  The researchers found that frequency of evaluation, identification of goals to eliminate weaknesses, and supervisory knowledge of a


subordinate’s level of performance and job duties were significantly related to perceptions of fairness and accuracy of performance appraisal. Their results confirmed traditionally held perceptions that performance appraisal should be done as frequently as possible, that the supervisor should work with the subordinate to agree on responsibilities; and, that the supervisor should devote sufficient time to observe and evaluate and ...

This is a preview of the whole essay