The Power Of Benchmarking
Since the inception of the total quality management (TQM) movement, the power of comparative assessment, specifically benchmarking, has been well documented in the private sector. Only recently, with the introduction of high quality national benchmarking studies, have institutions in higher education begun to recognize the value of benchmarking as an assessment methodology to support introspection, strategic planning and continuous improvement initiatives.
While there are many approaches to benchmarking, we will focus our attention here on studies that assess stakeholder perceptions of quality. The principles of stakeholder benchmarking studies are well suited to assist colleges and universities in the development of a comprehensive, long-term assessment strategy. Stakeholder benchmarking is effective because it addresses two aspects essential to the continuous quality improvement process. Benchmarking is effective because it is a powerful tool for (1) identifying the factors most important for improving quality and (2) initiating and sustaining the process of change essential for continuous quality improvement.
Why Benchmarking is a Powerful Continuous Quality Improvement Tool
It Assesses What is Most Important: Successful benchmarking assessment studies evaluate the degree to which an organization is successfully fulfilling its mission from the perspective of key stakeholders. If you believe in the old adage "you get what you measure", it is essential assessment studies focus on mission critical factors. A successful benchmarking study will identify and assess the factors critical to the successful fulfillment of the mission. The contents of the studies are determined by experts who assure the instruments capture the factors essential to the mission of the discipline. Quality benchmarking studies measure mission critical factors.
It Challenges Long Held Beliefs: Benchmarking studies provide a comprehensive internal and comparative evaluation of performance serving to identify strengths and weaknesses. Educators (and other as well) have a tendency to overestimate their strengths and underestimate their weaknesses (evidenced by the 50 or so schools who contend to be in the "top 20" of any rankings). Little progress can be made when performance evaluation is left to a debate based solely on experience and anecdotal evidence.
Benchmarking studies can provide comprehensive, credible results to guide and motivate those in a position to have the greatest impact on quality improvement. When professionals review benchmarking results, inevitably two types of conclusions are reached. First, a good percentage of the results reinforce what professionals already believe, based on their previous education, training and experience in the field. This falls under the category of "we knew this all along." This is to be expected from professionals who have years of experiences. The difference is that now there is credible, comprehensive, comparable evidence to support what was previously opinion or supposition.
Second, professionals are inevitably presented with results that challenge their long-held beliefs. These results are typically questioned because the evidence is contrary to long held assumptions. Once the credibility of the results has been established, professionals face the challenge integrating the new information into their overall view of performance. These results typically have the greatest impact on the improvement process. Credible results provide evidence for professionals to rethink their assumptions about strengths and weaknesses. It requires them to incorporate new insights into a revised perspective of problems and opportunities. Benchmarking results challenge previously held beliefs and challenge professionals to address the issues most critical to improved efficiency and effectiveness.
It Informs Decision-Making: Few organizations have unlimited resources to invest in all aspects of their operation. Each year educators are faced with making resource allocation decisions that will result in the accomplishment of their mission. One of the major barriers to change is the inability of managers to shift resources from historically established budget lines. Stakeholder benchmarking studies can provide information that details the level of performance as well as the importance of factors to stakeholders perception of quality. Identifying low performance factors that have great impact on perceived quality allows managers to focus their attention and deploy their resources in the most efficient and effective manner. It prioritizes for the decision-maker where an investment of resources will have the greatest impact on improving performance in the eyes of key stakeholders.
It is essential to understand (1) areas of strength and weakness and (2) the importance of the factors to overall satisfaction of stakeholders. For example, the factor with the lowest performance score may not be the factor that is most important to constituents’ overall satisfaction. By identifying the factors that are predictors of overall satisfaction in order of importance, educators are able to identify exactly where their resources will have the most positive impact on performance. Simply stated, it is possible for benchmarking studies to identify where managers should invest their resources to have the greatest positive impact on performance. Solid evidence of performance and identifying which factors are important for improving quality provide managers with the evidence they need to shift resources.
It Motivates Staff: Even the most well-intentioned faculty and administrators become frustrated and discouraged when they receive no feedback regarding the impact of their efforts. Benchmarking motivates staff in four ways:
- Reinforce Performance: Evidence of good performance is an opportunity to congratulate and reward staff for a job well done, serving to reinforcement and motivate staff to maintain and improve performance.
- Identify mission critical factors essential for quality improvement: Provide staff with evidence of where their efforts will have the greatest positive impact on improving performance. Benchmarking results identify for staff the areas that are most important for improving overall performance on mission critical factors. Identifying areas where the performance is below that of peers/competitors has the effect of challenging the staff to improve performance by tapping into their natural competitive nature.
- The power comparisons: Comparative results with selected peers remove all doubt that it "can’t be done by anyone else better than we are doing it". With evidence that others perform at a higher level, staff rise to the challenge and commit themselves to improvement.
- The power of continuous assessment: With a continuous benchmarking process, staff members come to know what needs to be improved -- and they know how and when their performance will be assessed again. Knowing performance will be measured and evaluated over time has proven to be a powerful motivator.
The Essentials of Benchmarking
Benchmarking takes many forms and has been associated with many processes. From our experience, the following are essential to successful benchmarking studies. Studies must be:
Credible: Studies must be designed to gather feedback on those aspects of the program that are directly related to the successful fulfillment of the organization’s mission. In other words, measures of stakeholders’ perceptions, resource allocations, or other performance measures that are critical to success. Most importantly, respected professionals from the field must be involved in the development of the content of the study. The statistical reliability and validity should far exceed the minimum standards recognized by academics for statistically sound studies. Studies conducted by external organizations increase credibility.
Comparative: National survey instruments assure comparability of results across the profession. Comparison with a small set of peers selected by participating institutions is essential for providing valid benchmarks for performance. Comparisons that only provide national standards or comparisons with predetermined groups do not provide the benchmarks necessary to most accurately evaluate performance. Studies must either include only schools who see themselves as peers or they must allow each participating school to select the schools to be included in their results analysis to assure comparisons are relevant.
Confidential: There are two levels of confidentiality, one to protect the identity of the participants and the other to assure that results are not used to the disadvantage of any participating institution. Based on the scope and breadth of the study, each benchmarking group must determine the importance of confidentiality. The criteria for establishing the levels of confidentially are based on the legality of sharing the information, the trust among the participants, the sensitivity of the data, and the ultimate use of the data once the results are distributed to the participants. One of the most important issues is whether the results can be publicly released, allowing the participants to indicate their performance is better than their peers/competitors, individually or as a group.
Comprehensive: The data from benchmarking studies should be analyzed to provide summaries that identify areas of strength and weakness in a variety of ways. Descriptive and prescriptive statistical analyses should be provided to identify statistical difference between means and factors that are most critical to overall satisfaction. The results should be designed to provide decision-makers with the information they need to more effectively deploy resources and alter processes directly related to quality improvement.
Continuous: Individual institutional results should be analyzed longitudinally to provide a comprehensive picture of the success of change initiatives and overall progress over time. Longitudinal analysis allows institutions to evaluate changes each year that result in improvements in performance. Longitudinal analysis provides the feedback to continually evaluate initiatives implemented to improve quality. This iterative cycle of initiating changes and evaluating performance results is central to the continuous quality improvement process.
Glenn Detrick and Joe Pica
