- prepared as an account of work sponsored by an agency of the United States Government. - The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.. - development provided by the laboratorỵ This report presents the customer survey component of the PNNL CSAP. - The customer survey questionnaire is composed of two major sections: Strategic Value and Project Performancẹ Both sections contain a set of. - questions that can be answered with a 5-point Likert scale responsẹ The strategic value section consists of five questions that are designed to deterinine if a project directly contributes to critical future national needs. - The project Performance section consists of nine questions designed to determine PNNL performance in meeting customer expectations.. - The properties of the statistical model cti be used to. - establish a “gold standard” or performance expectation for the laboratory, =d then to . - The gold standard is defied using laboratory management input. - answers to four questions, in tẽs of the infõation obtained from the customer SUPRY:. - What should the average Strategic Value be for the laboratory project portfoliỏ. - What Strategic Value interval should include most of the projects in the laboratory portfoliỏ. - What should average Project Performance be for projects with a Strategic Value of about 2?. - What should average Project Performance be for projects with a Strategic Value of about 4?. - PNNL is one of the laboratories operated by the DOE ,Ofice of Sciencẹ PNNL is tasked with the. - This paper presents the customer survey component of laboratory customer satisfaction. - Compelling arguments in support of customer satisfaction surveys can be found in [I&xsler, 1995], [Vavra, 1996], [Kessler, 1996] and [Hayes, 1998].. - Section 2 is an overview of the customer s,wvey process. - Section 3 discusses the analysis of cystomer survey data, including a statistical model of the data that C* be used to. - annually assess customer satisfaction. - The mathematical and statistical details are in Appendix A, and a copy of the questionnaire is shown in Appendix B. - PNNL Customer Satisfaction Survey. - it does not provide a quantitative, impartial process for collecting a broad spectrum of feedback: A formal customer satisfaction survey can be analyzed to detect shifts in performancẹ This analysis can initiate laboratory level improvement decisions. - This can be a marked change in day-t~day operations. - The core of the customer survey is a statistical analysis of customer answers provided on questionnaires. - The questionnaire reproduced in Appendix B is composed of two major sections, Strategic Value and Project Performancẹ The Strategic Value section consists of five questions that can be answered with a 5-point Likert scale responsẹ The Project Performance section consists of nine questions that can be answered with a 5-point Likert scale responsẹ The questionnaire also asks the customer to comment on issues that may not be represented in these sections. - For each returned questionnaire, an average of the Likert responses is computed for both major sections. - These averages are termed the Stra-tegic Value and the Project Performance composite scores.. - Prior to the survey, business unit managers review the projects for which they are responsible and give a Strategic Value composite score for each project. - Projects with low Strategic Value may include work at the end of important national initiatives or important short-term work.. - The information from this exercise can be compared visually with customer assessment of Strategic Valuẹ Significant discordance may or may not indicate a need to redirect. - The Strategic Value composite is modeled with a ~ distribution. - The Project Performance composite is a simple linear regression on Strategic Value, embeđed into a @ distribution — Project Performance = A + p Strategic Valuẹ The ~ distribution models the fact that Likert composites are bounded (in the interval 1 to 5) and it also can capture various degrees of skewness in the datạ A formal mathematical description of this model is provided in Appendix Ạ Example plots of the model follow in Section 3.. - The composite scores are used to analyze general laboratory performance and determine the relationship between Strategic Value and Project Performancẹ The statistical model provides the framework to determine if Project Performance is unduly influenced by Strategic Value and to assess whether the data are consistent with laboratory performance expectations. - The properties of the statistical model can be used to establish a gold standard or expectation for laboratory performance, and then assess progress. - The gold standard is defined by input from laboratory management. - Linking model structure directly to understandable descriptions (or attributes) of Strategic Value and Project Performance is the key to defining a gold standard model. - As an example, a pilot survey of customer satisfaction. - should the average Strategic Value be for the laboratory project portfoliỏ — 2. - Strategic Value. - Figure 1: Strategic Value Distribution of Laboratory Portfoliọ. - What Strategic Value interval should include most of the projects in the laboratory “ portfoliỏ —3t05. - What should average Project Performance be for projects with a Strategic Value of about 2. - What should average Project Performance be for projects with a Strategic Value of about 4. - In summary, the customer survey analysis is composed of an easily interpreted composite score model, the development of a gold standard model (a laboratory expectation), an analysis of the agreement between customer and PNNL manager assessments of Strategic. - Section 3 demonstrates that customer satisfaction assessment can be based on a “process.. - Strategic Value = 2 I. - Strategic Value = 3. - Strategic Value = 4. - Figure 2: Project Performance Distribution for Projects with a Given Strategic Valuẹ. - Strategic Value Comparison. - actually disrupt good customer relations and undermine staff moralẹ An objective method for identifying projects that need extra attention is cost effective and healthy for the morale of the laboratorỵ. - Manager Assessments of Strategic Value. - For each returned questionnaire, the composite scores (average of the Likert responses) are computed from the Strategic Value and Project Per@rmance questions. - ..Xn be the Strategic Value composites from the questionnaires (customers) and let x~, zj. - The intent is an analysis that will determine if there is agreement between the paired Strategic Value composites xi and z:. - versus z~ can be used as catalyst for this analysis. - agreement between customer and PNNL manager assessments of Strategic Value will be evident in a scatter plot with points clustered around a near 45 degree line (slope less than one). - A conclusion that leans toward Strategic Value discordance may not be cause for radical changes in marketing and capability development funds. - PNNL managers may have information beyond the scope of the questionnaire that would lead them to continue to support certain business strategies. - First, it can be used to assess whether laboratory management and customers agree on the strategic relevance of funded work at PNNL. - Second, it can be used as evidence to. - 3.2 Agreement Between the Survey Data and the Gold Standard Answers to the four questions discussed in Section 2 define the laboratory gold standard. - A simple statistical method can be constructed to determine if annual survey data are. - If the gold standard model is a tenable representation of laboratory performance, no more than about p~o of the data will be below this linẹ A statistically large number of points below this line indicate disagreement with the gold standard model. - The PNNL assessment procedure involves ascribing adjectives to a count of the number of points below the LCL.. - In Figure 4, the cross lines of the gold standard model represent an average target for Project Per~omnance and Strategic Value, the gray line is Project Performance = A + p Strategic Value, and the gold line is the 5t~ percentile linẹ. - As a result of the CSAP. - program, the culture of the laboratory will become keenly focused on customer satisfaction.. - If an adequate customer database does not exist, a customer survey can be labor intensivẹ The database must be linked to financial information and to internal project contacts such as project managers and relationship managers.. - All projects in the laboratory portfolio are important. - To attain a high response rate, a follow-up process should be implemented as an integral part of the customer satisfaction surveỵ An individual trained in phone interview techniques can call customers who have not responded. - This is done with the approval of the relationship manager for those clients. - Figure 4: Gold Standard Model and Adjectival Rating Regions for a 5t~ percentile Testing Rulẹ. - Without a proper statistical design, the true customer satisfaction baseline cannot be adequately determined and organizational expectations can be poorly defined. - ikfeasuting and Managing Customer Satisfaction. - Improving Your Measurement of Customer Satisfaction. - Ạ2 Establishing a Gold Standard. - po, co is developed with an examplẹ Suppose the laboratory expectation is to have the average Strategic Value of all projects equal to p =4.. - Also, most of the projects should have a Strategic Value in the intervzd 3 to 5, which gives the range R . - If the mean and standard deviation of the Strategic Value model are written in terms of Z, U, Z, the Strategic Value laboratory standard is. - Then specify average Project Performance at these levels. - Let X = VM denote the laboratory Strategic Value composite, and Y = Vc denote the customer Strategic Value compositẹ The Strategic Value agreement model is derived with the generalized ~ distribution. - distribution correctly describes the properties of the Strategic Value composites because of its ability to represent skewness. - Equations 11 and 12 will provide the constraints that will result in a Strategic Value agreement model that can realistically represent composite score datạ Intuitively, these constraints state that E(Y IX = U) =24 is unrealistic because this implies an agreement model with no variability — all the data would need to be tightly packed around the point (U, 24). - Equations 11 and 12 defie agreement in the presence of uncertaintỵ Under these constraints, upper and lower boundary lines can be computed that define a (l-p)%. - percentile of Y can be computed. - Thus, the analysis assumes that returned surveys are a good representation of the true laboratory performance for all projects in the laboratory portfoliọ Also, each project in the sample is independently selected for inclusion in the surveỵ The logic of the analysis is to assume the gold standard model truly represents the performance of the laboratory, and then see if the survey data are in contradiction. - If the gold standard model is the correct representation of laboratory performance, approximately p~o of the random sample will be below the LCL by chance alonẹ For the survey data to contradict the gold standard model, much more than p% of the random sample needs to be below the LCL. - If m of the n’ surveys have composite scores below the LCL, then compute T = P(A4 z m). - The adjectival ratings can be assigned with the rules. - n- <0.001 Marginal/Poor. - n <0.008 Good 0.008<. - n <0.04 Excellent 0.04<77’ Outstanding.. - B PNNL Customer Satisfaction Questionnaire. - ,.@<=@,.<. - -&z..,%-..<. - Strategic Value: (Whyis thisprojectbeingconducted?). - Project Performance RatePactic Northwest NationalLaboratory’s abilityto. - project performance
Xem thử không khả dụng, vui lòng xem tại trang nguồn hoặc xem
Tóm tắt