Performance Analysis for Process Improvement

Originally appeared in ASQ’s 52nd Annual
Quality Congress Proceedings, May 1998

Michael V. Petrovich
MVP Programs

SUMMARY

Process performance analysis, introduced here, uses performance measures such as P_p, P_pk, and P_pm with analysis to assess performance, sources of variation, process potential, and improvement opportunity. This analysis may be conducted where control may not have been achieved and where a large number of process streams exist.

KEY WORDS

performance measures, P_pk, P_pm

INTRODUCTION

The quality goals of an organization include the elimination of nonconformance and the minimization of variation around appropriate targets. The quality practitioner needs to know how well these goals are being realized. Specifically, one must ask, “How well are my processes performing and what must I do to improve?”

For many years process assessment has employed the use of process capability measures. These include C_p, C_pk, and more recently, C_pm. For a review of these see Rodriguez (1992). The use of these measures requires demonstration of statistical control followed by a distribution assessment compared against the targets and specifications. Difficulties arise when statistical control cannot be assumed and also where a large number of process streams may be present. Here are a few examples.

A plastic lid manufacturer has 40 molding machines, 54 tools per molder. Tools are replaced every few weeks, and tool-to-tool differences are found. In addition, slight differences are found as batches of raw materials are changed. Other sources of variation include maintenance cycles, startup periods, and any possible adjustments made by the operating personnel.
A metal bottlecap company has eight presses, each containing 22 dies. Slight differences exist die-to-die and press-to-press. In addition, the process exhibits tool wear and slight fluctuations are observed with lot-to-lot changes in steel.
An aluminum can manufacturer has two lines in one of its plants. Each line contains 16 stations. Stationto- station differences are found. This process also undergoes tool wear and perfect through-time stability (or control) is not observed in many of the stations.

Traditional capability assessments are inadequate to handle these situations. If control cannot be demonstrated, traditional assessments of process capability cannot be properly made. In addition, the presence of a large number of process streams would yield a combination of distributions that would present a nightmare to mathematically model. In the meantime, quality practitioners still need a means of assessing their processes and the effectiveness of their improvement strategies.

In recent years process performance measures have gained popularity. These measures, introduced by Herman (1989), are analogous to their counterpart capability measures. A “P” representing “performance” replaces the “C” found in “capability” indices. The most popular performance measures are P_p, P_pk, and P_pm. These measures may be used to answer questions concerning how well the process is performing and may be employed before a process is documented to be in a state of statistical control.

As introduced here, process performance analysis utilizes performance measures and graphical techniques to display performance, potential performance, and to determine the sources of quality losses. process performance analysis provides a means to

Assess processes before statistical control is achieved
Determine sources of process loss
Make comparisons among characteristics, products, plants, and suppliers
Determine control and improvement priorities
Assess the results of process improvement efforts

SAMPLING

To study processes, samples must be taken and measurements made for critical characteristics. The approach suggested here is to measure total outgoing variation as sent to the customer. Total outgoing variation results from variation inherent to the process as well as variation from lot-to-lot, tool-to-tool, line-to-line, shift-to-shift, setup-tosetup, day-to-day, and throughout maintenance cycles. Sampling will generally be done at end-of-line. To provide diagnostic capability, stratified sampling by lines, tools, or other process streams may also be desired.

To assess the performance of a process, and to reflect the total outgoing variation, long periods of data collection with large data sets are preferred. A minimum of one month, and more preferably three months, worth of data is desired. While the analysis may be done with a few hundred data values, 1000 is a desired minimum; 10,000 or more values may be used to provide better analysis. If 100 values were obtained each day, 9000 values would be accumulated during a calendar quarter.

Of course, an appropriate measurement system analysis should be conducted prior to data collection. This analysis should assess both the stability of the measurement system as well as the variance contributed by measurement error. The methodology for conducting this type of analysis may be found in numerous texts. See, for example, Chrysler, Ford, and General Motors (1995).

PROCESS PERFORMANCE MEASURES

Process performance measures provide an assessment of processes in which process control may not have been achieved. These measures take into account all observed variation from both common and special causes, whereas traditional process capability measures consider only the inherent, common cause process variation.

Performance measures assess the historical performance of a process. Any prediction from them would be useful only to the extent that the future is expected to resemble the past. To capture total variability in a process, performance measures should be assessed from data collected over long periods of time.

When a process is not in a state of control the process distribution is changing through time. Hence, attempts to determine a specific form of a process distribution would be inappropriate. For this reason, performance measures should not consider distributional shape.

Several measures of process performance may be calculated. In this discussion, P_pm, P_pk, and P_p will be shown. Another measure, P_{p(process
stream)}, will be used to examine the performance within a process stream. Finally, a potential measure, C_p, will be estimated. C_p may be used to assess the potential of the process. Making comparisons among these measures will highlight process losses and improvement opportunities.

Taguchi (1986) defined a loss function which quantifies the losses due to manufacturing variation. As product characteristics depart from their design target, this loss function states that the loss will be proportional to the target deviation squared. Chan, et al. (1988) created a capability measure, C_pm, which is inversely related to this loss.

The first performance measure discussed here, P_pm, is derived from C_pm and is inversely related to the losses in a manufacturing process. The higher the P_pm value, the better the process performance. When a target, upper, and lower specification are given, P_pm may be calculated by

where

USL is the Upper Specification Limit
LSL is the Lower Specification Limit
X is the observed value
T is the Target
s² is the sample variance from all combined data
X-bar is the sample mean from all combined data,
and n is the total number of observations

For a single specification and target the following may be used

where SL is the Specification Limit.

The use of P_pm assumes valid specifications and targets are given and representative samples are generated during the period of interest. When appropriately employed, P_pm may be used as a primary measure to assess the performance of a process. Process performance will improve, as measured by increases in P_pm, as the process is brought into better control; systematic variation such as differences in tooling, setup, operators, machines, and materials is reduced; and the process is brought on target. P_pk is recommended when no target is given.

P_pk may be calculated as follows.

P_p is another performance measure which assesses variation but not location. P_p is calculated as follows.

The difference between P_pm and P_p is the term which includes the deviation from target. If the observed process average was on target, P_pm and P_p would be identical. Calculation of P_p and making comparisons to P_pm will yield an understanding of the loss from being off target. An additional diagnostic measure, % Off-Target, may also be calculated. This measure represents the percent of the specification range the mean falls from target.

As the process average departs from target, P_pm is reduced. Figure 1 graphically displays the relationship between P_pm and % Off-Target. The resulting P_pm is shown for four levels of P_p (at infinity, 3, 2, 1.5, 1) as the offset from target varies. The highest line represents a process with a standard deviation of zero (P_p at infinity). If a process is running with a mean 10 percent off target, the largest P_pm that may be achieved is 1.67. At 30 percent off target, the largest P_pm that may be achieved is 0.56.

If the process under study contains multiple lines, tools, or stations (process streams), the losses from differences between these process streams should be understood. P_{p(process stream)} is a measure of the process performance within process streams. If no process stream differences exist, and the process is on target, P_pm would be equal to P_{p(process stream)}. P_{p(process
stream)} may be calculated by

where s_{within stream} is the square root of the average of the variances within each process stream calculated by

where J is the number of process streams.

A useful diagnostic measure is the percent process stream difference. This measure is simply the range of the process stream means divided by the specification range expressed as a percentage.

Another important diagnostic assessment concerns the losses which arise from a lack of through-time stability or control. The swithin stream includes any sources of through-time variability. The question to answer is, “If perfect through-time stability were achieved, how much variation would be observed?” To address this question, the minimum potential process variation, s_potential, may be estimated. Several approaches may be used for this estimation. Two common approaches are

Short-term capability studies
Estimating variance within process streams using average or median dispersion statistics

Short-term capability studies may be conducted to determine the inherent potential variation of the process. This may be done by generating one or more studies of relatively small samples of consecutive items and then estimating the potential variability. Preferably, multiple studies should be conducted at different points in time.

Several approaches to estimate the potential variation may be employed when using data sampled over an extended period of time and where control may not be observed. One common approach is to remove individual outof- control points and to estimate process variation from those points remaining. Unfortunately, this methodology is not feasible with large data sets.

Another approach to estimate the potential variation is to use averages or medians of dispersion statistics such as the range, moving range, or standard deviation. These average or median statistics may be used to generate an estimated s by dividing by appropriate constants. [See Wheeler (1995) for tables of constants and methods of calculation.] Median dispersion statistics are more robust to significant shifts in the process than average dispersion statistics. Although there is more sampling error associated with median dispersion statistics, this is not an issue with a large number of samples.

The potential variation includes measurement error. If desired, measurement error variance may be subtracted from this estimate to assess the true product variation (σ² _product = σ²_potential – σ²_measurement ). This may be a concern if large measurement error variance exists (Herman 1989).

The rate of sampling may influence the estimated potential variation. When using statistics which assess item-to-item variation, shorter time intervals between sampled items would generate lower estimates of potential variance. Sampling consecutive items would result in minimum estimates.

When multiple process streams are present, estimate the potential variation for each process stream. In these cases, use the average of the estimated standard deviations for estimating process potential, assuming potential variances are equal among process streams. If large differences exist in variation, separate analysis for each process stream may be desired.

Using the estimated potential variation, s_potential, a measure of the potential capability of the process, C_p(potential), may be calculated as follows.

This potential capability is the potential performance measure (P_pm) that would be obtained if the process were brought into control, process stream variation was eliminated, and the process was brought on target.

OBSERVED NONCONFORMING RATES

Since distributional analysis is not done with performance measures, and through-time stability may not be achieved, the use of process performance measures should be accompanied with examining the actual observed nonconforming proportion. Using the total number of items examined and the number outside the specification limits the nonconforming proportion may be calculated. Translating the total nonconforming to parts per million (ppm) may be useful.

PROCESS PERFORMANCE ANALYSIS

The following relationships will hold when generating performance measures.

A stacked bar chart may be used to display these relationships. Using the performance measure P_pm as a base, stacked bar charts may be created by using sections which represent the incremental differences between the performance measures. The height of each section displays the various performance measures. These bar charts show the observed performance and the improvement potential if the process is brought on target, the process stream variation is removed, and the process is brought into control. An example is shown in Figure 2. These bar charts represent data from two plants manufacturing aluminum cans.

The sections of the bars in the chart show the potential performance improvement that would occur in the P_pm value when various improvements are made.

The lowest section of the bars show the observed performance measure, P_pm.
The top of the second section of the bars show the performance had the process average been on target.
The top of the third section of the bars show the performance of the process had the means for each process stream been on target.
The top of the highest bar sections show the potential of the process if through-time variation is removed (control achieved) and each process stream mean is on target.

While the areas of the top three bar sections can be thought of as observed losses, these areas cannot be compared. The improvement potential seen on the stacked bar chart occurs when the process is brought on target, then process stream differences are removed, and finally through-time stability is achieved. Note that changes in higher bars represent a smaller change in process variation than a change in a lower bar.

The total variance about target, τ², can be decomposed into the various components that can be compared to assess improvement priority.

These components may be approximated as follows and displayed in a pie, bar, or Pareto chart.

σ²_potential may be estimated as described previously and may also be decomposed into σ²_product and σ²_measurement, as described earlier.

σ²_off-target may be simply estimated as follows.

The variance from process stream differences may be estimated many ways. If the data have not been stratified, studies may be conducted to determine the variance contribution from process stream differences. These studies may include all streams of interest or a random selection of streams. Methods such as analysis of variance (ANOVA) may be used to estimate this component. Remember though, that the process is not assumed to be stable and these assessments should be considered approximate. If the data have been stratified by all process streams of interest, the process stream variance may be simply estimated as follows.

where, again, s² is the sample variance from all combined data. The variance from through-time process changes, σ²_time, may be estimated as follows.

EXAMPLE

Aluminum lids are produced on a machine with 22 stations. Each day, lids from each station are sampled and the lid height is measured. Data are collected over a three-month period. The specifications for this characteristic are 98 ± 5. Table 1 provides an analysis of these data.

Table 1 displays the descriptive measures, the variance components, and the performance calculations. Figure 3 displays a histogram of all the data. Figure 4 displays box and whisker plots for each of the 22 stations with the target and specifications. Figure 5 displays a process performance analysis stacked bar chart. Figure 6 shows an extension of the stacked bar chart. Figure 6 shows the original stacked bar chart plus the effects on performance for various improvements. The second bar height shows the performance if brought on target, the third shows the performance if the process stream differences were removed, the fourth shows both the target and stream effects removed, and the fifth shows the effect if the through-time instability was removed.

Figure 7 displays a pie chart of the variance components. Figure 8 shows the combined data through time. Figures 9 and 10 show control charts for two stations. The same data from these stations are displayed in Figures 11 and 12 on a run chart with the specifications drawn.

From these charts, it can be seen that the biggest opportunity comes from getting the individual stations on target. While improvement in control will minimize variation, the through-time sources of variability do not represent the largest opportunity. Measurement error analysis is not shown, but from the process potential assessment, measurement error may be considered negligible.

These results are typical of many observed processes. Improvements in targeting and reduction of stream-tostream differences often represent an important opportunity. While improvement in through-time stability or control does not represent a large improvement opportunity in this case, it may be critical in others.

CONCLUSION

The use of process performance measures generated with data collected over extended periods, combined with performance analysis which decomposes sources of variation, provides quality practitioners the ability to answer questions about how their processes are performing and what opportunities exist for improvement. The graphical display of these measures allows one to view the improvement potential. This display may be used to compare multiple processes and characteristics and assist in improvement prioritization. Performance measures such as the P_pm may be used to validate improvement efforts. If through-time stability is improved, tooling or station differences are minimized, and the process is brought on target, P_pm will display an improvement.

REFERENCES

Chrysler Corporation, Ford Motor Company, General Motors Corporation. 1995. Measurement Systems Analysis. Southfield, MI: Automotive Industry Action Group (AIAG).

Herman, John T. 1989. Capability Index—Enough for Process Industries? 1989—ASQC Quality Congress Transactions. 92–104.

Rodriguez, Robert N. 1992. Recent Developments in Process Capability Analysis. Journal of Quality Technology, Vol. 24, No. 4, 176–187.

Chan, Lai K., Cheng, Smiley W., and Spiring, Frederick A. 1988. A New Measures of Process Capability: C_pm, Journal of Quality Technology, Vol.20, No. 3, 162–175.

Taguchi, G. 1986. Introduction to Quality Engineering. Tokyo, Japan: Asian Productivity Organization.

Wheeler, Donald J. 1995. Advanced Topics in Statistical Process Control. Knoxville, TN: SPC Press.

MVP Programs