Transcat Webinars and Online LearningTranscat Webinars and Online Learning

Process Accuracy Ratio vs
Process Uncertainty Ratio

(Risk Mitigation: Calibration and the Customer’s Process)


The requirements for the process accuracy may be dictated by the requirements of the expected output of the process or product. The ratio between the process accuracy (i.e., the acceptance limits of the manufacturing process, or it could be the product itself) and the instrument accuracy (i.e., the equipment used to measure the process) is the Process Accuracy Ratio (PAR). Just as a calibration standard’s accuracy is an incomplete representation of a calibration process, the sole use of accuracies of the manufacturing measurement process may omit large errors that could change the outcome of the measurement or test. The measuring process can be impacted by many factors. We will define the inclusion of possible sources of error as uncertainty components of the process and thus more exactly define the ratio as the Process Uncertainty Ratio (PUR).

Calibration laboratories have the same need to determine the uncertainties of their measurement processes. An important part of metrological traceability is uncertainty and a good measurement assurance program. We must build an uncertainty budget for the measurement process whether it relates to calibration of instruments or the use of instruments to measure a manufacturing process or the product. Metrology laboratories that are accredited to ISO17025 are required to calculate the uncertainties of their calibration processes. In a similar manner, you can demonstrate that all the possible components of error are accounted for when determining the process uncertainty.

Questions that should be asked: What possible components of error will affect my output or product (time, atmospheric conditions, uncertainty of the calibration performed on the instrument I am using, proper use of the instrument upon which pass/fail decisions are being made, et.al.)? What is the potential risk involved if the process isn’t evaluated for potential components of error?

We may find that the accuracy of the instrument used to measure the process may not be the biggest contributor of error in the process. Once we determine the potential errors, we can begin to eliminate as much of that error as possible through statistical process control (SPC) or other means. As the customer of calibration services, it is your responsibility to ensure the calibration received supports your process requirements and that you take into account all sources of error when using instruments to make decisions about your manufacturing processes or concerning your product’s quality.

Introduction

The act of measuring a part of a process to determine how it is functioning or to ensure that it remains within some required parameter is something that companies do all the time. Companies perform such tests as part of the manufacturing, SPC process or an internal calibration process. The process could be measuring the temperature on a heating surface in an injection molding machine, measuring the force provided by a pneumatic actuator or using a torque wrench to tighten lug nuts on car tires. The process accuracy ratio (PAR) is the ratio between the accuracy of the process being measured or checked and the accuracy of the instrument being used to make that measurement. But to get this ratio, these accuracies must first be converted to tolerances in the same unit of measure.

However, the PAR is inherently incomplete. The important question is, why is the PAR incomplete? To explain why we need to start by discussing the definitions of calibration and Metrological traceability. The requirements of the calibration process and traceability go hand and hand for accredited calibration laboratories. It is a fundamental part of the accreditation process and integral to using the information to calculate uncertainties for the purposes of knowing the TUR and traceability.

Calibration: an operation that, under specified conditions, in a first step, establishes a relation between the quantity values with measurement uncertainties provided by measurement standards and corresponding indications with associated measurement uncertainties and, in a second step, uses this information to establish a relation for obtaining a measurement result from an indication.1

To sum up this definition, calibration is the comparison of an unknown measure and to a known measurement and having the uncertainty information to go with it so that the measured value has meaning. So, by this definition, we can see that part of calibration is including the measurement uncertainties. Why? That is where Metrological Traceability comes into the picture.

Metrological Traceability: a property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty. 1

As can be seen from the definition for Metrological Traceability, measurement uncertainty is part of the requirement. Given this information, the process accuracy to instrument accuracy would not be adequate to provide traceability to the measurement and doesn’t take into account outside sources of error. Within those definitions, we can begin to discuss the missing components of our measurements and show the difference between a PAR and process uncertainty ratio (PUR).

Temperature Example PAR (Process Accuracy Ratio)

Let’s take the last example of the heating surface of an injection molding machine mentioned in the Introduction and determine our PAR (process accuracy ratio). As this is a non-specific example, let’s say that the measurement that must be made is at 150°C with a tolerance of ±5°C and a resolution of 0.01. The second part of this PAR equation is the instrument used to measure the temperature. The thermometer used has an accuracy of ± (0.05% rdg + 0.5°C). From this information, we can calculate the PAR.

1. Determine the Thermometer Tolerance

Thermometer Tolerance = 150°C*0.05% = 0.075°C + 0.5°C = 0.575°C = 0.58°C

Note: In the last step of this equation, the tolerance was rounded, based on the thermometer’s resolution.

2. Determination of PAR

Determinatin of PAR

A ratio of 8.6 to 1 is usually acceptable. So the first question that comes to mind, other than is the meter calibrated, is what about the probe? Even a simple thermocouple bead probe has an accuracy. The probe accuracy should be considered even if you are not calculating uncertainties. The probe is critical to the measurement but you can’t just use any thermocouple probe. It should be a calibrated thermocouple probe.

There are actually 2 components to the measuring instrument; typically, this includes the thermometer and the probe for measuring temperature. The probe for this example has an accuracy of ± (1.1°C or 0.4%rdg) whichever is greater. As you can see our simple PAR is already becoming less simple. What do you do with the accuracies of the thermocouple and the probe? The accuracies have to be combined. Following the law of propagation of uncertainty, the Root-Sum-Square (RSS) method is used to combine sources of error.

1. Determine the Thermometer Tolerance

Thermometer Tolerance = 150°C*0.05% = 0.075°C + 0.5°C = 0.575°C = 0.58°C

2. Determination of Probe Tolerance

Probe Tolerance = 150°C*0.4% = 0.6°C Therefore 1.1°C will be used.

3. Determination of PAR

Determinatin of PAR

Now we are less than half of our original PAR of 8.6:1. This is a more complete PAR than the first, however, what comes from the PAR analysis is that the instruments being used further encroach on the process tolerance. A PAR of 4:1 may be more than adequate except looking back at the definitions of calibration and traceability we also need to know the calibration measurement uncertainty to further understand the impact of the measurement on the process. With the limited information provided at this point, we cannot do this.

4. Temperature Example PUR (Process Uncertainty Ratio)

Using the same example from above, the first thing we have to do is establish the uncertainty for using the thermometer and probe to make the measurement before we can calculate our PUR. While this isn’t a paper on calculating uncertainties or statistics, it would be almost impossible to talk about the difference between PAR and PUR without including some discussion. Calibration laboratories that are accredited are required to create what is called an uncertainty budget for every measurement parameter on their scopes. The point of this is to quantify and combine the possible sources of uncertainty that could affect the measurement. This same process can be used as a template to determine the uncertainty of the measurement process. What are the possible sources of uncertainty beyond the accuracy of the instrument being used in the process? Well, looking back at the definitions again, we can deduce that the calibration uncertainty of the indicator and the calibration uncertainty of the probe are additional sources of error. Other possible sources of uncertainty could be the placement of the probe during the measurement or the length of the thermocouple probe wire (wire longer than 6 ft can induce additional error) however these would be more complicated to quantify. The uncertainty budget will combine all these components using an RSS method.

Let’s assume the thermometer and probe calibration included a statement of uncertainty; from this, we can develop the process uncertainty. Using a calibration uncertainty for the thermometer of ±0.15°C and ±0.27°C for the probe, the table below shows the simplified budget.

Figure 3

The divisors are used to bring all the values to the same frame of reference namely one standard deviation, also known as k=1 or 1 sigma. If you are unfamiliar with these statistical terms you are only a one-semester class away from understanding just enough about statistics to help you in your everyday life – not just at work, because statistics are everywhere you look! The basis of the divisors relates to the type of probability distribution of each of the uncertainty components. Regardless of each component’s type of distribution, the Central Limit Theorem tells us that, combining multiple distribution types results in a normal distribution. Therefore, the combined standard uncertainty of ±0.73 is at a k=1 value of a normal distribution. These concepts may be a little confusing but it is outside the scope of this paper to try to cover a course in statistics. The majority of calibration uncertainties are referenced to k=2 or 2 sigmas. The 2 sigma value of the above budget would be ±1.5°C (rounded up to 2 significant digits). What this means is that, over the span of its calibration interval, the thermometer with probe could actually be measuring off from the 150°C value, within the range of 148.5°C to 151.5°C. This doesn’t mean that measuring the process will be within that tolerance.This idea is based on the probability that each uncertainty component is drifting, each at different rates over time and the combination of these changes results in the thermometer/probe combination changing during its calibration interval. Detailed information for calculating uncertainties can be found in the GUM (Guide to the Expression of Uncertainty in Measurement) 2 or NIST Technical Note 1297 3.

Finally, the PUR can be calculated using the above information.

Determinatin of PAR

In the above example, the addition of the calibration uncertainties did not have a huge impact on the process however it shows the additional uncertainty components did have an effect on the ratio, see Figure 4. You may be asking, what if the measured value was not nominal? What if the value measured was, let’s say 154°C? Those would be valid questions and outside of the scope of this paper. However briefly explained, it would mean that there is some chance that this process measurement at 154° C, considered a passing value, could actually be lower or higher than this value, depending on the error that exists in the thermometer/probe at that moment in time, see Figure 5. This is what is considered a False Acceptance situation, in which the operator believes the product/process to be good when it is not.

Determinatin of PAR

5. Mechanical Example PAR (Process Accuracy Ratio)

Let’s take the last example from the introduction of using a torque wrench to tighten lug nuts on car tires. As this is a non-specific example let’s use the example of torquing the lug nut to 100 lbf-ft with an accuracy of ±10% as the first part of our PAR. The second part of our PAR is the torque wrench used to perform the action of tightening to the specific value. If a torque wrench with a range of 250 lbf-ft and has an accuracy of 2% rdg is used, the PAR can be determined, see Figure 6 for graphical representation.

Step 1. Determine the Torque Wrench Tolerance

Torque Wrench Tolerance = 100lbf - ft * 2%rdg = 2lbf - ft

Step 2. Determination of PAR

Determinatin of PAR

A PAR of 5 to 1 is great, right? There should be no issue with this process but, in reality, there is…

6. Mechanical Example PUR (Process Uncertainty Ratio)

So, what is the calibration uncertainty of the wrench? The calibration uncertainty would be the uncertainty reported by the calibration service provider (CSP), whether it is internal or external to your company. Let’s say that we have a calibration certificate from the CSP and it listed the uncertainty of 0.8%. This would mean that the CSP measurement was only 2.5 times better than the torque wrench.

So what other sources of uncertainty could affect the use of the torque wrench in a process. What about the operator of the wrench? As part of determining the PUR, the sources of uncertainty must be identified and then each source, or component, quantified. The uncertainty for operator error can be done by performing an experiment to determine the repeatability/reproducibility of 3 different operators. The data is shown below in Figure 7.

Determinatin of PAR

What we see from the data above is that there is greater uncertainty in the user’s ability to reproduce a specific reading than in the wrench itself or from the calibration uncertainty. Unfortunately, this is common when using a torque wrench. The reason calibration providers can get better uncertainties is because they employ techniques and fixtures to eliminate the error of the calibration technicians. In Figure 8 is the simplified process uncertainty budget with the standard deviation from Figure 7 included.

Determinatin of PAR

Finally, the PUR can be calculated using the above information.

Determinatin of PAR

In the above example, the addition of the calibration uncertainty and the uncertainty of the operators had a large impact on the process uncertainty ratio (PUR), see graphical representation in Figure 9. What if the value measured was, let’s say 105 lbf-ft? Notice there is a significant part of the bell curve outside the process limit, see Figure 10 below.

Determinatin of PAR

7. Conclusion

It is in the process owner’s best interest to ensure that they understand the measurement they are making and, as part of that, how the additional errors that are present but unaccounted can affect that measurement. As demonstrated in the examples, the PAR is not a realistic representation of the error that exists in a measurement process. The outside sources of error can and will contribute to the overall measurement uncertainty, subsequent PUR, and the risk that exists in the measurement. The addition of the sources of uncertainty had a much larger effect on the mechanical example then it did on the electrical example, within these two examples. Understanding the potential sources and including them allows the process owner to minimize the uncertainty by knowing where they can better control the measurement process. It is apparent from the two examples that each situation can be unique.

References

1. JCGM, International Vocabulary of Metrology – Basic and general concepts and associated terms (VIM), 3rd Edition, 2012.

2. JCGM, Evaluation of Measurement Data — Guide to the expression of uncertainty in measurement, 2008-09.

3. NIST Technical Note 1297 – Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results