On the evaluation of reliability, repeatability, and reproducibility of instrumental evaluation methods and measurement systems.

Validation of instrumental evaluation methods or measurement systems plays an important role in both pharmaceutical and cosmetic research and development. In practice, it is suggested that validation should be performed according to performance characteristics as described in the United States Pharmacopedia and National Formulary (USP/NF, 2000) for analytical methods validation. A validated method or measurement system is expected to achieve a certain degree of accuracy and reliability. However, it is a concern whether the test results obtained are repeatable (with similar test samples) and/or reproducible (under similar but slightly different experimental conditions). In this article, reliability and repeatability/reproducibility of a measurement system estimated within a mixed-effects nested design are monitored by relevant variability acceptance limits. A method based on the concept of empirical power (reproducibility) is used to determine these acceptance limits and thus ensure that there is a high probability of repeatability/reproducibility of the tests results. Formulas or procedures for sample size requirements for comparing the variabilities between products are derived. An example is presented to illustrate the use of the proposed method.