Tải bản đầy đủ - 0 (trang)
3 The Importance of the Standard Operating Procedure (SOP)

3 The Importance of the Standard Operating Procedure (SOP)

Tải bản đầy đủ - 0trang


Practical Aspects of Designing and Conducting Validation Studies Involving…


ings should be avoided. Some of the questions that should be asked by the authors

of in vitro methods are:

• Is the description and the definition for the intended use of the in vitro method


• Is the mechanistic basis of the in vitro method adequately described?

• Are the reference and control items clearly identified in the in vitro method?

• Are the acceptance criteria clearly defined and are they based on solid and verified experimental data?

• Does the solvent interfere with the results? What type of solvents may be used?

• Are the limits of the in vitro method adequately characterised?

• Are there gaps or missing information in the overview of how the in vitro method

should be conducted?

• Is it possible for a person with the adequate technical background to reproduce

the method based only on the information included in the SOP?

• Is a dose-range finding procedure established that allows the selection of a meaningful dose-range for the test item?


Limitations and Applicability of the In Vitro Method

It is important to clearly describe in the SOP the applicability domain of the in vitro

method, as well as any limitations or exceptions. For instance, some in vitro methods will only be compatible for technical reasons with liquid chemicals but not with

solids or other physic-chemical states. This will enable the proper application of the

method and will avoid the generation of misleading data. For example, limitations

in terms of applicability could stem from already known limits of use for a particular class of method, difficult chemical types (e.g. volatiles), lack of metabolic competence (biotransformation) of the test system or an absence of critical transporters.


Apparatus, Reagents and Special Consumables

A brief description of the essential requirements of the apparatus (analytical and life

science measurement techniques) required for the in vitro method should be

included into the SOP. Trademarks should be avoided, unless a specific manufacturer’s product is required for a well-defined reason. If special types of plastic ware

are required, then the significant characteristic desired should be clearly stated.

Reagents and materials required for each procedure should be listed as a separate

section under each subdivision, including purity (if applicable), CAS or identification number. Finally, it is important to describe any specific requirements in case a

complex apparatus is used (e.g. precision, detection limit, limit of quantitation), or

in case a critical reagent is used (e.g. purity or special handling etc.).



S. Coecke et al.

Establishing Acceptance and Decision Criteria

Acceptance criteria should be established and described in the SOP for each of the


Test system—performance (assessed by the response of positive and negative

controls items), growth rate/curves, contamination free (e.g. mycoplasma), passage

number range, cell recovery.

Test chemicals—performance (reference items, negative and positive control

items), solubility, cytotoxicity, dose response, etc.

Analytical or life science measurement technique—linearity, accuracy, sensitivity, reproducibility, performance (reference item(s), internal controls, standards,

quantitation limit).

Data analysis—number of runs (SD, % CV, etc.), acceptable failure rate, statistics to be employed (e.g. curve fitting), curve fitting acceptance.

Data analysis templates (spreadsheets)—ensure they are a true reflection of the

SOP (i.e. are all data analysis described in SOP)

Reporting—outcome to be reported and units of reporting

Additionally, decision criteria to decide whether to accept or reject a test run

should be described in detail in the SOP. These decision criteria should be realistic

and take into account:

• What response of the biological test system can be achieved?

• What definitive activities or SOP steps must occur?

• What tools will be needed to execute the SOP steps? How one knows that the

SOP steps are successfully completed?

• What processing apparatus is involved or will impact the SOP steps execution?

• Who is responsible for executing the SOP step(s) or following the procedure?

• In what order must these SOP steps occur in order to succeed?

• How should the steps be executed and the response achieved should be


All appropriate Quality Assurance (QA) and Quality Control (QC) activities for

that SOP should be described with significant references.


Selection of Test Chemicals

The selection of test chemicals for validation studies is a complex process and the

following considerations should be addressed during the selection process:

1. Data on chemical activity/toxicity ideally in the species of interest (e.g. humans)

should be available so that the relevance and reliability of the in vitro method can

be assessed. A small set of chemicals to further test the limits of the method may

be included;


Practical Aspects of Designing and Conducting Validation Studies Involving…


2. The selected total pool of chemicals should be diverse in terms of their structure

and biological effect (potency) to ensure the robustness of the method. Although

not easy to practically implement, the known applicability domain of the method

is extended if validated with diverse chemicals and further confidence can be

achieved if the in vitro method has also been assessed with a small but representative set of chemicals of particular interest such as nanomaterials and


3. Test chemicals should be suitable for testing and comply with practical constraints such as solubility, chemical stability, commercial availability and cost.

Typically 10–50 test chemicals, depending on the availability of high quality

reference data, are selected for a multi-study trial involving the EU-NETVAL

test facilities. Besides the availability of high quality human (or other target species) reference data, above mentioned practical constraints as well as processing

time, handling, storage and safety requirements, limit the number of test items.

To achieve such a complex task, manual selection based solely on expert knowledge becomes more difficult and computational tools may facilitate decision



Collection of Reference Data Associated with the Test Chemicals

The most reliable way to assess the biological behaviour of selected test chemicals

is to explore the existing literature and documentation. Chemistry-based systems,

such as PubChem or Scifinder allow querying by molecule (e.g. name, CAS number, structure) and retrieving a significant part of the scientific literature concerning

the query chemical. Specific databases, depending on the biological mechanism

studied, are more suitable. It is also acceptable to cite the Toxcast/Tox21 initiative

that generated in vitro responses for a large set of chemicals and assays (US EPA

2010a, b; Tice et al. 2013). In the end, it is recommendable to collect a state-of-theart of all (or most of) the chemical substances based on human exposure data, or on

similar biological in vitro test systems with same or similar characteristics and conditions. It is then feasible to assign for each chemical substance a confidence score

defined by the relevance of the data collected and the frequency of the behaviour


Literature findings can be completed and/or confirmed by in silico predictions.

For example, many reliable Quantitative Structure-Activity Relationship (QSAR)

or docking models exist for receptor binding measurements and can be a good supplement to fill in gaps in datasets or reinforce a given observation. Phenomena

involving very specific chemical mechanisms (e.g. protein or DNA binding) are

tightly connected to well-known chemical reactivity and reliable models have been

produced. The known space of biological response will define the chemical selection of a validation study. Exploratory compounds can nevertheless be added to the

selection set for the sake of new data generation or mechanistic elucidation; but it

remains quite minor compared to the whole set of selected chemicals. In order to


S. Coecke et al.

have a better picture of the selected test items, it is often possible to collect in vivo

data related to the tested system. However, in toxicology, the tested system may

only be a part of a broader toxicity pathway (or Adverse Outcome Pathway, AOP)

and finding a direct correlation between the tested in vitro method (often accounting

for an effect at receptor or cellular level) and an underlying adverse effect (at the

organism level) is not always easy. For example, when testing an endocrine-related

receptor (oestrogen or androgen receptors), it is not obvious to relate the tested

phenomenon (most probably agonism or antagonism) to an actual in vivo observation such as developmental, reproductive, or carcinogenic effects.


The Diversity Issue

When validating an in vitro method, the applicability domain is of major importance

since it will define the chemical classes for which the method is able to give reliable

measurements. The applicability domain is defined by the chemical substances

tested during validation and in principle the method should only be reliable for similar chemicals by interpolation or, to some extent, with a slight extrapolation. It is

then advisable to maximise the diversity of the chemicals selected for validation in

order to extend the applicability domain of the tested method.

From a chemistry point of view, it is desirable to cover a wide chemical space,

meaning a large number of different structures. The exercise of covering chemical

space is a trade-off between the number of chemical classes covered (the “area”)

and the number of representatives of each class (the “density”). It should be kept in

mind that the chemical selection is always limited by the number of compounds that

can be used for validation. Therefore, it is not possible to cover all chemical classes

while maintaining a high density. Depending on the aim of the validation study, one

wants to focus on the area covered or the density. A large area with low density will

ensure a large applicability domain of the tested system but with a loss in reliability.

On the other hand, a smaller area covered more densely will allow more reliable

responses but will only be applicable for a limited count of chemical classes.

The structural diversity can be evaluated with structural descriptors like structural keys or fragment fingerprints. Similarity measures (such as the Tanimoto

index) are able to quantify the diversity (i.e. a low average similarity between compounds) of a dataset. Strategies based on chemo-informatics methods are used to

pick up a restricted, yet diverse, subset out of a source dataset. The sphere exclusion

(or cell-based) method and the clustering method, which are illustrated in Fig. 5.2,

are examples of such strategies. In the sphere exclusion, chemicals are projected in

a multidimensional space made of user defined descriptors (either structural or

physical/chemical properties). A compound is randomly picked as the seed and

selected. All the compounds in the neighbourhood (defined by a threshold) are

excluded and another molecule is then picked up outside the excluded sphere.

Iteratively, compounds are picked-up and their closest neighbours excluded until

there are no more compounds left to select or until the target number of chemicals

is reached. It is also possible to use clustering methods. The dataset is clustered


Practical Aspects of Designing and Conducting Validation Studies Involving…


Fig. 5.2 Example of application of two diversity selection methods: sphere exclusion and cluster


according to the inter-compound similarity (again, either based on structural or

chemical descriptors). A threshold is set to define n clusters (n being the target number of chemicals used for the validation study) and one compound is selected (randomly or according to other criteria) inside each cluster giving eventually a selection

of n compounds.

Following the same reasoning, diversity of biological response should not be

neglected. As chemical diversity has to be maximised to ensure a large applicability

domain, similarly the potency spectrum should be adequately covered. The data

compiled in the data collection step help at ranking compounds, triggering high,

medium or low responses from the system. The final selection should then include

compounds covering a wide range of biological response to be sure that the system

is tested in all possible values of the spectrum. Obviously, it concerns only the positive response (triggering an observed effect) while the negatives are considered as

zeros or close to it. At this stage, QSAR models could also help at evaluating or

confirming the expected behaviour of the compounds. Note that, due to the nature

of the biology and the specificity of the system tested, positive compounds typically

cover a smaller structural space than the negative ones. This is especially true for

specific receptor-based assays for which only very fine structural details may trigger

a response (agonist or antagonist).


Property Predictions

Existing computational models can help evaluate the biological response of the

tested system. An extensive number of quantitative (QSAR) and qualitative (structural alerts, decision trees, profiler) models are available nowadays covering a large

landscape of biological effects, from receptor binding to systemic toxicity. The


S. Coecke et al.

OECD’s QSAR Toolbox (OECD 2012) is a recommended approach as it is free of

charge and developed to generate predictions in various scenarios related to toxicity.

Advanced simulations, combining docking and QSAR could help to evaluate enzymatic assay responses (EC50, IC50), like Virtual ToxLab (Vedani et al. 2009). In

silico predictors could also be used for the calculation of additional selection criteria. Particularly, it can be interesting, for experimental design, to generate data on

thermodynamics and kinetics of the compounds. Some models are able to calculate,

for example, protein binding that could be crucial in certain assays. Also, kinetic

parameters can be derived from simulation with physiologically-based pharmacokinetic (PBPK) modelling. Finally, metabolism can be simulated by rule-based

engines in order to predict the most probable metabolites that could influence the

results of cell or tissue based assay.


Practical Examples

For the validation of an androgen receptor transactivation assay (namely,

AR-CALUX) at EURL ECVAM, the test item selection relied mainly on collection

of available knowledge from various sources. We gathered expert opinions, published literature, collaborators’ results as well as in-house results in a comprehensive data table for about 80 chemicals. They were evaluated based on a score

accounting for the number of observations of three possible behaviours in the assay

(agonist, antagonist, negative). Ranked by confidence (higher scores), the top chemicals were selected to obtain a balanced set of 45 (15 of each of the 3 classes).

Available potency data were also gathered from the same data sources (combined

with in-house experiment on solubility of compounds) in order to set up experimental protocols (e.g. test doses range).

In conclusion, in the context of a prospective validation study, chemical selection

is often totally based on theoretical work. That is why in silico tools can be used

appropriately at several stages of the chemical selection. Ideally, data on the potential chemicals should be collected through dedicated databases or inventories. The

selection should maximize the chemical diversity covered with the help of chemoinformatics that provides similarity-based tools. Also, the biological response

should be thoroughly explored by selecting compounds with different profiles

(determined by literature findings that could be supported with predictions). Finally,

theoretical models are able to predict a whole range of properties that can help at

fine-tuning the chemical selection taking into account the experimental design and

the underlying limitations.


Test Chemical Management

Solubility and stability testing of test chemicals (test, reference and control items)

is part of the test chemical management activities. The process starts with their

acquisition and ends with their shipment to the EU-NETVAL test facilities for the


Practical Aspects of Designing and Conducting Validation Studies Involving…


Fig. 5.3 Test chemical management: from acquisition to distribution

multi-study trial. The entire process includes first checking on the availability of the

test chemicals, the acquisition, the registration, the labelling for their identification

and the control of the correct storage conditions in order to trace and guarantee the

chain of custody. The aliquoting, coding (when required), and packaging of the test

items for the appropriate distribution is also an important test chemical management

activity during a validation study. Tests for solubility, stability and variability

between batches are performed at EURL ECVAM test facility. Figure 5.3 illustrates

all these steps.

Solubility verification ensures compatibility of the test chemical with the in vitro

method under evaluation. Before selecting a test chemical for a validation study, it

is important to check if the test chemical is and remains soluble both in the stock

solution and in the in vitro method incubation medium (working solution) at the

desired concentration and under the in vitro method experimental conditions.

After a preliminary phase, theoretical in vitro method performance evaluation

and theoretical SOP optimisation, solubility testing is performed in standard

throughput mode to generate sufficient data in a reasonable time. The traditional

solubility testing approach by visual inspection of the solution is subjective. A more

objective method that has been used in EURL ECVAM validation studies is using

the Tyndall-based nephelometer method. Nephelometry provides an objective

indicator of solubility, differentiating trace dispersions from limpid solutions by

relative turbidity, detected as Tyndall-effect light scattering of a transmitted laser

beam. The method is relevant and accurate at threshold concentration ranges of

solubility in contrast to the visual inspection. Automated instrumentation for multiwell microplates allows rapid serial measurement of sample aliquots, applicable to


S. Coecke et al.

incremental concentrations of solute over a relevant range and/or batch screening of

multiple test items under specific conditions. For instance, for the needs of a validation study it is feasible to obtain data on 12 test items/week, at 3 different concentrations, in pure solvent (usually dimethyl sulfoxide—DMSO) and in culture medium

by preparing samples manually in 96-well plates.


Solvent Compatibility Assessment

The use of in vitro methods to study biological endpoints can be confounded by the

interaction of the solvent used to prepare the stock solutions (carrier solvent) and

the biological test system. The most common effect is toxicity, which might stem

not only from the test item but also from the carrier solvent. Strong toxic solvents

with properties in terms of corrosivity, mutagenicity, carcinogenicity, genotoxicity

or teratogenicity, which have the potential risk to induce adverse effects, should be

avoided and only a compatible scale of solvents for stock solutions preparation may

be considered. In some cases, carrier solvent might not cause toxicity but it might

interfere with the test system in such a way that it masks the in vitro response. For

instance, some solvents might interact with Cytochrome P450 (CYP) enzymes and

thus interfere with the results of in vitro CYP induction or in vitro metabolic clearance methods. Also in such cases care should be taken to use appropriate solvent

and at acceptable final concentration in the incubation mixture of the in vitro

method. Commonly, a sequence of compatible solvents can be resumed as: water,

DMSO ethanol and methanol. In general DMSO is an appropriate solvent for

organic test items such as pharmaceuticals. Alternatively water would be suitable

for inorganic compounds. In conclusion, solubility testing must be restricted to carrier solvents that are compatible with the in vitro biological system employed in the

in vitro method.


Test Chemical Purchase and Distribution

In practice, chemical substances are produced for laboratory research and development and are available from retail suppliers who provide a certificate of analysis.

The facility responsible for test chemical recipient should ensure the chemicals are

stored at the recommended temperature (refrigerator, ambient, freezer) with attention to any additional conditions (i.e., inert gas for air sensitivity, and darkness for

light sensitivity). Unless otherwise indicated, chemicals are allocated an expiry date

2 years from acquisition. This practice should be handled with care and identity and

stability verifications are highly recommended.

Distribution of the test items used for the validation should be in compliance

with International Air Transport Association regulations, potentially hazardous

goods require declaration by UN number, if applicable, indicated in the Material


Practical Aspects of Designing and Conducting Validation Studies Involving…


Safety Data Sheet (MSDS). The shipment package should include the MSDS of

each chemical, with the corresponding test item code, enclosed in a sealed envelope

labelled for customs use only (to be discarded unopened, on arrival at the participating test facilities). Relevant to remedial procedures at a participating test facility in

case of accident or emergency only, an additional MSDS for each test item should

also be included, sealed individually in opaque envelopes identified only by code.

The MSDS envelopes should remain sealed during the study, with any recourse for

consultation reported to the VMG.

Each shipment should be addressed to a nominated responsible person at the test

facility. Shipments should be made early in the week, avoiding potential delays due

to week-ends. On arrival, recipients should confirm delivery of complete and intact

test items, including integrity of the sealed MSDS envelopes.

Following the modular approach to validation studies, only certain modules

would normally require coding of test items. In general the training and transfer

phases would not involve coded test items, while the EU-NETVAL multi-study trial

testing for reproducibility and predictive capacity may justify test item coding to

ensure full independence of the experimental work by the participating test


Coding of test chemicals (chemical aliquots) for distribution to participants in a

validation study should be systematic, to facilitate logistical management, but also

unique, to ensure identity encryption.


Good Experimental Design and Data Interpretation

Based on Statistical Analysis

Drawing conclusions on the basis of the data obtained from in vitro methods depends

a lot on the correct interpretation of the information obtained and their statistical

analysis. For a validation study a first statistical analysis should be done on data

provided by the in vitro method submitter. EURL ECVAM assesses if acceptance

criteria can be met by the EURL ECVAM GLP test facility. It is of crucial importance that the in vitro method submitter sets its acceptance criteria on historical data

and uses the necessary statistical tool to set the acceptance window. During the validation study, a global statistical analysis is conducted centrally by EURL ECVAM.

The main analysis should reflect the purpose of the validation study. It is therefore needed to extract all of the useful information and present the data in a way that

it can be interpreted, taking account of biological variability and measurement

error. The methods applied should be of such a kind that any knowledgeable reader

with access to the original data can verify the results. For a validation study it is

therefore necessary to perform a statistical analysis to describe the within-laboratory reproducibility, the transferability (including goodness of fit and robustness)

and the between-laboratory reproducibility. The statistical analysis also needs to

assess the predictive capacity and the applicability domain of the proposed in vitro

method. The predictive capacity will be influenced by the number and range of


S. Coecke et al.

chemical substances and the quality of the reference chemical substances and

should maximise the specificity (also called the negative rate) and sensitivity (or the

true positive rate).

Often users of in vitro methods are confronted with the fact that the received

in vitro method and its original method description and definition should be optimised for a variety of different reasons such as the need for the method to become

a formal test guideline, the adaptation of the method for specific user-requirements,

etc.. A systematic approach should be taken when standardising the in vitro

method. In general, the parts that are to be standardised and improved should be

clearly identified including problem formulation, and indicate how the standardisation and implementation will be measured (what parameters are to be optimised

and how these optimised parameters will be measured e.g. using standard deviations, % CV, etc.).

Factorial design can be used as an experimental approach to obtain a well-defined

in vitro method and can be subsequently used as a tool for any further standardisation and improvement needs as it permits simultaneous evaluation of multiple factors that might influence the performance of the in vitro method. Specific questions

that can be asked are: Are the numbers of replicate and/or repeat experiments appropriate for each experimental step in the in vitro method? And is a good in vitro

method experimental design of those steps used in the SOP where such design is

critical in terms of adequate placing of test, reference and control items, generating

enough data to draw conclusions, etc.?

A good statistical practice is a necessary element in the experimental design of

in vitro methods and facilitate the method definition and optimisation and also, if

applicable, the subsequent validation steps. If there is need to optimise an in vitro

method, the parameters to be optimised should be well described, an objective function should be established and results should be supported by sound data. However,

in vitro method developers should pay careful attention to ensure that the statistical

practices are implemented correctly and the results are appropriately interpreted.


Importance of Good Data Management

When data is not well-defined there is the risk that they are misused and wrongly

interpreted which could lead to false conclusions. In order to avoid this, good data

management should be followed throughout the whole data lifecycle i.e. from the

planning to the creation of data up to the storage and eventual when the data can be

considered obsolete to the deletion of the data. Tools to obtain well-documented

data are pre-defined guidance documents proposing terminologies and a fixed data

reporting format. When eventually data is produced, it should be reported in the

correct format and accompanied with a detailed, well-documented, description of

the procedure followed (including also background literature used). Finally, clearly

described instructions of acceptance criteria and data transformation should be


Practical Aspects of Designing and Conducting Validation Studies Involving…


provided in order to be able to draw correct conclusions from data. At the end of a

validation study all data and documentation should be stored in such a way that any

subsequent regulatory question that might come up in the future can be addressed.

After data recording of in vitro methods, data analysis is performed. Spreadsheet

forms are used for such data analysis. Such forms belong to the category of computerised systems. Indeed, computerised systems can vary in type (e.g. hardware, software), in complexity and in dimension. Some examples of computerised systems

are: programmable analytical instruments, personal computers and laboratory information management systems (LIMS), but also electronic spreadsheets used for the

storage, processing and reporting of data. Spreadsheets (e.g. MS Excel) are widely

used for data analysis and storage of electronic data.

Their complexity varies enormously depending on the actual in vitro method

being performed. When using spreadsheets for performing routine data handling,

these spreadsheets should be considered as part of the SOP. The design and validation of spreadsheets when used in a quality environment such as GLP has been

addressed in specific guidelines for the development and validation of spreadsheets.

When developing spreadsheet forms or other applications (e.g. database), both their

design and validation should be planned and documented. For complex applications, their use should be documented either in the in vitro method SOP or in separated SOPs. The statistical method required for data treatment and analysis should

be documented in the in vitro method SOP, including a description how to interpret

the final results.

The use of computerised systems by test facilities for the generation, measurement or assessment of data is nowadays consolidated and computerised systems are

fully integrated into the in vitro method. It is essential that for regulatory applications, computerised systems are developed, validated, operated and maintained in

accordance with the OECD Consensus document No. 10 on “The application of the

principles of GLP to computerised systems” (OECD 1995, undergoing a revision in

2014 and specific guidance documents (PIC/S 2007; AGIT 2007). All statistical

methods and calculations to be used should be described in the method. Some

checks can be done when evaluating the completeness of the SOP for the statistical

aspects such as:

• A clear description and definition of the statistical or non-statistical methods

used to analyse the resulting data are provided.

• A clear description and definition of the decision criteria and the basis for the

prediction model used to evaluate the test item in relation to its required response

are provided.

• Control if the relevance (e.g. accuracy/concordance, sensitivity, specificity, positive, and negative predictivity, false positive and false negative rates) of the

in vitro method is adequately described.

• Assess if specific measures of variability are adequately included.

• Evaluate if the acceptance criteria are based on historical experimental data.

Tài liệu bạn tìm kiếm đã sẵn sàng tải về

3 The Importance of the Standard Operating Procedure (SOP)

Tải bản đầy đủ ngay(0 tr)