The enzyme-linked immunosorbent assay (ELISA), as a universal method for the determination of Microcystins, is of great significance for the rapid detection of Microcystins pollution. This study aimed to propose a simplified validation method for Microcystins ELISA kit by summarizing related documents and guidelines. After summarizing and clarifying from 20 validation parameters, 11 parameters were selected to simplify the validation of Microcystins ELISA kit. In addition, the acceptable range and validation details of each parameter were analyzed. The results indicated that the coefficient of determination of the Microcystin-LR standard curve was higher than 0.99. The concentration of quality control samples was within control limits. The accuracy of spiked and proficient samples was within 70%–130%. The variability of intra-assay, inter-assay, and reproducibility was less than 11, 15 and 21%, respectively. The LOD and LLOQ were 0.002 μg/L and 0.05 μg/L, respectively. When the concentration of Microcystins exceeded 5 μg/L, it was recommended to dilute the samples to the working range before detection. The specificity was estimated with seven Microcystin analogues and three amino acids, indicating that the cross-reactivity was less than 30%. These results revealed that the ELISA kit was satisfactory for detecting Microcystins in water.

  • 20 validation parameters were summarized from related regulatory documents and studies.

  • 11 parameters were selected for Microcystins ELISA kit validation after the clarification and simplification of summarized parameters.

  • Acceptance range and validation details of selected parameters were investigated.

  • A simplified method for Microcystins ELISA kit validation was proposed.

Graphical Abstract

Graphical Abstract
ADDA

(2S,3S,8S,9S)-3-amino-9-methoxy-2,6,8-trimethyl (2S,3S,8S,9S)-3-amino-9-methoxy-2,6,8-trimethyl-10-phenyldeca-4E, 6E-dienoic acid

CRM

certified reference material

CV

coefficient of variation

EC

European Commission

EC50

Concentration for 50% of maximal effect

EMA

European Medicines Agency

ELISA

The enzyme-linked immunosorbent assay

FDA

US Food and Drug Administration

GCC

Global CRO Council

HPLC

high-performance liquid chromatography

PGIMV

Practical guide to immunoassay method validation

QC

quality control

RSD

relative standard deviation

LC-MS

liquid chromatography-mass spectrometry

LOD

limit of detection

LLOQ

lower limit of quantitation

LRB

laboratory reagent blank

SAES

streptavidin-amplification enhanced sensitivity

ULOQ

upper limit of quantification

UNODC

United Nations Office on Drugs and Crime

SD

standard deviation

Climate change and nutrient loading have increased the threat of toxic cyanobacteria blooming in surface water worldwide (Teodoro et al. 2021; Valipour et al. 2021). Several genera of cyanobacteria, including Microcystis, Anabaena, and Planktothrix, can be potentially toxic and produce a family of toxins called Microcystins that share a common cyclic heptapeptide structure carrying a toxic moiety ((2S,3S,8S,9S)-3-amino-9-methoxy-2,6,8-trimethyl (2S,3S,8S,9S)-3-amino-9-methoxy-2,6,8-trimethyl-10-phenyldeca-4E, 6E-dienoic acid (ADDA)) (Foss et al. 2020). The Microcystins family of cyanotoxins is a potent inhibitor of the mammalian protein phosphatase enzyme. Microcystins pose a potential threat to water environmental safety and human health (Mash & Wittkorn 2016). Additionally, the World Health Organization (WHO) suggests 1 μg/L of MC-LR concentration as the maximum limit in drinking water, and the maximum daily human intake value of MC-LR was 0.04 μg/kg (Kohoutek et al. 2019). Due to the adverse effects of Microcystins on human and animal health, the rapid, accurate and low-cost determination of Microcystins in drinking water, irrigation water and recreational surface waters is essential.

The prevalent methods for the determination of Microcystins, such as liquid chromatography-tandem mass spectrometry (LC-MS/MS), high-performance liquid chromatography (HPLC), and enzyme-linked immunosorbent assays (ELISA), have been reported (Geis-Asteggiante et al. 2011; Balest et al. 2016). Although traditional chromatographic methods possess the advantages of high selectivity and accuracy, they suffer from the shortcomings of time-consuming detection, expensive equipment, and cumbersome pretreatment. In contrast, ELISA is an easy-to-use, time-saving, highly specific, and low-cost method for Microcystins detection based on immunological reaction, which combines the specific reaction of antigen and antibody (Li et al. 2006). Thus, due to these advantages of ELISA methods, many commercial ELISA kits have been successfully developed for Microcystins determination. However, some ELISA kits are directly used for the determination of drinking water and surface water samples without prior confirmation of the performance of ELISA kits. This can be questionable because the levels of analytes may be different between water samples and are affected by matrix effects owing to the presence of various interfering substances in waters (He et al. 2017). Therefore, users and institutions need to analyze whether the performance of the ELISA kit achieves the expected purpose. Rivasseau et al. emphasized the detection limit and matrix effect of the Microcystins ELISA kit, and then evaluated these parameters by comparison with HPLC method (Rivasseau et al. 1999). A higher concentration of Microcystin was obtained by ELISA kit than that measured by HPLC method, which was consistent with the studies of Trifirò et al., who observed that the concentrations of Microcystins determined by ELISA were higher than by HPLC-MS/MS (Trifirò et al. 2016). However, a broadly specific competitive indirect ELISA kit was developed for highly sensitive Microcystins determination, which was also evaluated with LC-MS/MS. The results of accuracy and recovery showed that the concentrations of Microcystins measured by the two methods were consistent, indicating that the ELISA kit was successfully developed (Lu et al. 2020). From all mentioned above, the inconsistent results between the determination of Microcystins concentrations by ELISA and HPLC/LC-MS methods in previous studies increased the difficulty for users to confirm the performance of ELISA kits. Moreover, the evaluation of ELISA kits needs to be combined with HPLC/LC-MS methods, which limits the promotion and application of ELISA kits in real water samples detection (Geis-Asteggiante et al. 2011). The evaluation is defined as the critical assessment, in as objective a manner as possible, of the degree to which a service or its parts fulfill stated goals (Reeve & Peerbhoy 2007). It is different from the definition of validation, which is defined as work to perform a set of experiments that reliably estimate the validity and reliability of analytical methods or modifications to previously verified methods (LeBeau 2020). Thus, the performance of the Microcystins ELISA kits can be determined by the validation methods and validated following a series of experiments according to guidelines. The ELISA kit validation studies do not need to be compared with HPLC or other methods, avoiding the disadvantages of evaluation methods such as complicated pre-treatment, expensive equipment and time-consuming assay procedures. Therefore, it is highly important to conduct Microcystins ELISA kit validation studies.

Fortunately, there are many ELISA kits that have been validated according to guidelines or immunoassay validations documents. Nicholas et al. validated the fentanyl ELISA kit for blood and urine according to the forensic toxicology laboratory guidelines, and the validation parameters including sensitivity, precision, specificity, carryover, plate drift, and robustness were investigated (Tiscione & Kristin 2017). Based on the European Commission (EC) decision 2002/657/EC, the selectivity, specificity, ruggedness, precision, and stability of the 3-amino-2-oxazolidinone ELISA kit were analyzed (Rana et al. 2020). This provides the feasibility for the validation of Microcystins ELISA kit. However, among the validation studies of ELISA kit, researchers choose various regulations and documents to validate the performance of ELISA kit, which leads to differences in validation parameters, and a large number of validation parameters are difficult to fully validate. Furthermore, many validation parameters are ambiguous in terminology, and contradictory information is normally present among the validation documents (Rudaz & Feinberg 2018; Raposo & Ibelli-Bianco 2020). For example, the inter-assay and intra-assay (within run and between run) were used as validation parameters of precision in American National Standard/Academy Standards Board (ANSI/ASB) Standard 036, Global CRO Council (GCC) and EMA. However, United Nations Office on Drugs and Crime (UNODC) used reproducibility and repeatability, and Practical guide to immunoassay method validation (PGIMV) used reproducibility, repeatability, and intermediate precision for the validation of precision. The differences in validation parameters choices and ambiguous parameters terminology lead to more challenging and complex validation of Microcystins ELISA kit.

In this study, the Microcystins-ADDA ELISA kit was firstly validated with simplified validation method. We summarized and clarified 20 validation parameters of commercial ELISA kit by outlining relevant regulations and documents. Then, after sorting out the controversies and discrepancies of validation parameters, 11 validation parameters were proposed to simplify the validation of Microcystins ELISA kit. Moreover, the acceptable range of each selected parameter of the Microcystin ELISA kit was discussed and confirmed according to the relevant regulations, which facilitated the application of the ELISA kit by the testers. Finally, the performance of the Microcystins ELISA kit was validated by using the proposed method.

Chemicals and instrumentation

The competitive Microcystins-ADDA ELISA kits were purchased from Abraxis (Product No. 520011SAES, America). The ELISA kit contained six Microcystin-LR (MC-LR) calibrators (purity >97%), positive control solution at 0.75±0.185 μg/L, laboratory reagent blank (LRB), antibody solution, conjugate solution, substrate solution, acidic stop solution and wash buffer solution. The certified reference material (CRM) of MC-LR and drinking water proficiency samples were obtained from Abraxis (America). Other chemicals and reagents were analytical grade.

The adjustable manual pipette (WondaPipet PM II, Japan) and eight-channel pipette (Eppendorf, Germany) were used for pipetting reagents. The microtiter plate was shaken with the mixer (V8 Vortex, American) for sample incubation. The absorbance values of microtiter plate were tested with an ELISA microplate reader (Biobase-el 10A, China) at 450 nm.

Real water samples

The real water samples were collected from Yuqiao Reservoir (reservoir A), Erwangzhuang Reservoir (reservoir B) and Bolong Lake (lake A), and then transferred to the laboratory with ice packs (Tianjin, China). Water samples from reservoirs A and B were collected at different times from fixed sampling points. The lake A water samples were taken from sewage pump pond, rainwater pump pond and lakeside. All samples were filtered by GF/C filter (pore size=0.45 μm) and stored at −20 °C. Finally, the intracellular Microcystins were released after two freeze-thaw cycles and detected within 48 hours. The ammonia nitrogen (NH4+-N), nitrate (NO3-N), total phosphorus (TP), and Chlorophyll a were analyzed according to the methods of Yang et al. (2013). The permanganate index was determined with the study of Tian et al. (2008). The algal density was tested with the method of Kochel et al. (Kochel & Bagwell 2017). The raw water quality data are shown in Tables S1, S2 and S3. Cyanobacteria existed in Yuqiao Reservoir, at the genus level, mainly including Microcystis, Phytocystis, Achthyosoma, and Cytocystis. Cyanobacteria in Erwangzhuang Reservoir mainly included Cyanophyta, Chlorophyta, Diatoms, Euglena, Dinophyta and Cryptophyta at the phylum level.

ELISA procedure

The determination of water samples was performed following the kit instructions. Briefly, a 50 μL volume of standard calibrators, positive control and real water samples was introduced into the microwells. Then, the antibody solution (50 μL) was dropped rapidly using the eight-channel pipette. The microwells were incubated for 30 min at ambient temperature with shaking at 400 rpm. The microwells were emptied and washed five times with 200 μL 1X buffer solution. Then, 100 μL conjugate solution was added and incubated for 90 min, and the microwells were emptied and washed. Subsequently, a 100 μL aliquot of the substrate was introduced into the microwells and incubated for 25 min. Finally, the substrate was transformed by the enzyme conjugate into blue compound, and 50 μL stop solution was added to halt the reaction. The absorbance of 96 well microtiter plates was recorded with ELISA microplate reader at 450 nm.

Validation parameters

Standard curve, QC, accuracy, precision, matrix effects, dilution linearity

The standard curve was established by plotting the percent of the maximum absorbance versus the concentration of the six calibrators in log scale. The concentration of unknown samples was directly estimated from the standard curve. The positive controls at a mid-level concentration of 0.75±0.185 μg/L and negative control samples of tap water samples were used for quality control. A total of 32 replicates (16 tests) of quality control samples were determined.

Accuracy is defined as the closeness of the agreement between the result of a measurement and a true value of the measurement (Bièvre 2008). In this study, the accuracy of the ELISA kit was tested by using certified reference materials spiked samples and proficiency water samples with blind concentration. The CRM was spiked with laboratory reagent blank at a series of concentrations of 0.15 μg/L, 0.50 μg/L, 1.00 μg/L, 1.25 μg/L, 2.00 μg/L and 3.00 μg/L, and used for accuracy measurement. The proficiency samples of drinking water and surface water were analyzed with eight replicates.

Precision is defined as the closeness of agreement between independent test results obtained under stipulated conditions (Nadovich 2005). According to the definition, there were three different types of precision, namely intra-assay, inter-assay, and reproducibility. The intra-assay was completed in one plate with eight strips (96 wells) and inter-assay was tested with 10-day interval (96 wells). The reproducibility relates to the agreement of test results with different operators, test apparatus, and laboratory locations (Nadovich 2005). Thus, reproducibility was measured by inter-laboratory proficiency tests. The reproducibility data were obtained from 32 laboratories of 10 countries with drinking water proficiency samples (from Abraxis drinking water proficiency data). The precision (intra-assay, inter-assay and reproducibility) was estimated by the coefficient of variation (CV) of calibrators and drinking water proficiency samples.

Matrix effect is the influence of the components of a sample other than the analyte of interest in the measurement (Zhong et al. 2011). The standard addition method was an easy way to investigate whether the matrix effect was present. Therefore, a small volume of CRM was added into the surface water samples to validate the matrix effects. The surface water samples with various concentrations of Microcystins were spiked with CRM and retested to investigate the matrix effect.

Dilution linearity is performed to demonstrate that a sample with a spiked concentration above the upper limit of quantification (ULOQ) can be diluted to a concentration within the working range and still give a reliable result (Lee et al. 2006). The CRM was diluted 1,000, 5,000, and 50,000 times with LRB, and the final concentration was 5 μg/L, 1 μg/L, and 0.1 μg/L, respectively. Additionally, the real surface water samples exceeding 5 μg/L were also used to evaluate the dilution linearity.

LOD, LLOQ, specificity, stability and assay drift

Limit of detection (LOD) is defined as the lowest concentration of an analyte in a sample that can be determined. The lower limit of quantitation (LLOQ) is the lowest concentration of an analyte in a sample that could be determined with acceptable precision and accuracy under the stated conditions of the test (Elfving 2014). The LOD combines instrumental performance, sample matrix, and inherent procedural limitations. Therefore, the LOD must be evaluated over multiple runs using a laboratory reagent blank. In this study, 32 repeated LRBs were used to verify the LOD of ELISA kit. The LLOQ was the lowest calibration standard, and the analyte signal of the LLOQ samples should be at least five times over the signal of the blank samples.

Specificity was determined by adding materials that might be encountered in samples (United Nations Office on Drugs and Crime 2009). Seven Microcystins analogues, D-phenylalanine, L-phenylalanine and DL-phenylalanine were used for specificity validation. Stability is defined as the ability of a substance to remain unchanged over time under stated or reasonably expected conditions of storage and use (Brodie & Hill 2002). Proficiency drinking water samples with the concentration of 0 μg/L, 0.5 μg/L, 1.5 μg/L, and 2.0 μg/L were used for analysis. The frequency of measurement was once a month, a total of four times. Assay drift refers to the systematic error rather than the random error of the analyte concentration, and its magnitude depends on the position of the sample in the plate (Wild & Sheehan 2013). A possible cause of the assay drift was determined by an eight-channel pipette and ELISA reader.

Evaluation of the validation parameters of ELISA kit

Many regulatory documents and publications concerning immunoassay or ELISA kit validation parameters have been published. In this study, six international guidelines and four validation studies of ELISA kit were investigated and are summarized in Table 1. A total of 20 parameters were identified in the ten publications. The + sign indicates that the parameters needed to be validated. The number of occurrences of each parameter is listed in the last column. As shown in Table 1, six international guidelines and four validation studies differed in the choice of validation parameters. This is mainly because the definition of some validation parameters from regulatory documents was controversial and different (Raposo & Ibelli-Bianco 2020). For example, the inter-assay and intra-assay (within run and between run) were used as validation parameters of precision in ANSI/ASB Standard 036, GCC and EMA. However, UNODC used reproducibility and repeatability, and PGIMV used reproducibility, repeatability and intermediate precision for the validation of precision. The validation parameters mentioned in regulatory documents were confused and complicated. Thus, the confusing terminologies in validation need to be clarified, which can simplify the validation processes. The differences among precision terms are based on the coefficient of variation (CV) of various runs. Diverse factors such as operators, reagents, days and equipment can be varied during runs. The precision contained reproducibility and repeatability (inter-assay and intra-assay), so we recommended validating the inter-assay and intra-assay, rather than repeatability, to avoid multiple definitions of precision. Furthermore, not all 20 validation parameters were validated in the international guidelines. It was greatly important for ELISA kit users to analyze the meaning of each validation parameter, ensuring that important parameters were validated.

Table 1

Summaries of ELISA kits validation parameters

ParameterUNODCANSI/ASB Standard 036GCCFDAEMAPGIMVStudies of ELISA kit validation
Frequency score
DiazinonAtrazine etc.Wheat/glutenPorcine C-peptide
Precision 10 
Specificity   
LOD     
Selectivity    
Recovery     
Accuracy      
Matrix effect      
Stability      
Standard curve      
Dilution linearity      
Repeatability        
Parallelism        
Robustness        
LOQ         
Quality control         
Reproducibility          
Assay drift          
Trueness          
Uncertainty          
ParameterUNODCANSI/ASB Standard 036GCCFDAEMAPGIMVStudies of ELISA kit validation
Frequency score
DiazinonAtrazine etc.Wheat/glutenPorcine C-peptide
Precision 10 
Specificity   
LOD     
Selectivity    
Recovery     
Accuracy      
Matrix effect      
Stability      
Standard curve      
Dilution linearity      
Repeatability        
Parallelism        
Robustness        
LOQ         
Quality control         
Reproducibility          
Assay drift          
Trueness          
Uncertainty          

Acronyms: UNODC, United Nations Office on Drugs and Crime (United Nations Office on Drugs & Crime 2009); ANSI/ASB Standard 036, American National Standard/Academy Standards Board Standard 036 (LeBeau 2020); GCC, Global CRO Council (Hougton et al. 2012); FDA (US Food and Drug Administration 2018); EMA, European Medicines Agency (2011); PGIMV, Practical guide to immunoassay method validation (Andreasson et al. 2015). Diazinon (Sullivan & Goh 2000); Atrazine etc. (Banks & Hernandez 2003); wheat/gluten (Saito et al. 2019); Porcine C-peptide (Graham et al. 2015).

As shown in Figure 1, the most frequently occurring parameter included was precision (100%), followed by specificity (80%), recovery (70%), LOD selectivity (60%), stability (60%) and recovery (60%); while the occurrent frequency of accuracy, matrix effect, standard curve and dilution linearity were 50%. The occurrence frequency of repeatability, parallelism, robustness, LLOQ, QC, sensitivity, reproducibility, assay drift, trueness, and uncertainty was less than 30%. It was considered that the validation parameters with higher frequency were more worthy of validation. Moreover, this study clarified the summarized validation parameters and proposed a simplified ELISA kit validation method.

Figure 1

Frequency of each validation parameter summarized from regulatory documents and related studies.

Figure 1

Frequency of each validation parameter summarized from regulatory documents and related studies.

Close modal

By one stringent definition, trueness can only be established through CRM, which is the closeness of agreement between the average values obtained from a large population of test results and an accepted reference value (ISO 1994). The reference value was derived from CRM or materials which could be traced to the CRM. The accuracy and QC were evaluated with CRM, and trueness could be viewed as one component of accuracy or QC in many researches. Trueness was rarely seen in the research, and almost all were replaced by accuracy or QC (Table 1). It can be seen in Table 1 that only PGIMV used trueness, while it does not evaluate accuracy or QC. Therefore, for the simplicity and convenience of ELISA validation, it was recommended to omit the parameter of trueness.

The definition of parallelism is the relative accuracy of recovery tests on the biological matrix or diluted matrix against the calibrators in a substitute matrix (Lee et al. 2006). According to EMA, PGIMV and GCC guidance, the parallelism was evaluated by serially diluted study samples. The parallelism was part of the dilution linearity, which was divided into spikes and real water samples dilution linearity.

Robustness, or ruggedness, is defined as the degree to which a measurement procedure or method is immune to variations induced by operational parameters including, but not restricted to, environmental factors, chemical parameters, electrical/site services, and human activity (Hou et al. 2011). Uncertainty can come from the measuring instrument, the item being measured, the ambient environment, the operators, and other sources (Bell 2001). In the study of ELISA kit validation, robustness and uncertainty were not a quantifiable term and should be substituted by assay drift, inter-assay variability and/or inter-laboratory proficiency testing. Robustness test normally is scheduled at the end of method development and therefore is not considered strictly as a performance parameter (Kadian et al. 2016). Considering the simplicity of ELISA kit validation, it was recommended to omit robustness and uncertainty.

Selectivity is defined as the ability of the bioanalytical method to measure and differentiate the analytes in the presence of components that may be expected to be present (US 2018). The terms ‘selectivity’ and ‘specificity’ were often used interchangeably, while their significance was different. Selectivity was something that could be graded, while specificity was an absolute characteristic. Specificity can be considered as the ultimate selectivity (Vessman et al. 2001). In addition, the FDA defined the sensitivity as the lowest nonzero standard on the calibration curve (US 2018). Thus, the sensitivity has the same definition as LLOQ, and can be determined by LLOQ in simplified ELISA kit validation.

Recovery is the recovered fraction of analyte added to a test sample before analysis (Buigues & Rey 2013). Although the definition of recovery was different from matrix effect, both of them were determined with standard addition method. Then, recovery has been validated and analyzed in the part of matrix effect.

This study sorted out the controversies and discrepancies of validation parameters. After clarifying and simplifying the logic behind validation parameters, 11 parameters were selected for ELISA kit validation, including standard curve, accuracy, precision, QC, matrix effect, specificity, stability, dilution linearity, recovery, assay drift, LOD and LLOQ.

Standard curve and quality control

The standard curve of MC-LR calibrators was fitted with four-parameter equations (Figure S1). The CV and back-calculated values of each calibrator were lower than 13% and within 89%–108%, respectively. The coefficient of determination (r2) of the MC-LR standard curve was higher than 0.99. Furthermore, EMA and FDA guidelines recommended that the back-fitted concentration of calibrators should be within 20% to pass acceptance criteria. The Environmental Protection Agency (EPA) suggested the r2 of the four-parameter curve must be greater than or equal to 0.98, and the CV of the duplicate well should be <15% (EPA 2016). The results showed that ELISA kit has a good performance in standard curve. The B/B0 value of 0.05 μg/L calibrator was 84.43% (B0 was the concentration of zero-analyte, B was the concentration of different MC-LR calibrators), indicating a significant difference between 0.05 μg/L calibrator and zero analyte concentration. Thus, the sensitivity of ELISA kit was at least 0.05 μg/L. The concentration for 50% of maximal effect (EC50) of the standard curve was 0.34 μg/L, which was consistent with the data provided by the manufacturer (0.40 μg/L). As mentioned above, results showed that the standard curve fitted by ELISA met the experimental requirements.

The concentration of positive control was within control limits (Figure 2). The CV of positive control was lower than 15%. Additionally, Microcystins were not expected to be present in adequately treated tap water, which was used as a negative control for Microcystins determination. The concentration of Microcystins in tap water samples was less than 0.008 μg/L, which was lower than LLOQ (3.4). Therefore, the authors believe that there were no Microcystins in treated tap water samples. The EMA guidance is that at least 67% QC samples should be within 20% of the nominal value, and GCC instruction suggested that any QC results should be within 3 SD. The results demonstrated that both positive and negative control samples were within the acceptance range.

Figure 2

Control chart for the Microcystin-LR positive control (0.75±0.185 μg/L).

Figure 2

Control chart for the Microcystin-LR positive control (0.75±0.185 μg/L).

Close modal

Accuracy and precision

The accuracy of the ELISA kit was tested with spiked samples, and proficiency water samples were obtained from Abraxis. The final range of MC-LR spiked samples was 0.15–3.00 μg/L, and the accuracy was within 70%–130% (Figure 3(a)). Compared with 1.25–3.00 μg/L spiked samples, 0.15–1.00 μg/L samples had higher accuracy and lower standard deviation. It was mainly because the concentrations of MC-LR spiked samples of 0.15–1.00 μg/L were near the EC50 of ELISA kit. As shown in Figure 3(b), the proficiency test of drinking water and surface water samples was within 70%–130%. In addition, the accuracy of drinking water was higher than that of surface water due to the matrix effect and interference exhibited in surface water samples (Tiscione & Kristin 2017). In summary, the accuracy of spiked and proficiency samples was within 70%–130%, indicating that the ELISA kit had excellent performance. It was recommended that the acceptance criteria of accuracy were within 70%–130% for ELISA kit according to ELISA kit validation documents (EPA 2016).

Figure 3

Accuracy of spiked samples (a), and drinking water and surface water proficiency (b). (The concentrations of MC-LR, MC-LA and MC-YR were 0.5 μg/L each in TK-1, 0.5 μg/L MC-LR in TK-3, and 1.5 μg/L MC-LR and 0.5 μg/L MC-YR in TK-4, respectively).

Figure 3

Accuracy of spiked samples (a), and drinking water and surface water proficiency (b). (The concentrations of MC-LR, MC-LA and MC-YR were 0.5 μg/L each in TK-1, 0.5 μg/L MC-LR in TK-3, and 1.5 μg/L MC-LR and 0.5 μg/L MC-YR in TK-4, respectively).

Close modal

The intra-assay, inter-assay and reproducibility of ELISA kit were estimated by determining the relative standard deviation (RSD) of calibrators and drinking water proficiency samples. The intra-assay and inter-assay were analyzed by calibrators, and the RSD of calibrators concentration ranged from 3% to 11% and from 6% to 15%, respectively (Tables S4 and S5). The Cochran test was used to compare the RSD values of different concentrations of calibrators. The results showed that the RSD and calibrators concentration of ELISA kit were uncorrelated. The reproducibility data of drinking water proficiency samples were obtained from 32 laboratories of 10 countries (Abraxis proficiency test). It was indicated that the reproducibility was ranging from 14% to 21% except for TK-2 (No MC-LR). The reproducibility was reasonably high because some laboratories failed to pass the certification of Abraxis. It was believed that the intra-assay and inter-assay precision of the mean concentration should be within 20%, and the total error should not exceed 30%. From all the above, the precision of the ELISA kit met the experimental requirements.

Matrix effect and dilution linearity

The spiked concentration, the mean of the concentration obtained using calibration curve, and recoveries calculated by the ratio of the spiked values and concentration given by ELISA measurements have been reported in Table 2. It suggested that recovery of the lake and reservoir water samples was within 70%–130% except for one of the samples at a concentration of 0.05 μg/L of lake A. The results indicated that the matrix effect was relatively minor in all tested samples except the sample of lake A. Lake A sample with strong matrix effect was collected from sewage pump pond and determined. A higher ionic strength than other samples was found in lake A sample, which might be the reason for the occurrence of matrix effect (Long et al. 2009).

Table 2

Matrix effect of surface water samples

Water sampleOriginal concentration (μg/L)Expected concentration (μg/L)Observed concentration (μg/L)Recovery (%)
reservoir A 0.07 0.79 0.68 86% 
0.54 1.02 0.93 88% 
0.61 1.06 1.10 106% 
0.88 1.19 1.16 96% 
1.00 1.25 1.25 100% 
1.43 1.47 1.65 125% 
2.70 2.10 2.19 112% 
reservoir B 0.02 0.77 0.78 103% 
0.43 0.97 1.08 115% 
0.50 1.00 1.09 112% 
1.16 1.33 1.52 125% 
1.99 1.74 1.52 70% 
2.62 2.06 1.99 91% 
Lake A 0.05 1.03 1.74 172% 
0.43 1.22 1.39 118% 
0.39 1.20 1.36 117% 
3.66 2.83 2.58 75% 
Water sampleOriginal concentration (μg/L)Expected concentration (μg/L)Observed concentration (μg/L)Recovery (%)
reservoir A 0.07 0.79 0.68 86% 
0.54 1.02 0.93 88% 
0.61 1.06 1.10 106% 
0.88 1.19 1.16 96% 
1.00 1.25 1.25 100% 
1.43 1.47 1.65 125% 
2.70 2.10 2.19 112% 
reservoir B 0.02 0.77 0.78 103% 
0.43 0.97 1.08 115% 
0.50 1.00 1.09 112% 
1.16 1.33 1.52 125% 
1.99 1.74 1.52 70% 
2.62 2.06 1.99 91% 
Lake A 0.05 1.03 1.74 172% 
0.43 1.22 1.39 118% 
0.39 1.20 1.36 117% 
3.66 2.83 2.58 75% 

The dilution linearity was determined with MC-LR CRM standard solution and real surface water samples. The MC-LR CRM standard solution was diluted 1,000, 5,000, 50,000 times, and the accuracy of back-calculated concentration of diluted samples was still within 80%. It was illustrated that the MC-LR CRM standard solution could be diluted by many-fold without the issue of dilution linearity. Furthermore, real surface water samples, which were selected to determine the dilution linearity, exceeded 5 μg/L, and they were diluted with LRB solution to the working range for further detection. The results showed that water samples were diluted by 15 times and 10 times, and the concentration of samples was overestimated by 5–10 times and 2–4 times, respectively. This was because the concentration of real surface water samples was plotted on a logarithmic scale. Thus, it is strongly recommended to dilute real water samples if the sample concentration is beyond the working range.

LOD, LLOQ and specificity

The LOD was estimated from the mean and standard deviation of the concentration of LRB solution based on the following formula. The average concentration of LRB solution was 0.002 μg/L, and the standard deviation was 0.003 μg/L. Then, the calculated LOD was 0.011 μg/L, which was sufficient for further analysis. The LLOQ was evaluated according to the definition. The lowest concentration of calibrator was 0.05 μg/L, which was higher than five times the analyte response of the LRB solution. In addition, the accuracy and precision of 0.05 μg/L calibrator were within ±20%, indicating that 0.05 μg/L satisfied the requirements of LLOQ.
(where Xb and Sb were the mean and standard deviation of the concentration of LRB solution).

The signal-to-noise ratio method, blank determination method and linear regression method were used to evaluate the quantitation limit in much of the literature (Long et al. 2009). However, users are confused when choosing the validation method. PGIMV recommended that the LLOQ was 10 SDs plus the mean of the blank sample, and EMA and FDA suggested that the LLOQ should be at least five times the concentration signal of blank sample. The study showed that the definition of PGIMV was not suitable for ELISA kit validation (Long & Winefordner 1983; Leal et al. 2012). The lasted LLOQ definition was proposed by EMA and FDA for ELISA method validation. Thus, it was recommended to use the EPA and FDA suggested method for LLOQ validation.

The specificity of D-phenylalanine, L-phenylalanine and DL-phenylalanine was estimated, which was less than 0.1%. Additionally, the cross-reactivity of Microcystins analogues was within the acceptable range of 70%–130% (Table S6). The specificity results were in good agreement with those provided by the manufacturer.

Stability of water samples

The stability of four proficiency drinking water samples was investigated over four months (Figure 4). The accuracy of proficiency water samples had an excellent stability in the first three months. At the fourth month, the concentration of TK-1 and TK-4 proficiency samples decreased significantly. The TK-1 and TK-4 were compound Microcystins samples, and MC-LR degraded considerably faster in the presence of other Microcystis extracts (Santos et al. 2021). Moreover, it was shown that the degradation rate of MC-YR was higher than that of MC-LA (Santos et al. 2021). These might be the reasons for the significantly decreased concentration of Microcystins in TK-1 and TK-4 samples in the fourth month. Therefore, it was suggested that the frozen Microcystins samples should be tested as soon as possible.

Figure 4

Stability of proficiency drinking water samples over four months.

Figure 4

Stability of proficiency drinking water samples over four months.

Close modal

Assay drift

Eight-channel pipette

Each channel of the eight-channel pipette dispensed 100 μL distilled water into a weigh boat repeatedly 24 times, and the weight changes were recorded by the analytical balance. The CV of the mass of distilled water added was used to evaluate the assay drift of the eight-channel pipette. The results indicated that the CV of each channel was less than 0.50%, and the CV of eight-channels was 0.66%.

ELISA reader

The color solution was prepared according to the instruction of JJG 861-2007 (Standardization Administration of People's Republic of China 2002). The color solution was diluted with distilled water until absorbance value was around 1.0, and then 100 μL color solution was dropped into each microplate well for the determination of the assay drift of ELISA reader. The average vertical CV of ELISA reader was 1.70%, suggesting that there was no significant light variation among the eight channels (n=8). Moreover, the average horizontal CV of ELISA reader was 4.20%, which was mainly owing to the different incubation time of the reactants (n=12). It was consistent with previous studies of Wild, which indicated that a possible cause of the assay drift was a slightly different incubation time as reagents were added sequentially over the entire plate (Wild & Sheehan 2013). In summary, the CV of ELISA reader was within the acceptable criteria range for further determination. Although the parameter of assay drift was only validated in the document of ANSI/ASB Standard 036, to avoid the systematic error, the conformation of assay drift was required before validation experiments.

This study summarized and clarified 20 parameters from related regulatory documents and studies. The confusion and discrepancy terminologies were investigated and clarified, and a simplified method for ELISA kit validation was proposed. Meanwhile, the validation parameters of standard curve, accuracy, precision, QC, matrix effects, specificity, stability, dilution linearity, assay drift, LOD and LLOQ were selected for simplifying the validation of ELISA kit. Furthermore, the proposed method was used to validate the Microcystins ELISA kit, and the acceptance range and validation details of selected validation parameters were investigated. The results indicated that the performance of Microcystins ELISA kit was satisfied with Microcystins detection in waters, and the assay drift was recommended to be validated to avoid systematic error before experiments. The suggested acceptance range of validation parameters was shown as follows.

  • (1)

    The r2 of the standard curve must be greater than or equal to 0.98, and the CV of the duplicate well should be <15%. Furthermore, the back-fitted concentration of calibrators should be within 20%.

  • (2)

    The accuracy of CRM spiked and proficiency water samples was within 70%–130%, and the concentration of QC samples should not exceed control limit.

  • (3)

    The precision of intra-assay and inter-assay of the ELISA kit was within 20%, and reproducibility variability was reasonably within 25%.

  • (4)

    The matrix effect, stability and specificity acceptance range were recommended within 70%–130%.

  • (5)

    The back-calculated concentration of each CRM dilution should not exceed 20%, and it was suggested that water samples beyond the working range should be diluted before determination.

The proposed Microcystins ELISA kit simplified validation method was validated as a case study commercial ELISA kit. This study only measured Microcystins in waters, which can be extended to medical clinical, pharmaceutical and food assays. In addition, the researcher could reasonably and clearly identify the parameters that need to be validated depending on the target to be measured, and then the performance of the ELISA kit could be analyzed based on the results of each parameter. By applying the simplified validation methods, significant efforts, expense and time would be saved.

This research was financially supported by Ministry of Science and Technology (2016YFC0400709), China; and TBNA Science and Technology Project (BHXQKJXM-PT-ZJSHJ-2017006), China.

The authors declare no competing interests.

All relevant data are included in the paper or its Supplementary Information.

Andreasson
U.
,
Perretliaudet
A.
,
Van Doorn
L. J. C. V. W.
,
Blennow
K.
,
Chiasserini
D.
,
Engelborghs
S.
,
Fladby
T.
,
Genc
S.
,
Kruse
N.
&
Kuiperij
H. B.
2015
A practical guide to immunoassay method validation
.
Frontiers in Neurology
6
(
19
),
179
.
https://doi.org/10.3389/fneur.2015.00179.
Balest
L.
,
Murgolo
S.
,
Sciancalepore
L.
,
Montemurro
P.
,
Abis
P. P.
,
Pastore
C.
&
Mascolo
G.
2016
Ultra-trace levels analysis of microcystins and nodularin in surface water by on-line solid-phase extraction with high-performance liquid chromatography tandem mass spectrometry
.
Analytical and Bioanalytical Chemistry
408
(
15
),
4063
4071
.
https://doi.org/10.1007/s00216-016-9495-y
.
Bell
S.
2001
Good Practice Guide No. 11 - Introductory Guide to Uncertainty of Measurement. Measurement Good Practice Guide, (2), 41
.
Bièvre
P. D.
2008
Measurement uncertainty is not synonym of measurement repeatability or measurement reproducibility
.
Accreditation & Quality Assurance
13
(
2
),
61
62
.
https://doi.org/10.1007/s00769-008-0371-0
.
Brodie
R. R.
&
Hill
H. M.
2002
Validation issues arising from the new FDA guidance for industry on bioanalytical method validation
.
Chromatographia
55
(
1
),
91
94
.
https://doi.org/10.1007/BF02493361
.
Buigues
P.
&
Rey
P.
2013
Commission Decision. The Economics of Antitrust and Regulation in Telecommunications, (C), 1–27
.
Elfving
J.
2014
In-House Validation of Capillary Electrophoresis Method. Lutpub
.
EPA
2016
Method 546: Determination of total microcystins and nodularins in drinking water and ambient water by Adda enzyme-linked immunosorbent assay. United States Environmental Protection Agency, (815-B-16–011), 21. Available from: https://www.epa.gov/sites/production/files/2016-09/documents/method-546-determination-total-microcystins-nodularins-drinking-water-ambient-water-adda-enzyme-linked-immunosorbent-assay.pdf.
European Medicines Agency
2011
Guidelines on bioanalytical method validation by the European Medicines Agency Committee for Medicinal Products for Human Use, 21 July 2011, EMEA/CHMP/EWP/192217/2009 Rev. 1 Corr. 2
.
Foss
A. J.
,
Miles
C. O.
,
Wilkins
A. L.
,
Rise
F.
,
Trovik
K. W.
,
Cieslik
K.
&
Aubel
M. T.
2020
Analysis of total microcystins and nodularins by oxidative cleavage of their ADMAdda, DMAdda, and Adda moieties
.
Analytica Chimica Acta: X
6
,
100060
.
https://doi.org/10.1016/j.acax.2020.100060
.
Graham
M. L.
,
Gresch
S. C.
,
Hardy
S. K.
,
Mutch
L. A.
,
Janecek
J. L.
&
Hegstaddavies
R. L.
2015
Evaluation of commercial ELISA and RIA for measuring porcine C-peptide: implications for research
.
Xenotransplantation
22
(
1
),
62
69
.
https://doi.org/10.1111/xen.12143.
Geis-Asteggiante
L.
,
Lehotay
S. J.
,
Fortis
L. L.
,
Paoli
G.
,
Wijey
C.
&
Heinzen
H.
2011
Development and validation of a rapid method for microcystins in fish and comparing LC-MS/MS results with ELISA
.
Analytical and Bioanalytical Chemistry
401
(
8
),
2617
2630
.
https://doi.org/10.1007/s00216-011-5345-0
.
He
X.
,
Stanford
B. D.
,
Adams
C.
,
Rosenfeldt
E. J.
&
Wert
E. C.
2017
Varied influence of microcystin structural difference on ELISA cross-reactivity and chlorination efficiency of congener mixtures
.
Water Research
126
,
515
523
.
https://doi.org/10.1016/j.watres.2017.09.037
.
Hou
J. J.
,
Wu
W. Y.
,
Da
J.
,
Yao
S.
,
Long
H. L.
,
Yang
Z.
,
Cai
L. Y.
,
Yang
M.
,
Liu
X.
,
Jiang
B. H.
&
Guo
D. A.
2011
Ruggedness and robustness of conversion factors in method of simultaneous determination of multi-components with single reference standard
.
Journal of Chromatography A
1218
(
33
),
5618
5627
.
https://doi.org/10.1016/j.chroma.2011.06.058
.
Hougton
R.
,
Gouty
D.
,
Allinson
J.
,
Green
R.
,
Losauro
M.
,
Lowes
S.
,
Lelacheur
R.
,
Garofolo
F.
,
Couerbe
P.
&
Bronner
S.
2012
Recommendations on biomarker bioanalytical method validation by GCC
.
Bioanalysis
4
(
20
),
2439
2446
.
https://doi.org/10.4155/bio.12.197.
ISO
1994
ISO 5725-1 Accuracy (Trueness and Precision) of Measurement Methods and Results
.
ISO
.
Kadian
N.
,
Raju
K. S. R.
,
Rashid
M.
,
Malik
M. Y.
,
Taneja
I.
&
Wahajuddin
M.
2016
Comparative assessment of bioanalytical method validation guidelines for pharmaceutical industry
.
Journal of Pharmaceutical and Biomedical Analysis
126
,
83
97
.
https://doi.org/10.1016/j.jpba.2016.03.052
.
Kohoutek
J.
,
Procházková
T.
,
Adamovský
O.
,
Palíková
M.
&
Hilscherová
K.
2019
Stable-isotope dilution LC-MS/MS method for quantitative determination of microcystin conjugates with cysteine and glutathione in biotic matrices
.
Analytical and Bioanalytical Chemistry
411
(
20
),
5267
5275
.
https://doi.org/10.1007/s00216-019-01904-0
.
Leal
C. R.
,
Augusto
D. F.
,
Fabiane
D.
,
Stella
B.
,
Nathalie
D.
,
Mónica
V.
&
Ernani
P.
2012
Co-occurrence of microcystin and microginin congeners in Brazilian strains of Microcystis sp
.
FEMS Microbiology Ecology
(
3
),
692
702
.
https://doi.org/10.1111/j.1574-6941.2012.01439.x
.
LeBeau
M. A.
2020
ANSI/ASB standard 036 for method validation in forensic toxicology has replaced SWGTOX's version
.
Journal of Analytical Toxicology
44
(
4
),
414
.
https://doi.org/10.1093/jat/bkz115
.
Lee
J. W.
,
Devanarayan
V.
,
Barrett
Y. C.
,
Weiner
R.
,
Allinson
J.
,
Fountain
S.
,
Keller
S.
,
Weinryb
I.
,
Green
M.
&
Duan
L.
2006
Fit-for-Purpose method development and validation for successful biomarker measurement
.
Pharmaceutical Research
23
(
2
),
312
328
.
https://doi.org/10.1007/s11095-005-9045-3
.
Li
C.-M.
,
Chu
R. Y.-Y.
&
Hsientang Hsieh
D. P.
2006
An enhanced LC-MS/MS method for microcystin-LR in lake water
.
Journal of Mass Spectrometry
41
(
2
),
169
174
.
https://doi.org/10.1002/jms.972
.
Long
G. L.
&
Winefordner
J. D.
1983
Limit of detection. A closer look at the IUPAC definition
.
Analytical Chemistry
55
(
7
),
712A
724A
.
Long
F.
,
Zhu
A. N.
,
Sheng
J. W.
,
He
M.
&
Shi
H. C.
2009
Matrix effects on the microcystin-LR fluorescent immunoassay based on optical biosensor
.
Sensors
9
(
4
),
3000
3010
.
https://doi.org/10.3390/s90403000
.
Lu
N.
,
Ling
L.
,
Guan
T.
,
Wang
L.
,
Wang
D.
,
Zhou
J.
,
Ruan
T.
,
Shen
X.
,
Li
X.
,
Sun
Y.
&
Lei
H.
2020
Broad-specificity ELISA with a heterogeneous strategy for sensitive detection of microcystins and nodularin
.
Toxicon
175
,
44
48
.
https://doi.org/10.1016/j.toxicon.2019.12.003
.
Mash
H.
&
Wittkorn
A.
2016
Effect of chlorination on the protein phosphatase inhibition activity for several microcystins
.
Water Research
95
,
230
239
.
https://doi.org/10.1016/j.watres.2016.03.024
.
Nadovich
C. T.
2005
Calibration and accuracy
. In:
Synthetic Instruments
, C. T. Nadovich, Newnes,
137
156
.
Rana
A. Y. K. M. M.
,
Prince
M. M. B.
,
Chakma
D.
,
Islam
H.
,
Nabi
M.
&
Saifullah
A. S. M.
2020
Validation of a commercial ELISA kit for screening 3-amino-2-oxazolidinone, a furazolidone antibiotic residue in shrimp
.
Annual Research & Review in Biology
,
1
10
.
https://doi.org/10.9734/arrb/2019/v34i230150
.
Raposo
F.
&
Ibelli-Bianco
C.
2020
Performance parameters for analytical method validation: controversies and discrepancies among numerous guidelines
.
TrAC Trends in Analytical Chemistry
129
,
115913
.
https://doi.org/10.1016/j.trac.2020.115913
.
Reeve
J.
&
Peerbhoy
D.
2007
Evaluating the evaluation: understanding the utility and limitations of evaluation as a tool for organizational learning
.
Health Education Journal
66
(
2
),
120
131
.
https://doi.org/10.1177/0017896907076750
.
Rivasseau
C.
,
Racaud
P.
,
Deguin
A.
&
Hennion
M. C.
1999
Evaluation of an ELISA kit for the monitoring of microcystins (cyanobacterial toxins) in water and algae environmental samples
.
Environmental Science and Technology
33
(
9
),
1520
1527
.
https://doi.org/10.1021/es980460 g
.
Rudaz
S.
&
Feinberg
M.
2018
From method validation to result assessment: established facts and pending questions
.
TrAC Trends in Analytical Chemistry
105
,
68
74
.
https://doi.org/10.1016/j.trac.2018.04.013
.
Saito
E.
,
Doi
H.
,
Kurihara
K.
,
Kato
K.
,
Aburatani
K.
,
Shoji
M.
,
Naka
Y.
,
Koerner
T.
,
Poepping
B.
&
Boison
J.
2019
The validation of the wheat gluten ELISA Kit
.
Journal of AOAC International
102
(
4
),
1162
1173
.
https://doi.org/10.5740/jaoacint.19-0005.
Santos
A. A.
,
Soldatou
S.
,
de Magalhães
V. F.
,
Azevedo
S. M. F. O.
,
Camacho-Muñoz
D.
,
Lawton
L. A.
&
Edwards
C.
2021
Degradation of multiple peptides by Microcystin-Degrader Paucibacter toxinivorans (2C20)
.
Toxins
13
(
4
).
https://doi.org/10.3390/toxins13040265
Standardization Administration of People's Republic of China 2002 Standard Practice for Applying Statistical Techniques to Evaluate Analytical Measurement System Performance (GB/T27407–2010/ASTM D6299:2002).
Sullivan
J. J.
&
Goh
K. S.
2000
Evaluation and validation of a commercial ELISA for diazinon in surface waters
.
Journal of Agricultural & Food Chemistry
48
(
9
),
4071
4078
.
https://doi.org/10.1021/jf000432t.
Teodoro
T. A.
,
Reboita
M. S.
,
Llopart
M.
,
da Rocha
R. P.
&
Ashfaq
M.
2021
Climate change impacts on the South American monsoon system and its surface–atmosphere processes through RegCM4 CORDEX-CORE projections
.
Earth Systems and Environment
5
(
4
),
825
847
.
https://doi.org/10.1007/s41748-021-00265-y
.
Tian
J.
,
Hu
Y.
&
Zhang
J.
2008
Chemiluminescence detection of permanganate index (CODMn)bya luminol-KMnO4 based reaction
.
Journal of Environmental Sciences
20
(
2
),
252
256
.
https://doi.org/10.1016/S1001-0742(08)60039-X
.
Tiscione
N. B.
&
Kristin
W.
2017
Validation of the neogen fentanyl ELISA Kit for blood and urine
.
Journal of Analytical Toxicology
44
(
4
),
313
317
.
https://doi.org/10.1093/jat/bkz115
.
Trifirò
G.
,
Barbaro
E.
,
Gambaro
A.
,
Vita
V.
,
Clausi
M. T.
,
Franchino
C.
,
Palumbo
M. P.
,
Floridi
F.
&
De Pace
R.
2016
Quantitative determination by screening ELISA and HPLC-MS/MS of microcystins LR, LY, LA, YR, RR, LF, LW, and nodularin in the water of Occhito lake and crops
.
Analytical and Bioanalytical Chemistry
408
(
27
),
7699
7708
.
https://doi.org/10.1007/s00216-016-9867-3
.
United Nations Office on Drugs and Crime
2009
A Commitment to Quality and Continuous Improvement
.
US
Food and Drug Administration 2018 Bioanalytical method validation guidance for industry
.
Food and Drug Administration
,
1
41
.
Valipour
M.
,
Bateni
S. M.
&
Jun
C.
2021
Global surface temperature: a new insight
.
Climate
9
(
5
),
81
.
https://doi.org/10.3390/cli9050081
.
Vessman
J.
,
Stefan
R. I.
,
Van Staden
J. F.
,
Danzer
K.
,
Lindner
W.
,
Burns
D. T.
,
Fajgelj
A.
&
Muller
H.
2001
Selectivity in analytical chemistry (IUPAC recommendations 2001)
.
Pure and Applied Chemistry
73
(
8
),
1381
1386
.
https://doi.org/10.1351/pac200173081381
.
Wild
D.
&
Sheehan
C.
,
2013
The Immunoassay Handbook || Principles of Competitive and Immunometric Assays (Including ELISA) 1. 29–59
.
Yang
X.
,
Xie
P.
,
Ma
Z.
,
Wang
Q.
,
Fan
H.
&
Shen
H.
2013
Decrease of NH4+-N by bacterioplankton accelerated the removal of cyanobacterial blooms in aerated aquatic ecosystem
.
Journal of Environmental Sciences
25
(
11
),
2223
2228
.
https://doi.org/10.1016/S1001-0742(12)60282-4
.
Zhong
X.
,
Zhang
H. Y.
,
Tan
H.
,
Zhou
Y.
,
Liu
F. L.
,
Chen
F. Q.
&
Shang
D. Y.
2011
Association of serum omentin-1 levels with coronary artery disease
.
Acta Pharmacologica Sinica
32
(
7
),
873
878
.
https://doi.org/10.1038/aps.2011.26
.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits copying, adaptation and redistribution, provided the original work is properly cited (http://creativecommons.org/licenses/by/4.0/).

Supplementary data