Abstract

Inter-laboratory reproducibility of biomethane potential (BMP) is dismal, with differences in BMP values for the same sample exceeding a factor of two in some cases. A large group of BMP researchers directly addressed this problem during a workshop held in Leysin, Switzerland, in June 2015. The workshop resulted in a new set of guidelines for BMP tests published in 2016, which is the subject of the present commentary. The work has continued with two international inter-laboratory studies and one additional workshop held in Freising, Germany, in 2018. The dataset generated by the two inter-laboratory studies were used to refine the validation criteria for BMP tests. Based on these new results an update to the original guidelines is proposed here.

HIGHLIGHTS

  • This commentary updates requirements and validation criteria defined previously in a Water Science and Technology paper published in 2016.

Graphical Abstract

Graphical Abstract
Graphical Abstract

Multiple national and international inter-laboratory studies show rather low reproducibility in measurement of biochemical methane potential (BMP), with differences for the same substrate exceeding a factor of two in some cases (Raposo et al. 2011; Ribeiro et al. 2020; Weinrich et al. 2018) despite the existence of well-known protocols (Angelidaki et al. 2009; VDI 4630). To help address this problem that affects anaerobic digestion (AD) research and industry, an international group of researchers met in Leysin, Switzerland, in June 2015 to discuss the standardization of BMP tests. The outcome of this workshop was the publication of a new set of BMP test guidelines where compulsory elements for the validation of BMP results were defined and recommendations on items that strongly influence the outcome of BMP tests, such as inoculum characteristics, substrate preparation, test setup, and data analysis, were presented (Holliger et al. 2016). The present commentary summarizes progress that has been made since the publication of this paper. Briefly, results from an additional workshop and two inter-laboratory studies led to the development of new validation criteria designed to improve BMP results, as described in detail in Hafner et al. (2020b). Here we propose that these validation criteria replace those given in Holliger et al. (2016), while the other requirements stand or are slightly modified. This combined set of requirements should be applied to all BMP tests from now on.

To assess the guidelines published in 2016 (Holliger et al. 2016), a first international inter-laboratory study (IIS) was conducted in 2016 and 2017 with 32 participating laboratories. Application of the 2016 validation criteria improved inter-laboratory reproducibility but only 16 of 62 BMP tests could be validated with these criteria. In order to continue the standardization and finally make BMP tests more reliable and reproducible, a second workshop was organized and a second IIS carried out. The workshop took place in April 2018 in Freising, Germany, and the second IIS concluded at the end of the same year. The two IIS generated a detailed dataset of BMP measurements made over a wide geographic range using several measurement methods (automated or manual volumetric, manual manometric, gravimetric, and direct GC). This dataset was used to quantify and identify sources of variability in BMP, and to test and refine validation criteria, and is publicly available as supplementary material of Hafner et al. (2020b). Application of the new criteria to this large dataset showed that they can substantially reduce variability in BMP measurements and validate a higher percentage of the BMP test results. Whereas the relative standard deviations among laboratories (RSDR, reproducibility) were below 8% and the relative ranges (RR) below 25% for all substrates and tests with the new validation criteria, RSDR and RR were as high as 14 and 55%, respectively, with the original validation criteria defined in the guidelines of 2016. In addition, on average 73% of the test results were rejected using the original criteria, compared to 55% with the refined criteria. We therefore recommend applying the new validation criteria listed in Table 1 in all BMP tests. Most of the other requirements were changed slightly or not at all (Table 1).

Table 1

Original and refined requirements and validation criteria for BMP measurements

Requirement/validation criterionOriginalaRefinedb
Replicates At least in triplicate Same as original 
Assays besides test substrate Blanks and positive control (unspecified) Blanks and microcrystalline cellulose positive control 
Duration of BMP test Terminate BMP test when daily methane production during three consecutive days is <1% of the accumulated volume of methane Same as original but with clarification that methane production is net (inoculum contribution subtracted) 
Expression of BMP BMP is expressed as the volume of dry methane gas under standard conditions (273.15 K and 101.33 kPa) per mass of volatile solids (VS) added, with the unit NLCH4 kgvs−1 Same as original 
Calculation of BMP BMP is determined by subtracting the mean methane production of the blanks from the gross methane production of the substrate/positive control assays Same as original but calculations now described in detail in Hafner et al. (2020a)  
Calculation of standard deviation of BMP For the calculation of the standard deviation of the BMP of the substrate and the positive control, the standard deviation (SD) of the blanks must be taken into account Same as original but with addition of SD of VS measurement and clarification of calculations (Hafner et al. 2020a
Validation criterion 1 Only if the RSD of the blank or the positive control is <5%, BMP results can be validated. It is possible to eliminate a single outlier after applying a statistical test Only if the RSD of cellulose is <6%, BMP results can be validated. Rejection based on too high RSD of blanks is not recommended and elimination of outliers is only allowed if triplicates are still available at the end 
Validation criterion 2 Only if the RSD of a homogeneous or heterogeneous substrate is <5% and <10%, respectively, BMP results can be validated. It is possible to eliminate a single outlier after applying a statistical test No validation criterion recommended for substrate BMP, but RSD has to be provided for each substrate tested 
Validation criterion 3 Only if the BMP of the positive control is between 85 and 100% of the theoretical BMP (e.g. for cellulose: between 352 NLCH4 kgvs−1 and 414 NLCH4 kgvs−1), BMP results can be validated Only if the BMP of microcrystalline cellulose is between 340 NLCH4 kgvs−1 and 395 NLCH4 kgvs−1, BMP results can be validated 
Requirement/validation criterionOriginalaRefinedb
Replicates At least in triplicate Same as original 
Assays besides test substrate Blanks and positive control (unspecified) Blanks and microcrystalline cellulose positive control 
Duration of BMP test Terminate BMP test when daily methane production during three consecutive days is <1% of the accumulated volume of methane Same as original but with clarification that methane production is net (inoculum contribution subtracted) 
Expression of BMP BMP is expressed as the volume of dry methane gas under standard conditions (273.15 K and 101.33 kPa) per mass of volatile solids (VS) added, with the unit NLCH4 kgvs−1 Same as original 
Calculation of BMP BMP is determined by subtracting the mean methane production of the blanks from the gross methane production of the substrate/positive control assays Same as original but calculations now described in detail in Hafner et al. (2020a)  
Calculation of standard deviation of BMP For the calculation of the standard deviation of the BMP of the substrate and the positive control, the standard deviation (SD) of the blanks must be taken into account Same as original but with addition of SD of VS measurement and clarification of calculations (Hafner et al. 2020a
Validation criterion 1 Only if the RSD of the blank or the positive control is <5%, BMP results can be validated. It is possible to eliminate a single outlier after applying a statistical test Only if the RSD of cellulose is <6%, BMP results can be validated. Rejection based on too high RSD of blanks is not recommended and elimination of outliers is only allowed if triplicates are still available at the end 
Validation criterion 2 Only if the RSD of a homogeneous or heterogeneous substrate is <5% and <10%, respectively, BMP results can be validated. It is possible to eliminate a single outlier after applying a statistical test No validation criterion recommended for substrate BMP, but RSD has to be provided for each substrate tested 
Validation criterion 3 Only if the BMP of the positive control is between 85 and 100% of the theoretical BMP (e.g. for cellulose: between 352 NLCH4 kgvs−1 and 414 NLCH4 kgvs−1), BMP results can be validated Only if the BMP of microcrystalline cellulose is between 340 NLCH4 kgvs−1 and 395 NLCH4 kgvs−1, BMP results can be validated 

By analyzing the large dataset, systematic biases were identified that were associated with both laboratories and tests within laboratories (Hafner et al. 2020b). Interestingly, errors in data processing or data entry were important sources of variability and there was evidence of negative biases in manual manometric and manual volumetric measurement methods. However, much of the observed variation in BMP values was not clearly related to any of these factors, and is probably the result of particular method details that vary among laboratories. Encouragingly, results show that it is possible to measure very similar BMP values with different measurement methods. Hence, in order to meet the validation criteria (Table 1), it is important that laboratories whose tests are rejected are willing to evaluate and make changes to their BMP methods.

To contribute to better standardization of BMP measurements, a new website was established to share detailed, up-to-date guidance on BMP measurement and data processing: https://www.dbfz.de/en/BMP. Users of this ‘Standard BMP Methods’ website can find the latest requirements and validation criteria described in detail (Holliger et al. 2020), documentation of calculations (e.g. Hafner et al. 2020a), links to standardized software for data processing (Hafner et al. 2018), and other guidance. We invite anyone interested in BMP tests to make use of the resource and to actively contribute to this ongoing project.

ACKNOWLEDGEMENTS

We are grateful to the continuous support by the Swiss Federal Office for Energy that financed the management part of both the inter-laboratory studies as well as participation in the AD15 and AD16 conferences. The financial support of the TUM Global Incentive Fund to host the second international workshop on standardization of BMP tests in Freising, Germany, is highly appreciated.

DATA AVAILABILITY STATEMENT

All relevant data are included in the paper or its Supplementary Information.

REFERENCES

REFERENCES
Angelidaki
I.
Alves
M.
Bolzonella
D.
Borzacconi
L.
Campos
J. L.
Guwy
A. J.
Kalyuzhnyi
S.
Jemicek
P.
van Lier
J. B.
2009
Defining the biomethane potential (BMP) of solid organic wastes and energy crops: a proposed protocol for batch assays
.
Water Science and Technology
59
(
5
),
927
934
.
Hafner
S.
Koch
K.
Carrere
H.
Astals
S.
Weinrich
S.
Rennuit
C.
2018
Software for biogas research: tools for measuring and prediction of methane production
.
Software X
7
,
205
210
.
Hafner
S. D.
Astals
S.
Holliger
C.
Koch
K.
Weinrich
S.
2020a
Calculation of Biochemical Methane Potential (BMP)
.
Standard BMP Methods document 200, version 1.6
.
Available online: https://www.dbfz.de/en/BMP (accessed on April 19, 2020)
.
Holliger
C.
Alves
M.
Andrade
D.
Angelidaki
I.
Astals
S.
Baier
U.
Bougrier
C.
Buffiere
P.
Carballa
M.
de Wilde
V.
Ebertseder
F.
Fernandez
B.
Ficara
E.
Fotidis
I.
Frigon
J. C.
Fruteau de Laclos
H.
Ghasimi
D. S. M.
Hack
G.
Hartel
M.
Heerenklage
J.
Horvath
I. S.
Jenicek
P.
Koch
K.
Krautwald
J.
Lizasoain
J.
Liu
j.
Mosberger
L.
Nistor
M.
Oechsner
H.
Oliveira
J. V.
Paterson
M.
Pauss
A.
Pommier
S.
Porqueddu
I.
Raposo
F.
Ribeiro
T.
Rusch Pfund
F.
Stromberg
S.
Torrijos
M.
van Eekert
M.
van Lier
J.
Wedwitschka
H.
Wierinck
I.
2016
Towards a standardization of biomethane potential tests
.
Water Science and Technology
74
(
11
),
2515
2522
.
Holliger
C.
Fruteau de Laclos
H.
Hafner
S. D.
Koch
K.
Weinrich
S.
Astals
S.
Alves
M.
Andrade
D.
Angelidaki
I.
Appels
L.
Astals
S.
Azman
S.
Bagnoud
A.
Baier
U.
Bajon Fernandez
Y.
Bartacek
J.
Battista
F.
Bolzonella
D.
Bougrier
C.
Braguglia
C.
Buffière
P.
Carballa
M.
Catenacci
A.
Dandikas
V.
de Wilde
F.
Ekwe
S.
Ficara
E.
Fotidis
I.
Frigon
J.-C.
Gallipoli
A.
Heerenklage
J.
Jenicek
P.
Krautwald
J.
Lindeboom
R.
Liu
J.
Lizasoain
J.
Marchetti
R.
Monlau
F.
Nistor
M.
Oechsner
H.
Oliveira
J. V.
Pauss
A.
Pommier
S.
Raposo
F.
Ribeiro
T.
Schaum
C.
Schuman
E.
Schwede
S.
Soldano
M.
Taboada
A.
Torrijos
M.
van Eekert
M.
van Lier
J.
Wierinck
I.
2020
Requirements for Measurement and Validation of Biochemical Methane Potential (BMP)
.
Standard BMP Methods document 100, version 1.7
.
Available online: https://www.dbfz.de/en/BMP (accessed on November 6, 2020)
Raposo
F.
Fernández-Cegrí
V.
De la Rubia
M. A.
Borja
R.
Béline
F.
Cavinato
C.
Demirer
G.
Fernández
B.
Fernández-Polanco
M.
Frigon
J. C.
Ganesh
R.
Kaparaju
P.
Koubova
J.
Méndez
R.
Menin
G.
Peene
A.
Scherer
P.
Torrijos
M.
Uellendahl
H.
Wierinck
I.
de Wilde
V.
2011
Biochemical methane potential (BMP) of solid organic substrates: evaluation of anaerobic biodegradability using data from an international interlaboratory study
.
Journal of Chemical Technology & Biotechnology
86
(
8
),
1088
1098
.
Ribeiro
T.
Cresson
R.
Pommier
S.
Preys
S.
André
L.
Béline
F.
Bouchez
T.
Bougrier
C.
Buffière
P.
Cacho
J.
Camacho
P.
Mazéas
L.
Pauss
A.
Pouech
P.
Rouez
M.
Torrijos
M.
2020
Measurement of biochemical methane potential of heterogeneous solid substrates: results of a two-phase French inter-laboratory study
.
Water
12
,
2814
.
VDI 4630
.
2016
Fermentation of Organic Materials – Characterization of the Substrate, Sampling, Collection of Material Data, Fermentation Tests
.
VDI- Handbuch Energietechnik
.
Weinrich
S.
Schäfer
F.
Bochmann
G.
Liebetrau
J.
2018
Value of batch tests for biogas potential analysis; method comparison and challenges of substrate and efficiency evaluation of biogas plants. Murphy, J. D. (ed.), IEA Bioenergy Task 37, Report 10. http://task37.ieabioenergy.com/files/daten-redaktion/download/Technical%20Brochures/Batch_tests_web_END.pdf.