Introducing the bgfanalyzer Package

Introduction

The package adds the concept of microbial fermentation data to the R programming language. It was developed for BioGas Fermentations, but concepts and workflows might also be applied to other types fermentations.

The central concept of this package is a novel S3 class object, the BGF which allows to store and visualize experimental data, and also supports experiment design.

This vignette at first provides some background on what biogas fermentations are, as this might be a rather exotic topic to most R users. Then, the BGF-concept will be announced and some easy methods how to set up a BGF are shown.

However, this vignette focuses on the basics and other vignettes provide more detailed explanations for distinct BGF-types and experimental workflow.

Background - A Microbial Biogas Fermentation

In the sense of the BGFanalyzer package, a biogas fermentation is the anaerobic breakdown of organic material by microorganisms into gaseous end products.

The most industrialized version of a biogas fermentation might be bio methane production, also known as anaerobic digestion. During this process, several microorganisms from the bacterial and archaeal domain of life break down sugars, proteins and fatty acids to terminally form methane (CH4) out of it.

Despite methane, biological fermentations can also yield in the formation of other (valuable) gases, such as hydrogen (H2) or CO2. Although only CH4 is commonly referred to as ‘biogas’, all microbial mediated processes that lead to a target gas formation are considered as ‘biogas’ fermentation in the sense of the BGFanalyzer package.

The BGF Concept

The heart of the BGFanalyzer package is the new BGF object class, which allows to store experimental data of several fermentations. The package is independent of the ‘experimental hardware’ and can be used to build BGF’s from external text files or interactively in R.

BGF structure

A BGF is an S3 data object with a list like structure. Each BGF has three standard layers. These are:

The decision to make the BGF an S3 type object was based on two considerations.

First, S3 is the most primitive object-orientated programming system in R and most of the objects and functions R beginners encounter are of type S3. We wanted to make working with a BGF’s as easy as possible for everyone and thus the S3 system seemed like the right choice as it’s familiar to R beginners and experts as well.

Second, the S3 system offers a high degree of flexibility, so every user is free to add new layers to the BGF as needed. One could imagine to add a fourth layer with e.g. sequencing results of microbial community of individual fermentations. However, the focus of the bgfanalyzer package is the analysis biogas fermentation data and the general advice is to not change the three-layer architecture of a BGF.

Building a BGF

The simplest way for an R user to create a BGF is the BGF() function. It requires only the ReactorLayout argument to be set and will then create a BGF.

library(bgfanalyzer)

# create an example BGF with five fermentations having the reactor layout "A", "B", "C", "D", "E"
myBGF <- BGF(ReactorLayout = LETTERS[1:5])

# inspect the BGF
myBGF
#> 'new_BGF' - a BGF with 5 fermentation(s)
#> 
#> 
#> $ExpParam: 4 experimental paramerters
#> $metaData: 3 meta variables
#> $BioGasData: 5 observations of 7 fermentation variables
#> 
#> 
#> A '$yield' is not calculated yet

When a BGF is printed to the R console, a textual summary is shown. In the first line, the name of the BGF and the number of fermentations within is displayed. For the above example, the name is ‘new_BGF’ and consists of 5 fermentations.

The name of a BGF can be set during its creation with BGF() and other helper functions:

# create a named BGF by specifying the 'name' argument in BGF()
namedBGF <- BGF(ReactorLayout = paste0("2*",LETTERS[1:5]),
                name = "named BGF")

# inspect namedBGF
namedBGF
#> 'named BGF' - a BGF with 10 fermentation(s)
#> 
#> 
#> $ExpParam: 4 experimental paramerters
#> $metaData: 3 meta variables
#> $BioGasData: 10 observations of 7 fermentation variables
#> 
#> 
#> A '$yield' is not calculated yet

The name of the new BGF is ‘named BGF’ and it contains 10 fermentations, instead. Note, that a shortcut for setting multiple identical reactor layouts was used during object creation of ‘namedBGF’ (type ?correct_RLayout in the R console for details).

Alternatively, a new name can be set using the function alter_whatever() or base R syntax:

# rename myBGF using alter_whatever()
myBGF <- alter_whatever(myBGF,
                        layer = "ExpParam",
                        what = "name",
                        value = "new name")

# rename namedBGF using base syntax
namedBGF$ExpParam$name <- "new name"

# do myBGF and namedBGF have the same name now?
myBGF$ExpParam$name == namedBGF$ExpParam$name
#> [1] TRUE

# inspect renaming results
get_whatever(myBGF,
             layer = "ExpParam",
             what = "name")
#> [1] "new name"

# or
namedBGF$ExpParam$name
#> [1] "new name"

Following the first line with name and number of fermentations of a BGF, comes a block providing the number of variables in each layer of a BGF. In case of ‘myBGF’ and ‘namedBGF’, both have only the default variables set in each layer. This means, the ExpParam-layer has four ‘experimental parameters’ (ExpParam$name,ExpParam$InocToSubRatio,ExpParam$ProcessTemp,ExpParam$MeasurementType), the metaData-layer has three ‘meta variables’ (metaData$Layout, metaData$Blank, metaData$Excluded) and the BioGasData-layer has seven ‘fermentation variables’ (BioGasData$reactor, BioGasData$time, BioGasData$product, BioGasData$production, BioGasData$net_product, BioGasData$yield, BioGasData$rel_production).

At this point the number of rows in the metaData- and BioGasData-layer are identical, but all columns in the BioGasData-layer contain only NA values.

Let’s take a closer look whats already inside our BGF objects:


# list 'ExpParam'-layers of both BGF's
# myBGF
myBGF$ExpParam
#> $name
#> [1] "new name"
#> 
#> $InocToSubRatio
#> [1] 2
#> 
#> $ProcessTemp
#> [1] NA
#> 
#> $MeasurementType
#> [1] NA

# namedBGF
namedBGF$ExpParam
#> $name
#> [1] "new name"
#> 
#> $InocToSubRatio
#> [1] 2
#> 
#> $ProcessTemp
#> [1] NA
#> 
#> $MeasurementType
#> [1] NA

The ExpParam-layers of both BGF‘s are identical, as we have not specified BioGasData$InocToSubRatio, BioGasData$ProcessTemp or BioGasData$MeasurementType manually during object creation, and renamed both objects to ’new name’ afterwards. Thus all parameters despite the name are on their default value.

Now we do the same for the metaData-layer:

# list 'metaData'-layers 
# myBGF
myBGF$metaData
#>    Layout Blank Excluded
#> R1      A FALSE    FALSE
#> R2      B FALSE    FALSE
#> R3      C FALSE    FALSE
#> R4      D FALSE    FALSE
#> R5      E FALSE    FALSE

# namedBGF
namedBGF$metaData
#>     Layout Blank Excluded
#> R1       A FALSE    FALSE
#> R2       A FALSE    FALSE
#> R3       B FALSE    FALSE
#> R4       B FALSE    FALSE
#> R5       C FALSE    FALSE
#> R6       C FALSE    FALSE
#> R7       D FALSE    FALSE
#> R8       D FALSE    FALSE
#> R9       E FALSE    FALSE
#> R10      E FALSE    FALSE

As we can see, both have identical columns (Layout, Blank, Excluded) but differ in the number of rows. In addition, the metaData-layer has row names, by default ‘Rn’ where ‘n’ is an increasing integer number.

Whether this ‘R’ is for ‘row’, ‘replicate’, ‘run’, ‘reactor’ or ‘relation’ remains unknown. However, these row names must match the unique identifiers used in BioGasData$reactor of a BGF.

While row names serve as unique identifiers for a fermentation in the metaData-layer, its Layout column is used to group fermentations according a user specified variable. As we have initially specified five, and ten reactor layouts during the creation of ‘myBGF’ and ‘namedBGF’, respectively, these columns are not identical for the two BGF’s.

For both BGF‘s, the Blank and Excluded columns contain only a logic FALSE, as none of the fermentations in either BGF was classified as ’Blank’ or ‘Excluded’ so far.

Finally, let’s compare the BioGasData-layers:

# list 'BioGasData'-layers
# myBGF
myBGF$BioGasData
#>   reactor time product production net_product yield rel_production
#> 1      NA   NA      NA         NA          NA    NA             NA
#> 2      NA   NA      NA         NA          NA    NA             NA
#> 3      NA   NA      NA         NA          NA    NA             NA
#> 4      NA   NA      NA         NA          NA    NA             NA
#> 5      NA   NA      NA         NA          NA    NA             NA

# namedBGF
namedBGF$BioGasData
#>    reactor time product production net_product yield rel_production
#> 1       NA   NA      NA         NA          NA    NA             NA
#> 2       NA   NA      NA         NA          NA    NA             NA
#> 3       NA   NA      NA         NA          NA    NA             NA
#> 4       NA   NA      NA         NA          NA    NA             NA
#> 5       NA   NA      NA         NA          NA    NA             NA
#> 6       NA   NA      NA         NA          NA    NA             NA
#> 7       NA   NA      NA         NA          NA    NA             NA
#> 8       NA   NA      NA         NA          NA    NA             NA
#> 9       NA   NA      NA         NA          NA    NA             NA
#> 10      NA   NA      NA         NA          NA    NA             NA

Both contain only NA values. This is because during object creation with BGF() no data exists to be filled in the BioGasData-layer, but as for each fermentation at least one observation in BioGasData-layer should exist, these empty lines are created. They will be removed as soon as a BGF is updated with update_BGF(), which automatically happens during object creation with from_standard_record() or from_AMPTSV2_report(), or when a BGF is manipulated with dedicated functions of the bgfanalyzer package.

Normally update_BGF() is not called by the package user directly, but here for illustration purpose:

# remove empty columns in 'BioGasData'-layer
updatedBGF <- update_BGF(myBGF)

# inspect updated BGF
updatedBGF
#> 'new name' - a BGF with 5 fermentation(s)
#> 
#> 
#> $ExpParam: 4 experimental paramerters
#> $metaData: 3 meta variables
#> $BioGasData: 0 observations of 7 fermentation variables
#> 
#> 
#> A '$yield' is not calculated yet

# and its 'BioGasData'-layer
updatedBGF$BioGasData
#> [1] reactor        time           product        production     net_product   
#> [6] yield          rel_production
#> <0 Zeilen> (oder row.names mit Länge 0)

The resulting BGF has no observations in the BioGasData-layer.

Visualizing fermentation data

Building BGF’s with BGF() is nice but it objects created that way lack any experimental data. For the topic of how to get data into a BGF, several workflows exist, which all have a dedicated vignette ().

For the rest of this vignette, the focus is on how data visualization works with the bgfanalyzer package.

To this end we need a BGF with data. The package provides an example data set LabscaleBiogas, which is sufficient for this purpose.

Let’s first inspect it:

# bind the example data set to an object
expBGF <- LabscaleBiogas

# rename it
expBGF <- alter_whatever(expBGF,"ExpParam","name","Example BGF")

# inspect it
expBGF
#> 'Example BGF' - a BGF with 15 fermentation(s)
#> 
#> 
#> $ExpParam: 11 experimental paramerters
#> $metaData: 16 meta variables
#> $BioGasData: 735 observations of 7 fermentation variables
#> 
#> 
#>                   yield    sd_yield production sd_production time_production
#> Blank     -2.997602e-14  2.88499567     75.325    5.26794552             1.0
#> Cellulose  2.899017e+01          NA    182.260            NA             3.0
#> S1 ctrl    6.188975e+02  9.87799079    700.350    0.77781746             3.0
#> S1 7d      2.618791e+02  9.49000623    359.790    6.49124025             3.0
#> S1 4d      1.087374e+03 11.21296479    644.655   20.73944189             3.0
#> S2 ctrl    7.781660e+01  0.02079726    119.845    0.04949747             1.0
#> S2 4d      4.011575e+02  2.21973211    110.935    0.21920310             2.0
#> S2 6d      1.335888e+02 42.32196395    128.480   25.54069694             4.5
#>           sd_time_production
#> Blank               0.000000
#> Cellulose                 NA
#> S1 ctrl             0.000000
#> S1 7d               0.000000
#> S1 4d               0.000000
#> S2 ctrl             0.000000
#> S2 4d               0.000000
#> S2 6d               3.535534

This BGF is named ‘Example BGF’ has a 15 fermentations. Its ExpParam- and metaData-layer have 11 experimental parameters and meta variables, respectively. In addition, it has 735 observations of 7 fermentation variables in its BioGasData-layer.

The rest of the listing is a bit different from the listings of ‘myBGF’ and ‘namedBGF’. Instead of the line `A '$yield' is not calculated yet` a tabular so called ‘yield summary’ is shown.

This summary is stored in the metaData-layer of a BGF and usually generated using summarize_yield(). It contains information on the (final) yield, max prduction and the moment when max production occurred per user-specified reactor layout. If one layout was specified for more than one fermentation, a standard deviation for these values is also calculated.

The product curve

But now, let’s finally start visualizing out BGF. The easiest way to do this, is simply plot() it.

# plot the 'BGF'
plot(expBGF)

In this plot, the values of BioGasData$product are shown on the y-axis and BioGasData$time is on the x-axis. Fermentations are colored by their identifier in BioGasData$reactor.

This is the minimal information needed to compare biogas fermentations and thus this plot can be created as soon as data was entered to the BioGasData-layer . The title contains ExpParam$name. A suitable x-axis label can be generated via the xlab argument in plot().

The unit ‘Nml’ in the y-axis label is read ‘norm ml’ and is a gas volume at 0°C and 1 atm pressure. This unit eventually needs correction if BioGasData$product has a different unit.

More advanced data visualization is possible using bgf_plot(). With this function the data in BioGasData$product, BioGasData$netProduct, BioGasData$production, BioGasData$rel_production and BioGasData$yield can be easily plotted:

# product plot
bgf_plot(expBGF,type = "product")

This produces a similar output like `plot(expBGF)`. However, values of BioGasData$product are visualized as color coded lines, not dots. In addition a unit in the x-axis label is automatically, if ExpParam$timeScale was added to the BGF.

The net product per layout curve

By specifying ‘product’ in the type argument of bgf_plot() a similar plot as with plot() was produced.

Now we will see what happens if we set type to ‘netProduct’:

# net product by layout plot
bgf_plot(expBGF,type = "netProduct")

A new plot is generated! Like in the previous plot, BioGasData$time is on the x-axis, but the y-axis scales with BioGasData$net_product and only eight instead of fifteen colored lines are plotted.

This is because each line in this plot belongs to a distinct reactor layout, not a fermentation! Consequently, the values on the y-axis are mean values of BioGasData$net_product calculated over all fermentations that share the same layout.

We can check the reactor layout of a BGF using get_ReactorLayout():

# print reactor layout of expBGF
get_ReactorLayout(expBGF,feedback = TRUE)
#> Example BGF has 'ReactorLayout': 
#> BlankExample BGF has 'ReactorLayout': 
#> BlankExample BGF has 'ReactorLayout': 
#> CelluloseExample BGF has 'ReactorLayout': 
#> S1 ctrlExample BGF has 'ReactorLayout': 
#> S1 ctrlExample BGF has 'ReactorLayout': 
#> S1 7dExample BGF has 'ReactorLayout': 
#> S1 7dExample BGF has 'ReactorLayout': 
#> S1 4dExample BGF has 'ReactorLayout': 
#> S1 4dExample BGF has 'ReactorLayout': 
#> S2 ctrlExample BGF has 'ReactorLayout': 
#> S2 ctrlExample BGF has 'ReactorLayout': 
#> S2 4dExample BGF has 'ReactorLayout': 
#> S2 4dExample BGF has 'ReactorLayout': 
#> S2 6dExample BGF has 'ReactorLayout': 
#> S2 6d
#>  [1] Blank     Blank     Cellulose S1 ctrl   S1 ctrl   S1 7d     S1 7d    
#>  [8] S1 4d     S1 4d     S2 ctrl   S2 ctrl   S2 4d     S2 4d     S2 6d    
#> [15] S2 6d    
#> Levels: Blank Cellulose S1 ctrl S1 7d S1 4d S2 ctrl S2 4d S2 6d

# check how many fermentations
length(get_ReactorLayout(expBGF))
#> [1] 15

# check how many distinct layouts exist in the BGF
length(levels(get_ReactorLayout(expBGF)))
#> [1] 8

We can see, the BGF‘s name, ’Example BGF’, and it’s ‘ReactorLayout’, which has eight levels. Each level except for ‘Cellulose’ belongs to two fermentations, so the total length of the layout is fifteen.

Now that we understood what the differences between the type ‘product’ and ‘netProduct’ plots produced by bgf_plot() is, let’s move on to the next type.

The production curve

When using ‘production’ in the type argument of bgf_plot(), a curve is generated using BioGasData$production.

# production plot
bgf_plot(expBGF,type = "production")

For each fermentation, the values of BioGasData$prodution are plotted as a colored line. The y-axis is labeled as ‘raw exhaust gas flow’, because in a biogas fermentation the ‘production’ usually resembles a volume produced over time. In case of our example it would have the unit ‘[Nml/d]’ (read ‘norm milliliter per day’). Thus, this plot displays how much biogas was produced per day for each fermentation.

The relative production curve

The next type presented is ‘relProduction’, which produces also a line graphic but with adjusted BioGasData$rel_production values on the y-axis.

# relative product plot
bgf_plot(expBGF,type = "relProduction")

In BioGasData$rel_production, (fermentation-specific) the share of BioGasData$product at each time of the max BioGasData$product in ‘%’ is calculated. Subtracting this value from 100% results in the curves plotted above, from which the remaining gas production (in ‘%’) of each fermentation at each time can be read.

yield plots

The last two plots presented in this vignette are a bit different from the previously shown, as they are made to visualize the ‘yield summary’ that is shown when a BGF is print()ed to the R console.

For the first yield plot, one can set type to ‘yield_col’. This plot is directly build from the metaData-layer, like the ‘yield summary’ of the print() result and thus such a summary has to be created using summarize_yield() for this plot type to be working!

# yield col plot
bgf_plot(expBGF,type = "yield_col")

If such a ‘yield summary’ is present in metaData, a column plot is generated. It has reactor layouts on the x-axis and a biogas yield on the y-axis. The biogas yield of a fermentation is the amount of biogas produced from a defined amount of organic input material. In our example the unit would be ‘Nml/goTS’ (read ‘norm milliliter per gram organic total solutes’).

The height of each column resembles the mean yield calculated for a given layout. Error bars on top of each column illustrate the standard deviation around this mean value. Error bars are only generated if at least two fermentations share the same reactor layout.

Lastly, if ‘yield_box’ is specified in the type argument of bgf_plot, a boxplot of BioGasData$yield values is generated for each reactor layout.

# yield box plot
bgf_plot(expBGF,type = "yield_box")

This plot is advantageous to the column plot, as it offers more flexibility in value selection, shows more details on the data and can be created without generating a ‘yield summary’ in metaData.

It is possible to omit the type argument when calling bgf_plot(). In this case the default value ‘all’ is used and all of the above mentioned plots are created at once.

Further information

As mentioned above, different workflows of getting data into a BGF exist, which all have their own vignettes. For educational purpose it is recommended to read at least the vignettes for AMPTS II-generated, and standard report-based BGF’s as this will build a holistic impression of the bgfanalyzer package.

Furthermore, the vignette on manual BGF’s provides a deeper impression and shows how the package can help planing and setting up real world experiments.

In addition to the vignettes, the bgfanalyzer package also provides a detailed documentation on each function. If still open questions remain, don’t hesitate in contacting the authors.