Package {mlr3torch}

Title:	Deep Learning with 'mlr3'
Version:	0.4.0
Description:	Deep Learning library that extends the mlr3 framework by building upon the 'torch' package. It makes it convenient to build, train, and evaluate deep learning models without having to worry about low level details. Custom architectures can be created using the graph language defined in 'mlr3pipelines'.
License:	LGPL (≥ 3)
BugReports:	https://github.com/mlr-org/mlr3torch/issues
URL:	https://mlr3torch.mlr-org.com/, https://github.com/mlr-org/mlr3torch/
Depends:	mlr3 (≥ 1.8.0), mlr3pipelines (≥ 0.11.0), torch (≥ 0.16.2), R (≥ 4.2.0)
Imports:	backports, cli, checkmate (≥ 2.2.0), coro, data.table, lgr, methods, mlr3misc (≥ 0.23), paradox (≥ 1.0.0), R6, withr
Suggests:	callr, curl, future, ggplot2, igraph, jsonlite, knitr, mirai, mlr3tuning (≥ 1.0.0), progress, rmarkdown, rpart, viridis, visNetwork, testthat (≥ 3.0.0), tibble, tfevents, torchvision (≥ 0.10.0), waldo
Config/testthat/edition:	3
NeedsCompilation:	no
ByteCompile:	yes
Encoding:	UTF-8
Collate:	'CallbackSet.R' 'aaa.R' 'TorchCallback.R' 'CallbackSetCheckpoint.R' 'CallbackSetEarlyStopping.R' 'CallbackSetHistory.R' 'CallbackSetLRScheduler.R' 'CallbackSetProgress.R' 'CallbackSetTB.R' 'CallbackSetUnfreeze.R' 'ContextTorch.R' 'DataBackendLazy.R' 'utils.R' 'DataDescriptor.R' 'LearnerTorch.R' 'GraphLearnerTorch.R' 'LearnerFTTransformer.R' 'LearnerTorchFeatureless.R' 'LearnerTorchImage.R' 'LearnerTorchMLP.R' 'TorchDescriptor.R' 'TorchLoss.R' 'task_dataset.R' 'shape.R' 'PipeOpTorchIngress.R' 'LearnerTorchModel.R' 'LearnerTorchModule.R' 'LearnerTorchRealMLP.R' 'LearnerTorchTabM.R' 'LearnerTorchTabResNet.R' 'LearnerTorchVision.R' 'MeasureTorch.R' 'ModelDescriptor.R' 'PipeOpModule.R' 'PipeOpTorch.R' 'PipeOpTaskPreprocTorch.R' 'PipeOpTorchActivation.R' 'PipeOpTorchAdaptiveAvgPool.R' 'PipeOpTorchAdaptiveMaxPool.R' 'PipeOpTorchAvgPool.R' 'PipeOpTorchBatchNorm.R' 'PipeOpTorchBlock.R' 'PipeOpTorchCallbacks.R' 'PipeOpTorchConv.R' 'PipeOpTorchConvTranspose.R' 'PipeOpTorchDropout.R' 'PipeOpTorchFTCLS.R' 'PipeOpTorchFTTransformerBlock.R' 'PipeOpTorchFn.R' 'PipeOpTorchGroupNorm.R' 'PipeOpTorchHead.R' 'PipeOpTorchIdentity.R' 'PipeOpTorchLayerNorm.R' 'PipeOpTorchLinear.R' 'PipeOpTorchLoss.R' 'PipeOpTorchMaxPool.R' 'PipeOpTorchMerge.R' 'PipeOpTorchModel.R' 'PipeOpTorchMultiheadAttention.R' 'PipeOpTorchOptimizer.R' 'PipeOpTorchRecurrent.R' 'PipeOpTorchReshape.R' 'PipeOpTorchSoftmax.R' 'PipeOpTorchTokenizer.R' 'PipeOpTorchTransformerEncoderLayer.R' 'PredictionTorch.R' 'Select.R' 'TaskClassif_cifar.R' 'TaskClassif_lazy_iris.R' 'TaskClassif_melanoma.R' 'TaskClassif_mnist.R' 'TaskClassif_tiny_imagenet.R' 'TaskTorch.R' 'TorchOptimizer.R' 'bibentries.R' 'cache.R' 'lazy_tensor.R' 'learner_torch_methods.R' 'materialize.R' 'merge_graphs.R' 'multi_tensor_dataset.R' 'nn.R' 'nn_graph.R' 'paramset_torchlearner.R' 'pipeop_torch.R' 'preprocess.R' 'rd_info.R' 'with_torch_settings.R' 'zzz.R'
Config/roxygen2/version:	8.1.0
Packaged:	2026-10-30 15:26:03 UTC; sebi
Author:	Sebastian Fischer [cre, aut], Bernd Bischl [ctb], Lukas Burk [ctb], Martin Binder [aut], Florian Pfisterer [ctb], Carson Zhang [ctb]
Maintainer:	Sebastian Fischer <sebf.fischer@gmail.com>
Repository:	CRAN
Date/Publication:	2026-10-30 16:40:02 UTC

mlr3torch: Deep Learning with 'mlr3'

Description

Deep Learning library that extends the mlr3 framework by building upon the 'torch' package. It makes it convenient to build, train, and evaluate deep learning models without having to worry about low level details. Custom architectures can be created using the graph language defined in 'mlr3pipelines'.

Options

• mlr3torch.cache: Whether to cache the downloaded data (TRUE) or not (FALSE, default). This can also be set to a specific folder on the file system to be used as the cache directory. Without caching, every task that downloads its data fetches it again in each session.

Author(s)

Maintainer: Sebastian Fischer sebf.fischer@gmail.com (ORCID)

Authors:

• Sebastian Fischer sebf.fischer@gmail.com (ORCID)

• Martin Binder mlr.developer@mb706.com

Other contributors:

• Bernd Bischl bernd_bischl@gmx.net (ORCID) [contributor]

• Lukas Burk github@quantenbrot.de (ORCID) [contributor]

• Florian Pfisterer pfistererf@googlemail.com (ORCID) [contributor]

• Carson Zhang carsonzhang4@gmail.com [contributor]

See Also

Useful links:

• https://mlr3torch.mlr-org.com/

• https://github.com/mlr-org/mlr3torch/

• Report bugs at https://github.com/mlr-org/mlr3torch/issues

Compare lazy tensors

Description

Compares lazy tensors using their indices and the data descriptor's hash. This means that if two lazy_tensors:

• are equal: they will mateterialize to the same tensors.

• are unequal: they might materialize to the same tensors.

Usage

## S3 method for class 'lazy_tensor'
x == y

Arguments

Data Descriptor

Description

A data descriptor is a rather internal data structure used in the lazy_tensor data type. In essence it is an annotated torch::dataset and a preprocessing graph (consisting mosty of PipeOpModule operators). The additional meta data (e.g. pointer, shapes) makes it possible to preprocess lazy_tensors in an mlr3pipelines::Graph just like any (non-lazy) data types. The preprocessing is applied when materialize() is called on the lazy_tensor.

To create a data descriptor, you can also use the as_data_descriptor() function.

Details

While it would be more natural to define this as an S3 class, we opted for an R6 class to avoid the usual trouble of serializing S3 objects. If each row contained a DataDescriptor as an S3 class, this would copy the object when serializing.

Public fields

Methods

Public methods

• DataDescriptor$new()

• DataDescriptor$print()

• DataDescriptor$clone()

DataDescriptor$new()

Creates a new instance of this R6 class.

Usage

DataDescriptor$new(
  dataset,
  dataset_shapes = NULL,
  graph = NULL,
  input_map = NULL,
  pointer = NULL,
  pointer_shape = NULL,
  pointer_shape_predict = NULL,
  clone_graph = TRUE
)

Arguments

DataDescriptor$print()

Prints the object

Usage

DataDescriptor$print(...)

Arguments

DataDescriptor$clone()

The objects of this class are cloneable with this method.

Usage

DataDescriptor$clone(deep = FALSE)

Arguments

See Also

ModelDescriptor, lazy_tensor

Examples

# Create a dataset
ds = dataset(
  initialize = function() self$x = torch_randn(10, 3, 3),
  .getitem = function(i) list(x = self$x[i, ]),
  .length = function() nrow(self$x)
)()
dd = DataDescriptor$new(ds, list(x = c(NA, 3, 3)))
dd
# is the same as using the converter:
as_data_descriptor(ds, list(x = c(NA, 3, 3)))

Graph Learner for Torch Networks

Description

The GraphLearner that as_learner_torch() returns. On top of a GraphLearner it has the ⁠$network⁠, ⁠$loss⁠, ⁠$optimizer⁠ and ⁠$callbacks⁠ fields and the ⁠$dataset()⁠ method of a LearnerTorch. If the graph contains a PipeOpTorchModel, the loss, optimizer and callbacks are those of its configuring operators, otherwise those of its base LearnerTorch. It is not in the mlr_learners dictionary, because it cannot be constructed without a graph.

Super classes

mlr3::Learner -> mlr3pipelines::GraphLearner -> GraphLearnerTorch

Active bindings

Methods

Public methods

• GraphLearnerTorch$dataset()

• GraphLearnerTorch$clone()

GraphLearnerTorch$dataset()

Create the dataset for a task, i.e. the tensors that are fed to the network.

Usage

GraphLearnerTorch$dataset(
  task,
  stage = if (is.null(self$model)) "train" else "predict"
)

Arguments

Returns

dataset

GraphLearnerTorch$clone()

The objects of this class are cloneable with this method.

Usage

GraphLearnerTorch$clone(deep = FALSE)

Arguments

See Also

Other Learner: as_learner_torch(), mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.module, mlr_learners.realmlp, mlr_learners.tab_resnet, mlr_learners.tabm, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Represent a Model with Meta-Info

Description

Represents a model; possibly a complete model, possibly one in the process of being built up.

This model takes input tensors of shapes shapes_in and pipes them through graph. Input shapes get mapped to input channels of graph. Output shapes are named by the output channels of graph; it is also possible to represent no-ops on tensors, in which case names of input and output should be identical.

ModelDescriptor objects typically represent partial models being built up, in which case the pointer slot indicates a specific point in the graph that produces a tensor of shape pointer_shape, on which the graph should be extended. It is allowed for the graph in this structure to be modified by-reference in different parts of the code. However, these modifications may never add edges with elements of the Graph as destination. In particular, no element of graph$input may be removed by reference, e.g. by adding an edge to the Graph that has the input channel of a PipeOp that was previously without parent as its destination.

In most cases it is better to create a specific ModelDescriptor by training a Graph consisting (mostly) of operators PipeOpTorchIngress, PipeOpTorch, PipeOpTorchLoss, PipeOpTorchOptimizer, and PipeOpTorchCallbacks.

A ModelDescriptor can be converted to a nn_graph via model_descriptor_to_module.

Usage

ModelDescriptor(
  graph,
  ingress,
  task,
  optimizer = NULL,
  loss = NULL,
  callbacks = NULL,
  pointer = NULL,
  pointer_shape = NULL
)

Arguments

Value

(ModelDescriptor)

See Also

Other Model Configuration: mlr_pipeops_torch_callbacks, mlr_pipeops_torch_loss, mlr_pipeops_torch_optimizer, model_descriptor_union()

Other Graph Network: TorchIngressToken(), as_learner_torch(), mlr_learners_torch_model, mlr_pipeops_module, mlr_pipeops_torch, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, model_descriptor_to_learner(), model_descriptor_to_module(), model_descriptor_union(), nn_graph(), pipeop_torch()

Prediction Object for a Generic Torch Task

Description

The Prediction object returned by learners that were trained on a TaskTorch.

Because a TaskTorch can represent very different learning problems, this class does not prescribe much about how truth, response, prob and se are stored. This is defined by the task's prediction encoder, where you need to ensure that the first dimension indexes the observations. Within that, an element may be an atomic vector, a matrix(), an array() of any dimensionality, a data.table or a lazy_tensor. truth is whatever task$truth() returned – a vector for one target, a data.table for several, a lazy_tensor for a lazy tensor column, and nothing at all for a task without a target.

When a prediction is converted to a data.table, which is e.g. used for the printer, the conversion depends on the type of the object. A prob matrix spreads into one column per class, the way it does for a classification prediction. Everything else that is wider than one value per observation becomes a single column whose cells hold that observation's own array, printed as its shape – ⁠<array[3]>⁠ for a response matrix with three columns, ⁠<array[3x224x224]>⁠ for a prob with a class dimension and two spatial ones.

Missing Predictions

Only a response with a single value per observation can report a missing prediction, so ⁠$missing⁠ can only be TRUE for scalars and is always FALSE for parially missing predictions.

Super class

mlr3::Prediction -> PredictionTorch

Active bindings

Methods

Public methods

• PredictionTorch$new()

• PredictionTorch$clone()

PredictionTorch$new()

Creates a new instance of this R6 class.

Usage

PredictionTorch$new(
  task = NULL,
  row_ids = task$row_ids,
  truth = if (!is.null(task)) task$truth(row_ids),
  response = NULL,
  prob = NULL,
  se = NULL,
  lazy_tensor = NULL,
  weights = NULL,
  check = TRUE
)

Arguments

PredictionTorch$clone()

The objects of this class are cloneable with this method.

Usage

PredictionTorch$clone(deep = FALSE)

Arguments

Examples

d = data.frame(x = rnorm(10), y1 = rnorm(10), y2 = rnorm(10))
task = as_task_torch(d, target = c("y1", "y2"))
PredictionTorch$new(task, response = as.matrix(d[, c("y1", "y2")]))

Selector Functions for Character Vectors

Description

A Select function subsets a character vector. They are used by the callback CallbackSetUnfreeze to select parameters to freeze or unfreeze during training.

Usage

select_all()

select_none()

select_grep(pattern, ignore.case = FALSE, perl = FALSE, fixed = FALSE)

select_name(param_names, assert_present = TRUE)

select_invert(select)

Arguments

Functions

• select_all(): select_all selects all elements

• select_none(): select_none selects no elements

• select_grep(): select_grep selects elements with names matching a regular expression

• select_name(): select_name selects elements with names matching the given names

• select_invert(): select_invert selects the elements NOT selected by the given selector

Examples

select_all()(c("a", "b"))
select_none()(c("a", "b"))
select_grep("b$")(c("ab", "ac"))
select_name("a")(c("a", "b"))
select_invert(select_all())(c("a", "b"))

Torch Callback

Description

This wraps a CallbackSet and annotates it with metadata, most importantly a ParamSet. The callback is created for the given parameter values by calling the ⁠$generate()⁠ method.

This class is usually used to configure the callback of a torch learner, e.g. when constructing a learner of in a ModelDescriptor.

For a list of available callbacks, see mlr3torch_callbacks. To conveniently retrieve a TorchCallback, use t_clbk().

Parameters

Defined by the constructor argument param_set. If no parameter set is provided during construction, the parameter set is constructed by creating a parameter for each argument of the wrapped callback, where the parameters are then of type ParamUty.

Super class

TorchDescriptor -> TorchCallback

Active bindings

Methods

Public methods

• TorchCallback$new()

• TorchCallback$generate()

• TorchCallback$clone()

TorchCallback$new()

Creates a new instance of this R6 class.

Usage

TorchCallback$new(
  callback_generator,
  param_set = NULL,
  id = NULL,
  label = NULL,
  packages = NULL,
  man = NULL,
  additional_args = NULL,
  weight = NULL
)

Arguments

TorchCallback$generate()

Generates the CallbackSet, applying ⁠$weight⁠ if it is set.

Usage

TorchCallback$generate()

TorchCallback$clone()

The objects of this class are cloneable with this method.

Usage

TorchCallback$clone(deep = FALSE)

Arguments

See Also

Other Callback: as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Other Torch Descriptor: TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_callbacks(), as_torch_loss(), as_torch_optimizer(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Examples

# Create a new torch callback from an existing callback set
torch_callback = TorchCallback$new(CallbackSetCheckpoint)
# The parameters are inferred
torch_callback$param_set

# Retrieve a torch callback from the dictionary
torch_callback = t_clbk("checkpoint",
  path = tempfile(), freq = 1
)
torch_callback
torch_callback$label
torch_callback$id

# open the help page of the wrapped callback set
# torch_callback$help()

# Create the callback set
callback = torch_callback$generate()
callback
# is the same as
CallbackSetCheckpoint$new(
  path = tempfile(), freq = 1
)

# Use in a learner
learner = lrn("regr.mlp", callbacks = t_clbk("checkpoint"))
# the parameters of the callback are added to the learner's parameter set
learner$param_set

Base Class for Torch Descriptors

Description

Abstract Base Class from which TorchLoss, TorchOptimizer, and TorchCallback inherit. This class wraps a generator (R6Class Generator or the torch version of such a generator) and annotates it with metadata such as a ParamSet, a label, an ID, packages, or a manual page.

The parameters are the construction arguments of the wrapped generator and the parameter ⁠$values⁠ are passed to the generator when calling the public method ⁠$generate()⁠.

Parameters

Defined by the constructor argument param_set. All parameters are tagged with "train", but this is done automatically during initialize.

Public fields

Active bindings

Methods

Public methods

• TorchDescriptor$new()

• TorchDescriptor$print()

• TorchDescriptor$generate()

• TorchDescriptor$help()

• TorchDescriptor$clone()

TorchDescriptor$new()

Creates a new instance of this R6 class.

Usage

TorchDescriptor$new(
  generator,
  id = NULL,
  param_set = NULL,
  packages = NULL,
  label = NULL,
  man = NULL,
  additional_args = NULL
)

Arguments

TorchDescriptor$print()

Prints the object

Usage

TorchDescriptor$print(...)

Arguments

TorchDescriptor$generate()

Calls the generator with the given parameter values.

Usage

TorchDescriptor$generate()

TorchDescriptor$help()

Displays the help file of the wrapped object.

Usage

TorchDescriptor$help()

TorchDescriptor$clone()

The objects of this class are cloneable with this method.

Usage

TorchDescriptor$clone(deep = FALSE)

Arguments

See Also

Other Torch Descriptor: TorchCallback, TorchLoss, TorchOptimizer, as_torch_callbacks(), as_torch_loss(), as_torch_optimizer(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Torch Ingress Token

Description

This function creates an S3 class of class "TorchIngressToken", which is an internal data structure. It contains the (meta-)information of how a batch is generated from a Task and fed into an entry point of the neural network. It is stored as the ingress field in a ModelDescriptor.

Usage

TorchIngressToken(features, batchgetter, shape = NULL)

Arguments

Value

TorchIngressToken object.

See Also

Other Graph Network: ModelDescriptor(), as_learner_torch(), mlr_learners_torch_model, mlr_pipeops_module, mlr_pipeops_torch, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, model_descriptor_to_learner(), model_descriptor_to_module(), model_descriptor_union(), nn_graph(), pipeop_torch()

Examples

# Define a task for which we want to define an ingress token
task = tsk("iris")

# We create an ingress token for two feature Sepal.Length and Petal.Length:
# We have to specify the features, the batchgetter and the shape
features = c("Sepal.Length", "Petal.Length")
# As a batchgetter we use batchgetter_num

batch_dt = task$data(rows = 1:10, cols =features)
batch_dt
batch_tensor = batchgetter_num(batch_dt, "cpu")
batch_tensor

# The shape is unknown in the first dimension (batch dimension)

ingress_token = TorchIngressToken(
  features = features,
  batchgetter = batchgetter_num,
  shape = c(NA, 2)
)
ingress_token

Torch Loss

Description

This wraps a torch::nn_loss and annotates it with metadata, most importantly a ParamSet. The loss function is created for the given parameter values by calling the ⁠$generate()⁠ method.

This class is usually used to configure the loss function of a torch learner, e.g. when construcing a learner or in a ModelDescriptor.

For a list of available losses, see mlr3torch_losses. Items from this dictionary can be retrieved using t_loss().

Parameters

Defined by the constructor argument param_set. If no parameter set is provided during construction, the parameter set is constructed by creating a parameter for each argument of the wrapped loss function, where the parameters are then of type ParamUty.

Super class

TorchDescriptor -> TorchLoss

Public fields

Methods

Public methods

• TorchLoss$new()

• TorchLoss$print()

• TorchLoss$generate()

• TorchLoss$clone()

TorchLoss$new()

Creates a new instance of this R6 class.

Usage

TorchLoss$new(
  torch_loss,
  task_types = NULL,
  param_set = NULL,
  id = NULL,
  label = NULL,
  packages = NULL,
  man = NULL
)

Arguments

TorchLoss$print()

Prints the object

Usage

TorchLoss$print(...)

Arguments

TorchLoss$generate()

Instantiates the loss function.

Usage

TorchLoss$generate(task = NULL)

Arguments

Returns

torch_loss

TorchLoss$clone()

The objects of this class are cloneable with this method.

Usage

TorchLoss$clone(deep = FALSE)

Arguments

See Also

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchOptimizer, as_torch_callbacks(), as_torch_loss(), as_torch_optimizer(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Examples

# Create a new torch loss
torch_loss = TorchLoss$new(torch_loss = nn_mse_loss, task_types = "regr")
torch_loss
# the parameters are inferred
torch_loss$param_set

# Retrieve a loss from the dictionary:
torch_loss = t_loss("mse", reduction = "mean")
# is the same as
torch_loss
torch_loss$param_set
torch_loss$label
torch_loss$task_types
torch_loss$id

# Create the loss function
loss_fn = torch_loss$generate()
loss_fn
# Is the same as
nn_mse_loss(reduction = "mean")

# open the help page of the wrapped loss function
# torch_loss$help()

# Use in a learner
learner = lrn("regr.mlp", loss = t_loss("mse"))
# The parameters of the loss are added to the learner's parameter set
learner$param_set

Torch Optimizer

Description

This wraps a torch::torch_optimizer_generator and annotates it with metadata, most importantly a ParamSet. The optimizer is created for the given parameter values by calling the ⁠$generate()⁠ method.

This class is usually used to configure the optimizer of a torch learner, e.g. when constructing a learner or in a ModelDescriptor.

For a list of available optimizers, see mlr3torch_optimizers. Items from this dictionary can be retrieved using t_opt().

Parameters

Defined by the constructor argument param_set. If no parameter set is provided during construction, the parameter set is constructed by creating a parameter for each argument of the wrapped optimizer, where the parameters are then of type ParamUty.

In addition, every TorchOptimizer has the parameter:

• param_groups :: ⁠function(params) -> list()⁠
  A function that receives the named list() of network parameters and returns a list() of parameter groups, in the format expected by the wrapped torch optimizer. This makes it possible to optimize different parts of the network differently, e.g. to give the first layer a different learning rate than the rest:

  param_groups = function(params) list(
    list(params = params[startsWith(names(params), "0.")], lr = 0.9),
    list(params = params[!startsWith(names(params), "0.")], lr = 0.001)
  )
  

  When a LearnerTorch is configured, this is reachable as opt.param_groups. Default is NULL, i.e. all parameters form a single group.

Super class

TorchDescriptor -> TorchOptimizer

Methods

Public methods

• TorchOptimizer$new()

• TorchOptimizer$generate()

• TorchOptimizer$clone()

TorchOptimizer$new()

Creates a new instance of this R6 class.

Usage

TorchOptimizer$new(
  torch_optimizer,
  param_set = NULL,
  id = NULL,
  label = NULL,
  packages = NULL,
  man = NULL
)

Arguments

TorchOptimizer$generate()

Instantiates the optimizer.

Usage

TorchOptimizer$generate(params)

Arguments

Returns

torch_optimizer

TorchOptimizer$clone()

The objects of this class are cloneable with this method.

Usage

TorchOptimizer$clone(deep = FALSE)

Arguments

See Also

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, as_torch_callbacks(), as_torch_loss(), as_torch_optimizer(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Examples

# Create a new torch optimizer
torch_opt = TorchOptimizer$new(optim_ignite_adam, label = "adam")
torch_opt
# If the param set is not specified, parameters are inferred but are of class ParamUty
torch_opt$param_set

# open the help page of the wrapped optimizer
# torch_opt$help()

# Retrieve an optimizer from the dictionary
torch_opt = t_opt("sgd", lr = 0.1)
torch_opt
torch_opt$param_set
torch_opt$label
torch_opt$id

# Create the optimizer for a network
net = nn_linear(10, 1)
opt = torch_opt$generate(net$parameters)

# is the same as
optim_sgd(net$parameters, lr = 0.1)

# Use in a learner
learner = lrn("regr.mlp", optimizer = t_opt("sgd"))
# The parameters of the optimizer are added to the learner's parameter set
learner$param_set

Convert to Data Descriptor

Description

Converts the input to a DataDescriptor.

Usage

as_data_descriptor(x, dataset_shapes, ...)

Arguments

Examples

ds = dataset("example",
  initialize = function() self$iris = iris[, -5],
  .getitem = function(i) list(x = torch_tensor(as.numeric(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
as_data_descriptor(ds, list(x = c(NA, 4L)))

# if the dataset has a .getbatch method, the shapes are inferred
ds2 = dataset("example",
  initialize = function() self$iris = iris[, -5],
  .getbatch = function(i) list(x = torch_tensor(as.matrix(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
as_data_descriptor(ds2)

Convert to Lazy Tensor

Description

Converts an object to a lazy_tensor. A dataset is wrapped as-is, a torch_tensor or a numeric() is first turned into a dataset whose first dimension indexes the observations.

Usage

as_lazy_tensor(x, ...)

## S3 method for class 'dataset'
as_lazy_tensor(x, dataset_shapes = NULL, ids = NULL, ...)

Arguments

Examples

iris_ds = dataset("iris",
  initialize = function() {
    self$iris = iris[, -5]
  },
  .getbatch = function(i) {
    list(x = torch_tensor(as.matrix(self$iris[i, ])))
  },
  .length = function() nrow(self$iris)
)()
# no need to specify the dataset shapes as they can be inferred from the .getbatch method
# only first 5 observations
as_lazy_tensor(iris_ds, ids = 1:5)
# all observations
head(as_lazy_tensor(iris_ds))

iris_ds2 = dataset("iris",
  initialize = function() self$iris = iris[, -5],
  .getitem = function(i) list(x = torch_tensor(as.numeric(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
# if .getitem is implemented we cannot infer the shapes as they might vary,
# so we have to annotate them explicitly
as_lazy_tensor(iris_ds2, dataset_shapes = list(x = c(NA, 4L)))[1:5]

# Convert a matrix
lt = as_lazy_tensor(matrix(rnorm(100), nrow = 20))
materialize(lt[1:5], rbind = TRUE)

Convert a Graph to a Torch Learner

Description

Converts a Graph representing a deep learning pipeline into a GraphLearnerTorch. The graph either builds the network from PipeOpTorch operators and ends in a PipeOpTorchModel, or its base learner is a LearnerTorch, e.g. po("scale") %>>% lrn("classif.mlp"). The advantage over using as_learner() is that the resulting learner exposes methods like ⁠$dataset()⁠ and fields like ⁠$network⁠.

Usage

as_learner_torch(x, ...)

## S3 method for class 'Graph'
as_learner_torch(x, id = NULL, ...)

## S3 method for class 'PipeOp'
as_learner_torch(x, ...)

## S3 method for class 'GraphLearner'
as_learner_torch(x, id = x$id, ...)

Arguments

Value

GraphLearner

See Also

Other Graph Network: ModelDescriptor(), TorchIngressToken(), mlr_learners_torch_model, mlr_pipeops_module, mlr_pipeops_torch, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, model_descriptor_to_learner(), model_descriptor_to_module(), model_descriptor_union(), nn_graph(), pipeop_torch()

Other Learner: GraphLearnerTorch, mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.module, mlr_learners.realmlp, mlr_learners.tab_resnet, mlr_learners.tabm, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

graph = po("scale") %>>%
  po("torch_ingress_num") %>>%
  nn("linear", out_features = 10) %>>%
  nn("relu") %>>%
  nn("head") %>>%
  po("torch_loss", "cross_entropy") %>>%
  po("torch_optimizer", "adam", lr = 0.1) %>>%
  po("torch_model_classif", epochs = 1, batch_size = 32)

learner = as_learner_torch(graph)
learner$id
learner$param_set$set_values(linear.out_features = 20, torch_optimizer.lr = 0.01)

task = tsk("iris")
learner$train(task)
learner$network
learner$predict(task)

learner$dataset(task, "train")

Convert to CallbackSetLRScheduler

Description

Convert a torch scheduler generator to a CallbackSetLRScheduler.

Usage

as_lr_scheduler(x, step_on_epoch)

Arguments

Create a Generic Torch Task

Description

Creates a TaskTorch, the general-purpose task type of mlr3torch, from a data.frame() or a DataBackend. See the Custom Learning Problems article for more information.

Usage

as_task_torch(x, target = NULL, id = deparse(substitute(x))[1L], ...)

Arguments

Value

TaskTorch

Examples

# multi-output regression
d = data.frame(x = rnorm(50))
d$y1 = d$x + rnorm(50)
d$y2 = 2 * d$x + rnorm(50)
as_task_torch(d, target = c("y1", "y2"))

# unsupervised: no target at all
as_task_torch(data.frame(a = rnorm(50), b = rnorm(50)))

Convert to a TorchCallback

Description

Converts an object to a TorchCallback.

Usage

as_torch_callback(x, clone = FALSE, ...)

Arguments

Value

TorchCallback.

See Also

Other Callback: TorchCallback, as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Convert to a list of Torch Callbacks

Description

Converts an object to a list of TorchCallback.

Usage

as_torch_callbacks(x, clone, ...)

Arguments

Value

list() of TorchCallbacks

See Also

Other Callback: TorchCallback, as_torch_callback(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_loss(), as_torch_optimizer(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Convert to TorchLoss

Description

Converts an object to a TorchLoss.

Usage

as_torch_loss(x, clone = FALSE, ...)

Arguments

Value

TorchLoss.

See Also

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_callbacks(), as_torch_optimizer(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Examples

# Define a custom loss, here the quantile (pinball) loss for quantile regression.
# The class must contain "nn_loss", as this is what as_torch_loss() dispatches on.
nn_quantile_loss = nn_module(c("nn_quantile_loss", "nn_loss"),
  initialize = function(q = 0.5) {
    self$q = q
  },
  forward = function(input, target) {
    d = target - input
    torch_mean(torch_max(self$q * d, other = (self$q - 1) * d))
  }
)

# additional arguments are passed to the TorchLoss constructor
quantile_loss = as_torch_loss(nn_quantile_loss, task_types = "regr")
quantile_loss
# the parameters are inferred from the loss' initialize method
quantile_loss$param_set
# and can be configured when using the loss in a learner
lrn("regr.mlp", loss = quantile_loss, loss.q = 0.9)

# predefined losses can be converted from their key,
# this is the same as t_loss("mse")
as_torch_loss("mse")

# TorchLosses are returned as-is, unless clone is TRUE
loss = t_loss("mse")
identical(as_torch_loss(loss), loss)
identical(as_torch_loss(loss, clone = TRUE), loss)

Convert to TorchOptimizer

Description

Converts an object to a TorchOptimizer.

Usage

as_torch_optimizer(x, clone = FALSE, ...)

Arguments

Value

TorchOptimizer

See Also

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_callbacks(), as_torch_loss(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Examples

# convert a `torch::torch_optimizer_generator`
as_torch_optimizer(optim_adamw)
# the id defaults to the name of the generator, but can be overwritten
as_torch_optimizer(optim_adamw, id = "my_adamw", label = "My AdamW")

# convert a key of mlr3torch_optimizers, this is the same as t_opt("adamw")
as_torch_optimizer("adamw")

# TorchOptimizers are returned as-is, unless clone is TRUE
opt = t_opt("adamw")
identical(as_torch_optimizer(opt), opt)
identical(as_torch_optimizer(opt, clone = TRUE), opt)

Assert that a Dimension Exists

Description

Rejects a dim parameter that does not address a dimension of shape. Resolve negative indices with resolve_dim() first and pass both, so that the error message can report the value the user actually specified.

Usage

assert_dim_in_range(dim, true_dim, shape, id)

Arguments

Value

The resolved dimension, invisibly.

See Also

Other Shape Assertions: assert_known_dims(), assert_ndim(), assert_not_batch_dim(), assert_positive_extent(), assert_same_batch_size(), assert_same_ndim(), assert_shape(), assert_shapes()

Examples

shape = c(NA, 3, 8, 8)
assert_dim_in_range(-1L, resolve_dim(-1L, shape), shape, id = "nn_squeeze")
try(assert_dim_in_range(-5L, resolve_dim(-5L, shape), shape, id = "nn_squeeze"))

Assert that Dimensions are Known

Description

Ensure that a specific dimension is known, i.e. not NA.

Usage

assert_known_dims(shape, dims, what, id = NULL)

Arguments

Value

The shape, invisibly.

See Also

Other Shape Assertions: assert_dim_in_range(), assert_ndim(), assert_not_batch_dim(), assert_positive_extent(), assert_same_batch_size(), assert_same_ndim(), assert_shape(), assert_shapes()

Examples

# a convolution needs the number of input channels, but not the spatial extent
assert_known_dims(c(NA, 3, NA, NA), 2, "the number of channels", id = "nn_conv2d")
try(assert_known_dims(c(NA, NA, 10, 10), 2, "the number of channels", id = "nn_conv2d"))

Assert Lazy Tensor

Description

Asserts whether something is a lazy tensor.

Usage

assert_lazy_tensor(x)

Arguments

Assert the Number of Dimensions of a Shape

Description

Rejects a shape with the wrong number of dimensions. Give either ndim, the number(s) of dimensions the operator accepts, or the bounds min and max (either on its own), which is what an operator that accepts a range of them uses, as batch normalization does.

Usage

assert_ndim(shape, ndim = NULL, id, min = NULL, max = NULL)

Arguments

Value

The shape, invisibly.

See Also

Other Shape Assertions: assert_dim_in_range(), assert_known_dims(), assert_not_batch_dim(), assert_positive_extent(), assert_same_batch_size(), assert_same_ndim(), assert_shape(), assert_shapes()

Examples

assert_ndim(c(NA, 3, 32, 32), 4L, id = "nn_conv2d")
try(assert_ndim(c(NA, 3), 4L, id = "nn_conv2d"))
# batch normalization accepts a range
try(assert_ndim(c(NA, 3, 4, 5), id = "nn_batch_norm1d", min = 2L, max = 3L))

Assert that a Dimension is not the Batch Dimension

Description

Rejects an operation on the first dimension, which is the batch dimension. An operator that changes it would silently change the number of observations, which fails much later with a mismatch against the target.

Usage

assert_not_batch_dim(dim, shape, id)

Arguments

Value

The dimension, invisibly.

See Also

Other Shape Assertions: assert_dim_in_range(), assert_known_dims(), assert_ndim(), assert_positive_extent(), assert_same_batch_size(), assert_same_ndim(), assert_shape(), assert_shapes()

Examples

assert_not_batch_dim(2L, c(NA, 3, 8), id = "nn_squeeze")
try(assert_not_batch_dim(1L, c(NA, 3, 8), id = "nn_squeeze"))

Assert that a Computed Output Size is Positive

Description

Rejects a computed output size that no tensor can have, i.e. one that is not a positive number. Operators such as convolutions and pooling compute their spatial output sizes from kernel_size, stride, padding and dilation, a combination of which can produce a size of zero or less. Unknown (NA) sizes are accepted, since nothing can be said about them.

Usage

assert_positive_extent(extent, shape_in, id)

Arguments

Value

The extent, invisibly.

See Also

Other Shape Assertions: assert_dim_in_range(), assert_known_dims(), assert_ndim(), assert_not_batch_dim(), assert_same_batch_size(), assert_same_ndim(), assert_shape(), assert_shapes()

Examples

assert_positive_extent(c(30, NA), c(NA, 3, 32, 32), id = "nn_conv2d")
try(assert_positive_extent(c(0, 4), c(NA, 3, 32, 32), id = "nn_conv2d"))

Assert that Shapes have the Same Batch Size

Description

Rejects inputs whose batch sizes, i.e. first dimensions, disagree. An unknown (NA) batch size is compatible with any other, so only the known ones have to agree.

Unlike the other assertions this returns the common batch size rather than its input, because that is what the caller needs next: an operator that drops the batch dimension while it works has to put it back afterwards.

Usage

assert_same_batch_size(shapes, id)

Arguments

Value

(integer(1)) The common batch size, invisibly, or NA_integer_ if no input has a known one.

See Also

Other Shape Assertions: assert_dim_in_range(), assert_known_dims(), assert_ndim(), assert_not_batch_dim(), assert_positive_extent(), assert_same_ndim(), assert_shape(), assert_shapes()

Examples

assert_same_batch_size(list(c(8, 3), c(8, 5)), id = "nn_block")
# an unknown batch size is compatible with a known one, which is the one that is returned
assert_same_batch_size(list(c(NA, 3), c(8, 5)), id = "nn_block")
try(assert_same_batch_size(list(c(8, 3), c(4, 5)), id = "nn_block"))

Assert that Shapes have the Same Number of Dimensions

Description

Rejects inputs that do not all have the same number of dimensions, which the operators that combine several inputs require: shorter shapes are not left-padded with 1s, because the first dimension is the batch dimension.

Usage

assert_same_ndim(shapes, id)

Arguments

Value

The shapes, invisibly.

See Also

Other Shape Assertions: assert_dim_in_range(), assert_known_dims(), assert_ndim(), assert_not_batch_dim(), assert_positive_extent(), assert_same_batch_size(), assert_shape(), assert_shapes()

Examples

assert_same_ndim(list(c(NA, 3), c(NA, 5)), id = "nn_merge_cat")
try(assert_same_ndim(list(c(NA, 3), c(NA, 5, 2)), id = "nn_merge_cat"))

Assert a Shape

Description

Checks that shape is a valid shape, i.e. an integer() with at least one dimension, NA where a dimension is unknown. See the "Shape Inference" section of PipeOpTorch for the conventions.

Usage

assert_shape(
  shape,
  null_ok = FALSE,
  coerce = TRUE,
  unknown_batch = NULL,
  len = NULL
)

Arguments

Value

The shape, coerced to an integer() if coerce is TRUE.

See Also

Other Shape Assertions: assert_dim_in_range(), assert_known_dims(), assert_ndim(), assert_not_batch_dim(), assert_positive_extent(), assert_same_batch_size(), assert_same_ndim(), assert_shapes()

Examples

assert_shape(c(NA, 3, 32, 32))
try(assert_shape("not a shape"))

Assert a List of Shapes

Description

Checks that shapes is a non-empty list() of valid shapes, see assert_shape().

Usage

assert_shapes(
  shapes,
  coerce = TRUE,
  named = FALSE,
  null_ok = FALSE,
  unknown_batch = NULL
)

Arguments

Value

The shapes, coerced to integer() if coerce is TRUE.

See Also

Other Shape Assertions: assert_dim_in_range(), assert_known_dims(), assert_ndim(), assert_not_batch_dim(), assert_positive_extent(), assert_same_batch_size(), assert_same_ndim(), assert_shape()

Examples

assert_shapes(list(c(NA, 3), c(NA, 5)))

Auto Device

Description

Resolves the device parameter of a learner. "auto" becomes "cuda" when a CUDA device is available and "cpu" otherwise; any other value is returned unchanged, except that an explicit "cuda" (or "cuda:<index>") without an available CUDA device with that index is an error rather than a failure deep inside libtorch later on.

Usage

auto_device(device = NULL)

Arguments

Batchgetter for Categorical data

Description

Converts a data frame of categorical data into a long tensor by converting the data to integers. No input checks are performed.

All columns are encoded with 1-based codes, i.e. the values of a feature with cardinality k are ⁠1, ..., k⁠.

Usage

batchgetter_categ(data, ...)

Arguments

Batchgetter for Numeric Data

Description

Converts a data frame of numeric data into a float tensor by calling as.matrix(). No input checks are performed

Usage

batchgetter_num(data, ...)

Arguments

Create a Set of Callbacks for Torch

Description

Creates an R6ClassGenerator inheriting from CallbackSet. Additionally performs checks such as that the stages are not accidentally misspelled. To create a TorchCallback use torch_callback().

In order for the resulting class to be cloneable, the private method ⁠$deep_clone()⁠ must be provided.

Usage

callback_set(
  classname,
  on_begin = NULL,
  on_end = NULL,
  on_exit = NULL,
  on_epoch_begin = NULL,
  on_before_valid = NULL,
  on_epoch_end = NULL,
  on_batch_begin = NULL,
  on_batch_end = NULL,
  on_after_backward = NULL,
  on_batch_valid_begin = NULL,
  on_batch_valid_end = NULL,
  on_valid_end = NULL,
  state_dict = NULL,
  load_state_dict = NULL,
  initialize = NULL,
  weight = NULL,
  public = NULL,
  private = NULL,
  active = NULL,
  parent_env = parent.frame(),
  inherit = CallbackSet,
  lock_objects = FALSE
)

Arguments

Value

CallbackSet

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Cross Entropy Loss

Description

The cross_entropy loss function selects the multi-class (nn_cross_entropy_loss) or binary (nn_bce_with_logits_loss) cross entropy loss based on the number of classes. Because of this, there is a slight reparameterization of the loss arguments, see Parameters.

Parameters

• class_weight:: torch_tensor
  The class weights. For multi-class problems, this must be a torch_tensor of length num_classes (and is passed as argument weight to nn_cross_entropy_loss). For binary problems, this must be a scalar (and is passed as argument pos_weight to nn_bce_with_logits_loss).

• ignore_index:: integer(1)
  Index of the class which to ignore and which does not contribute to the gradient. This is only available for multi-class loss.

• reduction :: character(1)
  The reduction to apply. Is either "mean" or "sum" and passed as argument reduction to either loss function. The default is "mean".

Examples

loss = t_loss("cross_entropy")
# multi-class
multi_ce = loss$generate(tsk("iris"))
multi_ce

# binary
binary_ce = loss$generate(tsk("sonar"))
binary_ce

Encode the Network Output as a Prediction

Description

Converts the raw output of a network into a list() that can be passed to mlr3::as_prediction_data(), which is what the private .encode_prediction() method of a LearnerTorch has to return.

This is the default implementation that is used by LearnerTorch and LearnerTorchModel, i.e. by all learners that don't overwrite .encode_prediction(). When adding support for a custom task type, implement a method for the corresponding Task class, which makes the generic torch learners work for that task type.

For the network output that is expected for the built-in task types, see section Network Head and Target Encoding of LearnerTorch.

Usage

encode_prediction(task, network_output, predict_type, ...)

Arguments

Value

named list()

Batch Constructor for a Task

Description

Returns the function that builds a whole batch of a task, i.e. both the features x and the target y. This is how task_dataset loads its batches.

The returned function takes the arguments

• data, a data.table with the feature and target columns of the batch, and

• cache, a rather internal hashtab that can be used as a cache when loading multiple lazy_tensor columns.

It returns a list() with elements x (a named list() of torch_tensors) and y (a torch_tensor or NULL).

The default method applies the ingress tokens to obtain x and the target batchgetter to obtain y.

Usage

get_batch_constructor(
  task,
  feature_ingress_tokens,
  target_batchgetter = NULL,
  ...
)

Arguments

Value

⁠function(data, cache) -> list(x = list<torch_tensor>, y = torch_tensor | NULL)⁠

Examples

task = tsk("iris")
token = TorchIngressToken(task$feature_names, batchgetter_num, c(NA, 4))
# the token's features are a `Selector`; `task_dataset()` resolves them for you
token$features = token$features(task)
batch_constructor = get_batch_constructor(
  task,
  feature_ingress_tokens = list(input = token),
  target_batchgetter = get_target_batchgetter(task)
)
batch = batch_constructor(data = task$data(rows = 1:2))
batch$x$input
batch$y

Extract the Batch Size for a Given Phase

Description

A LearnerTorch uses the batch_size parameter for both training and prediction, unless batch_size_predict is set, which then takes precedence during prediction. This helper resolves the batch size for one phase and is useful when overwriting the private .dataloader() method of a LearnerTorch.

Usage

get_batch_size(param_vals, phase)

Arguments

Value

(integer(1) or NULL)
The batch size for the given phase or NULL if none is set.

Examples

get_batch_size(list(batch_size = 16), "train")
get_batch_size(list(batch_size = 16, batch_size_predict = 32), "predict")
get_batch_size(list(batch_size_predict = 32), "train")

Target Batchgetter for a Task

Description

Returns the function that converts the target column(s) of a task into the target tensor y of a batch, i.e. the tensor that the loss is applied to. The returned function takes an argument data, a data.table containing only the target column(s), and returns a torch_tensor. It is NULL for a task with no target at all, whose batches have no y element and whose loss is called as loss(y_hat).

For the target encodings of the built-in task types, see section Network Head and Target Encoding of LearnerTorch.

When adding support for a custom task type, implement a method for the corresponding Task class.

Usage

get_target_batchgetter(task, ...)

Arguments

Value

⁠function(data)⁠, ⁠function(data, x)⁠, or NULL for a task with no target

Examples

batchgetter = get_target_batchgetter(tsk("iris"))
batchgetter(data.table::data.table(Species = factor(c("setosa", "virginica"))))

Infer Shapes

Description

Infer the shapes of the output of a function based on the shapes of the input. This works by running the function on the input and observing the results. For fully known input shapes this is always correct. For partially unknown shapes, the NAs are replaced with various concrete values and the output shape is computed from them. Note that this is a heuristic that might fail, so usually one wants to provide the shape (inference) explicitly.

Usage

infer_shapes(shapes_in, param_vals, output_names, fn, rowwise, id)

Arguments

Details

The inference is done as follows:

1. All NAs are replaced with three different values, which span a wide range: none of them is 1 (which broadcasts and is squeezed away), one of them is small (to detect operators that clamp to the input size, such as slicing or cropping) and the others are large (because operators such as a convolution with a large kernel need a minimum extent).

2. Three tensors are generated for the three shapes of step 1.

3. The function is called on these three tensors and the shapes are calculated. A call that fails is dropped, so that an operator is not rejected because of the smallest value; at least two of the three calls must succeed.

4. If:

   • the number of dimensions varies, an error is thrown.

   • the number of dimensions is the same, values are set to NA if the dimension is varying between the tensors and otherwise set to the unique value.

Value

(list())
A list of shapes of the output tensors.

See Also

Other Shape Inference: reshape_output_shape(), shape_to_str()

Ingress Token for Categorical Features

Description

Represents an entry point representing a tensor containing all categorical (factor(), ordered(), logical()) features of a task.

Usage

ingress_categ(shape = NULL)

Arguments

Value

TorchIngressToken

Ingress Token for Lazy Tensor Feature

Description

Represents an entry point representing a tensor containing a single lazy tensor feature.

Usage

ingress_ltnsr(feature_name = NULL, shape = NULL)

Arguments

Value

TorchIngressToken

Ingress Token for Numeric Features

Description

Represents an entry point representing a tensor containing all numeric (integer() and double()) features of a task.

Usage

ingress_num(shape = NULL)

Arguments

Value

TorchIngressToken

Check for lazy tensor

Description

Checks whether an object is a lazy tensor.

Usage

is_lazy_tensor(x)

Arguments

Shape of Lazy Tensor

Description

Shape of a lazy tensor. Might be NULL if the shapes is not known or varying between rows. Batch dimension is always NA.

Usage

lazy_shape(x)

Arguments

Value

(integer() or NULL)

Examples

lt = as_lazy_tensor(1:10)
lazy_shape(lt)
lt = as_lazy_tensor(matrix(1:10, nrow = 2))
lazy_shape(lt)

Create a lazy tensor

Description

Creates a lazy_tensor vector. Because it is a vector, it can be stored in a data.table, which gives mlr3torch the ability to use arbitrary tensors in its task. It is 'lazy', because the tensors are not stored in-memory, but only loaded when calling materialize(). The vector itself only describes how to load the data. It is also possible to preprocess lazy_tensors, e.g. via po("augment_<key>"), and po("trafo_<key>").

Usage

lazy_tensor(data_descriptor = NULL, ids = NULL)

Arguments

Examples

ds = dataset("example",
  initialize = function() self$iris = iris[, -5],
  .getitem = function(i) list(x = torch_tensor(as.numeric(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
dd = as_data_descriptor(ds, list(x = c(NA, 4L)))
lt = as_lazy_tensor(dd)

Materialize Lazy Tensor Columns

Description

This will materialize a lazy_tensor() or a data.frame() / list() containing – among other things – lazy_tensor() columns. I.e. the data described in the underlying DataDescriptors is loaded for the indices in the lazy_tensor(), is preprocessed and then put unto the specified device. Because not all elements in a lazy tensor must have the same shape, a list of tensors is returned by default. If all elements have the same shape, these tensors can also be rbinded into a single tensor (parameter rbind).

Usage

materialize(x, device = "cpu", rbind = FALSE, ...)

## S3 method for class 'list'
materialize(x, device = "cpu", rbind = FALSE, cache = "auto", ...)

Arguments

Details

Materializing a lazy tensor consists of:

1. Loading the data from the internal dataset of the DataDescriptor.

2. Processing these batches in the preprocessing Graphs.

3. Returning the result of the PipeOp pointed to by the DataDescriptor (pointer).

With multiple lazy_tensor columns we can benefit from caching because: a) Output(s) from the dataset might be input to multiple graphs. b) Different lazy tensors might be outputs from the same graph.

For this reason it is possible to provide a cache, which is a hashtab(). The key for a) is list(dataset, indices), the key for b) is list(indices, dataset, graph, input_map). The dataset and the graph go into the key as the objects themselves, so keys are compared with identical() rather than being digested into a string, and two different keys can never share an entry.

Value

(list() of torch_tensors or a torch_tensor)

Examples

lt1 = as_lazy_tensor(torch_randn(10, 3))
materialize(lt1, rbind = TRUE)
materialize(lt1, rbind = FALSE)
lt2 = as_lazy_tensor(torch_randn(10, 4))
d = data.table::data.table(lt1 = lt1, lt2 = lt2)
materialize(d, rbind = TRUE)
materialize(d, rbind = FALSE)

Materialize a Lazy Tensor

Description

Convert a lazy_tensor to a torch_tensor.

Usage

materialize_internal(x, device = "cpu", cache = NULL, rbind)

Arguments

Details

Materializing a lazy tensor consists of:

1. Loading the data from the internal dataset of the DataDescriptor.

2. Processing these batches in the preprocessing Graphs.

3. Returning the result of the PipeOp pointed to by the DataDescriptor (pointer).

When materializing multiple lazy_tensor columns, caching can be useful because: a) Output(s) from the dataset might be input to multiple graphs. (in task_dataset this should rarely be the case because we try to merge them). b) Different lazy tensors might be outputs from the same graph.

For this reason it is possible to provide a cache, which is a hashtab(). The key for a) is list(dataset, indices), the key for b) is list(indices, dataset, graph, input_map). The dataset and the graph go into the key as the objects themselves, not as their hashes, so they are compared with identical().

Value

lazy_tensor()

Dictionary of Torch Callbacks

Description

A mlr3misc::Dictionary of torch callbacks. Use t_clbk() to conveniently retrieve callbacks. Can be converted to a data.table using as.data.table.

Usage

mlr3torch_callbacks

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Other Dictionary: mlr3torch_losses, mlr3torch_optimizers, t_opt()

Examples

mlr3torch_callbacks$get("checkpoint")
# is the same as
t_clbk("checkpoint")
# convert to a data.table
as.data.table(mlr3torch_callbacks)

Loss Functions

Description

Dictionary of torch loss descriptors. See t_loss() for conveniently retrieving a loss function. Can be converted to a data.table using as.data.table.

Usage

mlr3torch_losses

Available Loss Functions

cross_entropy, l1, mse

See Also

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_callbacks(), as_torch_loss(), as_torch_optimizer(), mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Other Dictionary: mlr3torch_callbacks, mlr3torch_optimizers, t_opt()

Examples

mlr3torch_losses$get("mse")
# is equivalent to
t_loss("mse")
# convert to a data.table
as.data.table(mlr3torch_losses)

Optimizers

Description

Dictionary of torch optimizers. Use t_opt for conveniently retrieving optimizers. Can be converted to a data.table using as.data.table.

Usage

mlr3torch_optimizers

Available Optimizers

adagrad, adam, adamw, rmsprop, sgd

See Also

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_callbacks(), as_torch_loss(), as_torch_optimizer(), mlr3torch_losses, t_clbk(), t_loss(), t_opt()

Other Dictionary: mlr3torch_callbacks, mlr3torch_losses, t_opt()

Examples

mlr3torch_optimizers$get("adam")
# is equivalent to
t_opt("adam")
# convert to a data.table
as.data.table(mlr3torch_optimizers)

Lazy Data Backend

Description

This lazy data backend wraps a constructor that lazily creates another backend, e.g. by downloading (and caching) some data from the internet. This backend should be used, when some metadata of the backend is known in advance and should be accessible before downloading the actual data. When the backend is first constructed, it is verified that the provided metadata was correct, otherwise an informative error message is thrown. After the construction of the lazily constructed backend, calls like ⁠$data()⁠, ⁠$missings()⁠, ⁠$distinct()⁠, or ⁠$hash()⁠ are redirected to it.

Information that is available before the backend is constructed is:

• nrow - The number of rows (set as the length of the rownames).

• ncol - The number of columns (provided via the id column of col_info).

• colnames - The column names.

• rownames - The row names.

• col_info - The column information, which can be obtained via mlr3::col_info().

Beware that accessing the backend's hash also contructs the backend.

Note that while in most cases the data contains lazy_tensor columns, this is not necessary and the naming of this class has nothing to do with the lazy_tensor data type.

Important

When the constructor generates factor() variables it is important that the ordering of the levels in data corresponds to the ordering of the levels in the col_info argument.

Super class

mlr3::DataBackend -> DataBackendLazy

Active bindings

Methods

Public methods

• DataBackendLazy$new()

• DataBackendLazy$data()

• DataBackendLazy$head()

• DataBackendLazy$distinct()

• DataBackendLazy$missings()

• DataBackendLazy$print()

DataBackendLazy$new()

Creates a new instance of this R6 class.

Usage

DataBackendLazy$new(constructor, rownames, col_info, primary_key)

Arguments

DataBackendLazy$data()

Returns a slice of the data in the specified format. The rows must be addressed as vector of primary key values, columns must be referred to via column names. Queries for rows with no matching row id and queries for columns with no matching column name are silently ignored. Rows are guaranteed to be returned in the same order as rows, columns may be returned in an arbitrary order. Duplicated row ids result in duplicated rows, duplicated column names lead to an exception.

Accessing the data triggers the construction of the backend.

Usage

DataBackendLazy$data(rows, cols)

Arguments

DataBackendLazy$head()

Retrieve the first n rows. This triggers the construction of the backend.

Usage

DataBackendLazy$head(n = 6L)

Arguments

Returns

data.table::data.table() of the first n rows.

DataBackendLazy$distinct()

Returns a named list of vectors of distinct values for each column specified. If na_rm is TRUE, missing values are removed from the returned vectors of distinct values. Non-existing rows and columns are silently ignored.

This triggers the construction of the backend.

Usage

DataBackendLazy$distinct(rows, cols, na_rm = TRUE)

Arguments

Returns

Named list() of distinct values.

DataBackendLazy$missings()

Returns the number of missing values per column in the specified slice of data. Non-existing rows and columns are silently ignored.

This triggers the construction of the backend.

Usage

DataBackendLazy$missings(rows, cols)

Arguments

Returns

Total of missing values per column (named numeric()).

DataBackendLazy$print()

Printer.

Usage

DataBackendLazy$print()

Examples

# We first define a backend constructor
constructor = function(backend) {
  cat("Data is constructed!\n")
  DataBackendDataTable$new(
    data.table(x = rnorm(10), y = rnorm(10), row_id = 1:10),
    primary_key = "row_id"
  )
}

# to wrap this backend constructor in a lazy backend, we need to provide the correct metadata for it
column_info = data.table(
  id = c("x", "y", "row_id"),
  type = c("numeric", "numeric", "integer"),
  levels = list(NULL, NULL, NULL)
)
backend_lazy = DataBackendLazy$new(
  constructor = constructor,
  rownames = 1:10,
  col_info = column_info,
  primary_key = "row_id"
)

# Note that the constructor is not called for the calls below
# as they can be read from the metadata
backend_lazy$nrow
backend_lazy$rownames
backend_lazy$ncol
backend_lazy$colnames
col_info(backend_lazy)

# Only now the backend is constructed
backend_lazy$data(1, "x")
# Is the same as:
backend_lazy$backend$data(1, "x")

Base Class for Callbacks

Description

Base class from which callbacks should inherit (see section Inheriting). A callback set is a collection of functions that are executed at different stages of the training loop. They can be used to gain more control over the training process of a neural network without having to write everything from scratch.

When used in a torch learner, the CallbackSet is wrapped in a TorchCallback. The latter's parameter set represents the arguments of the CallbackSet's ⁠$initialize()⁠ method.

Inheriting

For each available stage (see section Stages) a public method ⁠$on_<stage>()⁠ can be defined. The evaluation context (a ContextTorch) can be accessed via self$ctx, which contains the current state of the training loop. This context is assigned at the beginning of the training loop and removed afterwards. Different stages of a callback can communicate with each other by assigning values to self.

State: To be able to store information in the ⁠$model⁠ slot of a LearnerTorch, callbacks support a state API. You can overload the ⁠$state_dict()⁠ public method to define what will be stored in ⁠learner$model$callbacks$<id>⁠ after training finishes. This then also requires to implement a ⁠$load_state_dict(state_dict)⁠ method that defines how to load a previously saved callback state into a different callback. Note that the ⁠$state_dict()⁠ should not include the parameter values that were used to initialize the callback.

In order to be able to resume a callback, it needs to work with saveRDS, so it cannot contain external pointers such as torch_tensors. Convert them to R data, e.g. via as.array().

For creating custom callbacks, the function torch_callback() is recommended, which creates a CallbackSet and then wraps it in a TorchCallback. To create a CallbackSet the convenience function callback_set() can be used. These functions perform checks such as that the stages are not accidentally misspelled.

Stages

• begin :: Run before the training loop begins.

• epoch_begin :: Run at the beginning of each epoch.

• batch_begin :: Run before the forward call.

• after_backward :: Run after the backward call.

• batch_end :: Run after the optimizer step.

• before_valid :: Run before the validation loop begins.

• batch_valid_begin :: Run before the forward call in the validation loop.

• batch_valid_end :: Run after the forward call in the validation loop.

• valid_end :: Run at the end of validation.

• epoch_end :: Run at the end of each epoch.

• end :: Run after last epoch.

• exit :: Run last, using on.exit().

Ordering

Within a stage, callbacks are called in the order in which they were passed to the learner. A callback can override this via its ⁠$weight⁠ field: callbacks with a higher weight are called after those with a lower one, and callbacks with the same weight keep the order in which they were passed. The default weight is 0. This matters for callbacks that observe what the others did, which is why CallbackSetCheckpoint has weight Inf: it always runs last and therefore saves the network and optimizer as the other callbacks left them at the end of the stage.

Terminate Training

If training is to be stopped, it is possible to set the field ⁠$terminate⁠ of ContextTorch. At the end of every epoch this field is checked and if it is TRUE, training stops. This can for example be used to implement custom early stopping.

Public fields

Active bindings

Methods

Public methods

• CallbackSet$print()

• CallbackSet$state_dict()

• CallbackSet$load_state_dict()

• CallbackSet$clone()

CallbackSet$print()

Prints the object.

Usage

CallbackSet$print(...)

Arguments

CallbackSet$state_dict()

Returns information that is kept in the LearnerTorch's state after training. This information should be loadable into the callback using ⁠$load_state_dict()⁠ to be able to continue training. This returns NULL by default.

Usage

CallbackSet$state_dict()

CallbackSet$load_state_dict()

Loads the state dict into the callback to continue training. A callback that keeps no state inherits this, which has nothing to load and therefore only warns when it is handed a state anyway – a callback that implements ⁠$state_dict()⁠ but not this method, so that a resuming run cannot give its state back.

Usage

CallbackSet$load_state_dict(state_dict)

Arguments

CallbackSet$clone()

The objects of this class are cloneable with this method.

Usage

CallbackSet$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Checkpoint Callback

Description

Saves the optimizer, weights, and callback states every freq epochs as well as the final state. This can be used to later continue a training run via the resume parameter of LearnerTorch.

Checkpoints are written at the end of an epoch. For one written after epoch ⁠<n>⁠, three files are created in path:

• ⁠network<n>.pt⁠ :: The ⁠$state_dict()⁠ of the network.

• ⁠optimizer<n>.pt⁠ :: The ⁠$state_dict()⁠ of the optimizer.

• ⁠state<n>.rds⁠ :: The epoch, the version of mlr3torch that wrote the checkpoint, the ⁠$state_dict()⁠s of the training run's other callbacks, so that a later run can continue, as well as some other information. Additionally, there is run.rds which contains some additioanl global meta information.

Details

Saving the learner itself in the callback with a trained model is impossible, as the model slot is set after the last callback step is executed.

Resuming

This callback is special because it enables resuming a training run. Its own state is the folder it writes to, which ⁠learner$model$callbacks$<id>$path⁠ reports – the only way to learn where a path function sent a run, e.g. one fit of a resample(). That state is not part of a checkpoint and is not restored: a resuming run writes where its own path says.

Ordering

This callback has weight Inf and therefore runs last, so it captures all the changes other callbacks made.

Super class

CallbackSet -> CallbackSetCheckpoint

Public fields

Methods

Public methods

• CallbackSetCheckpoint$new()

• CallbackSetCheckpoint$state_dict()

• CallbackSetCheckpoint$on_begin()

• CallbackSetCheckpoint$on_epoch_end()

• CallbackSetCheckpoint$on_end()

• CallbackSetCheckpoint$clone()

CallbackSetCheckpoint$new()

Creates a new instance of this R6 class.

Usage

CallbackSetCheckpoint$new(path, freq)

Arguments

CallbackSetCheckpoint$state_dict()

Returns the folder this callback writes to so it can be accessed from the learner when the path was a function.

Usage

CallbackSetCheckpoint$state_dict()

CallbackSetCheckpoint$on_begin()

Checks whether the checkpoint path is valid.

Usage

CallbackSetCheckpoint$on_begin()

CallbackSetCheckpoint$on_epoch_end()

Saves the network and optimizer state dict. Does nothing if freq is not met.

Usage

CallbackSetCheckpoint$on_epoch_end()

CallbackSetCheckpoint$on_end()

Saves the final network and optimizer, unless the last epoch was already saved.

Usage

CallbackSetCheckpoint$on_end()

CallbackSetCheckpoint$clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetCheckpoint$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Examples

cb = t_clbk("checkpoint", freq = 1)
task = tsk("iris")

pth = tempfile()
learner = lrn("classif.mlp", epochs = 3, batch_size = 1, callbacks = cb)
learner$param_set$set_values(cb.checkpoint.path = pth)

learner$train(task)

list.files(pth)

# continue training for 3 more epochs, starting from the last checkpoint
learner_resumed = lrn("classif.mlp", epochs = 6, batch_size = 1, resume = pth)
learner_resumed$train(task)
learner_resumed$model$epochs

History Callback

Description

Saves the training and validation history during training. The history is saved as a data.table where the validation measures are prefixed with "valid." and the training measures are prefixed with "train.".

Resuming

The epochs of a resumed run are appended to the history of the run it continues. A measure that only one of the two runs recorded is NA for the epochs of the other.

Super class

CallbackSet -> CallbackSetHistory

Methods

Public methods

• CallbackSetHistory$on_begin()

• CallbackSetHistory$state_dict()

• CallbackSetHistory$load_state_dict()

• CallbackSetHistory$on_before_valid()

• CallbackSetHistory$on_epoch_end()

• CallbackSetHistory$clone()

CallbackSetHistory$on_begin()

Initializes lists where the train and validation metrics are stored.

Usage

CallbackSetHistory$on_begin()

CallbackSetHistory$state_dict()

Converts the lists to data.tables.

Usage

CallbackSetHistory$state_dict()

CallbackSetHistory$load_state_dict()

Remembers the history contained in the state dict, so that the epochs of the current run are appended to it by ⁠$state_dict()⁠.

Usage

CallbackSetHistory$load_state_dict(state_dict)

Arguments

CallbackSetHistory$on_before_valid()

Add the latest training scores to the history.

Usage

CallbackSetHistory$on_before_valid()

CallbackSetHistory$on_epoch_end()

Add the latest validation scores to the history.

Usage

CallbackSetHistory$on_epoch_end()

CallbackSetHistory$clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetHistory$clone(deep = FALSE)

Arguments

Examples

cb = t_clbk("history")
task = tsk("iris")

learner = lrn("classif.mlp", epochs = 3, batch_size = 1,
  callbacks = t_clbk("history"), validate = 0.3)
learner$param_set$set_values(
  measures_train = msrs(c("classif.acc", "classif.ce")),
  measures_valid = msr("classif.ce")
)
learner$train(task)

print(learner$model$callbacks$history)

Learning Rate Scheduling Callback

Description

Changes the learning rate based on the schedule specified by a torch::lr_scheduler.

As of this writing, the following are available:

• torch::lr_cosine_annealing()

• torch::lr_lambda()

• torch::lr_multiplicative()

• torch::lr_one_cycle() (where the default values for epochs and steps_per_epoch are the number of training epochs and the number of batches per epoch)

• torch::lr_reduce_on_plateau()

• torch::lr_step()

• Custom schedulers defined with torch::lr_scheduler().

Resuming

The state of the wrapped torch scheduler is stored and restored, so a resumed run continues the schedule instead of starting it over. Creating a scheduler resets the optimizer's learning rate to the one the schedule started at, so the rate the restored schedule had reached is put back afterwards.

That state contains the scheduler's configuration as well as its progress, and restoring it overwrites what the resuming run was configured with. Resuming with different scheduler arguments, or a different opt.lr, which the schedule's base rates are derived from, therefore silently continues the schedule of the checkpointed run.

Super class

CallbackSet -> CallbackSetLRScheduler

Public fields

Methods

Public methods

• CallbackSetLRScheduler$new()

• CallbackSetLRScheduler$on_begin()

• CallbackSetLRScheduler$state_dict()

• CallbackSetLRScheduler$load_state_dict()

• CallbackSetLRScheduler$clone()

CallbackSetLRScheduler$new()

Creates a new instance of this R6 class.

Usage

CallbackSetLRScheduler$new(.scheduler, step_on_epoch, ...)

Arguments

CallbackSetLRScheduler$on_begin()

Creates the scheduler using the optimizer from the context

Usage

CallbackSetLRScheduler$on_begin()

CallbackSetLRScheduler$state_dict()

Returns the state of the wrapped torch scheduler, so that a later run can continue the schedule instead of starting it over. Returns NULL if the scheduler was not created yet, i.e. before the training loop began.

Usage

CallbackSetLRScheduler$state_dict()

CallbackSetLRScheduler$load_state_dict()

Loads the state of the wrapped torch scheduler.

Usage

CallbackSetLRScheduler$load_state_dict(state_dict)

Arguments

CallbackSetLRScheduler$clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetLRScheduler$clone(deep = FALSE)

Arguments

OneCycle Learning Rate Scheduling Callback

Description

Changes the learning rate based on the 1cycle learning rate policy.

Wraps torch::lr_one_cycle(), where the default values for epochs and steps_per_epoch are the number of training epochs and the number of batches per epoch.

Resuming

As for CallbackSetLRScheduler, with one additional restriction: the 1cycle policy is defined over the total number of steps of the run, so a resumed run must be configured for the same number of steps as the one that wrote the checkpoint, i.e. the same epochs and the same number of batches per epoch. A run that is not errors before its first epoch rather than somewhere in the middle.

Super classes

CallbackSet -> CallbackSetLRScheduler -> CallbackSetLRSchedulerOneCycle

Methods

Public methods

• CallbackSetLRSchedulerOneCycle$new()

• CallbackSetLRSchedulerOneCycle$on_begin()

• CallbackSetLRSchedulerOneCycle$clone()

CallbackSetLRSchedulerOneCycle$new()

Creates a new instance of this R6 class.

Usage

CallbackSetLRSchedulerOneCycle$new(...)

Arguments

CallbackSetLRSchedulerOneCycle$on_begin()

Creates the scheduler using the optimizer from the context

Usage

CallbackSetLRSchedulerOneCycle$on_begin()

CallbackSetLRSchedulerOneCycle$clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetLRSchedulerOneCycle$clone(deep = FALSE)

Arguments

Reduce On Plateau Learning Rate Scheduler

Description

Reduces the learning rate when the first validation measure, i.e. the first entry of the learner's measures_valid, stops improving for patience evaluation steps. Wraps torch::lr_reduce_on_plateau().

Note that patience counts evaluation steps, not epochs: when the learner's eval_freq is greater than 1, the scheduler only steps in the epochs in which the validation measures are evaluated, so patience evaluation steps correspond to patience * eval_freq epochs.

Unless mode is set, it is derived from the measure: "min" if the measure is minimized and "max" if it is maximized. Training fails when mode contradicts the measure. When the learner has no validation measure, e.g. in the final fit of an AutoTuner that tunes the epochs internally, a warning is given and the learning rate is not reduced.

Resuming

As for CallbackSetLRScheduler. For this schedule the restored state includes the best score seen so far and how long it has been stagnating, so patience keeps counting across runs instead of starting over.

Super classes

CallbackSet -> CallbackSetLRScheduler -> CallbackSetLRSchedulerReduceOnPlateau

Methods

Public methods

• CallbackSetLRSchedulerReduceOnPlateau$new()

• CallbackSetLRSchedulerReduceOnPlateau$on_begin()

• CallbackSetLRSchedulerReduceOnPlateau$clone()

CallbackSetLRSchedulerReduceOnPlateau$new()

Creates a new instance of this R6 class.

Usage

CallbackSetLRSchedulerReduceOnPlateau$new(...)

Arguments

CallbackSetLRSchedulerReduceOnPlateau$on_begin()

Creates the scheduler using the optimizer from the context, with the mode that fits the first validation measure.

Usage

CallbackSetLRSchedulerReduceOnPlateau$on_begin()

CallbackSetLRSchedulerReduceOnPlateau$clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetLRSchedulerReduceOnPlateau$clone(deep = FALSE)

Arguments

Progress Callback

Description

Prints a progress bar and the metrics for training and validation.

Resuming

This callback can be resumed without any problems.

Super class

CallbackSet -> CallbackSetProgress

Methods

Public methods

• CallbackSetProgress$new()

• CallbackSetProgress$on_begin()

• CallbackSetProgress$on_epoch_begin()

• CallbackSetProgress$on_batch_end()

• CallbackSetProgress$on_before_valid()

• CallbackSetProgress$on_batch_valid_end()

• CallbackSetProgress$on_epoch_end()

• CallbackSetProgress$on_end()

• CallbackSetProgress$state_dict()

• CallbackSetProgress$load_state_dict()

• CallbackSetProgress$clone()

CallbackSetProgress$new()

Creates a new instance of this R6 class.

Usage

CallbackSetProgress$new(digits = 2)

Arguments

CallbackSetProgress$on_begin()

Starts this run's timer.

Usage

CallbackSetProgress$on_begin()

CallbackSetProgress$on_epoch_begin()

Initializes the progress bar for training.

Usage

CallbackSetProgress$on_epoch_begin()

CallbackSetProgress$on_batch_end()

Increments the training progress bar.

Usage

CallbackSetProgress$on_batch_end()

CallbackSetProgress$on_before_valid()

Creates the progress bar for validation.

Usage

CallbackSetProgress$on_before_valid()

CallbackSetProgress$on_batch_valid_end()

Increments the validation progress bar.

Usage

CallbackSetProgress$on_batch_valid_end()

CallbackSetProgress$on_epoch_end()

Prints a summary of the training and validation process.

Usage

CallbackSetProgress$on_epoch_end()

CallbackSetProgress$on_end()

Prints the time at the end of training, and how long training took in total. A resumed run also reports how much of that total it contributed itself.

Usage

CallbackSetProgress$on_end()

CallbackSetProgress$state_dict()

Returns the seconds trained so far, so that a resumed run reports the time of all runs together rather than only its own.

Usage

CallbackSetProgress$state_dict()

CallbackSetProgress$load_state_dict()

Loads the time that the previous runs took.

Usage

CallbackSetProgress$load_state_dict(state_dict)

Arguments

CallbackSetProgress$clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetProgress$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Examples

task = tsk("iris")

learner = lrn("classif.mlp", epochs = 5, batch_size = 1,
  callbacks = t_clbk("progress"), validate = 0.3)
learner$param_set$set_values(
  measures_train = msrs(c("classif.acc", "classif.ce")),
  measures_valid = msr("classif.ce")
)

learner$train(task)

TensorBoard Logging Callback

Description

Logs training loss, training measures, and validation measures as events. To view them, use TensorBoard with tensorflow::tensorboard() (requires tensorflow) or the CLI.

Details

Logs events at most every epoch.

Resuming

This callback keeps no state of its own so it can trivially be resumed.

Super class

CallbackSet -> CallbackSetTB

Methods

Public methods

• CallbackSetTB$new()

• CallbackSetTB$on_epoch_end()

• CallbackSetTB$clone()

CallbackSetTB$new()

Creates a new instance of this R6 class.

Usage

CallbackSetTB$new(path, log_train_loss)

Arguments

CallbackSetTB$on_epoch_end()

Logs the training loss, training measures, and validation measures as TensorBoard events.

Usage

CallbackSetTB$on_epoch_end()

CallbackSetTB$clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetTB$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Unfreezing Weights Callback

Description

Unfreeze some weights (parameters of the network) after some number of steps or epochs.

Resuming

Which weights are trainable is stored and restored, so a resumed run does not freeze again what the run it continues had already unfrozen: starting_weights is applied first, and the restored weights are unfrozen on top of it.

Super class

CallbackSet -> CallbackSetUnfreeze

Methods

Public methods

• CallbackSetUnfreeze$new()

• CallbackSetUnfreeze$on_begin()

• CallbackSetUnfreeze$state_dict()

• CallbackSetUnfreeze$load_state_dict()

• CallbackSetUnfreeze$on_epoch_begin()

• CallbackSetUnfreeze$on_batch_begin()

• CallbackSetUnfreeze$clone()

CallbackSetUnfreeze$new()

Creates a new instance of this R6 class.

Usage

CallbackSetUnfreeze$new(starting_weights, unfreeze)

Arguments

CallbackSetUnfreeze$on_begin()

Sets the starting weights

Usage

CallbackSetUnfreeze$on_begin()

CallbackSetUnfreeze$state_dict()

Returns the names of the weights that are currently trainable, so that a later run does not freeze weights again that were already unfrozen.

Usage

CallbackSetUnfreeze$state_dict()

CallbackSetUnfreeze$load_state_dict()

Marks the weights of the state dict as trainable.

Usage

CallbackSetUnfreeze$load_state_dict(state_dict)

Arguments

CallbackSetUnfreeze$on_epoch_begin()

Unfreezes weights if the training is at the correct epoch

Usage

CallbackSetUnfreeze$on_epoch_begin()

CallbackSetUnfreeze$on_batch_begin()

Unfreezes weights if the training is at the correct batch

Usage

CallbackSetUnfreeze$on_batch_begin()

CallbackSetUnfreeze$clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetUnfreeze$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_context_torch, t_clbk(), torch_callback()

Examples

task = tsk("iris")
cb = t_clbk("unfreeze")
mlp = lrn("classif.mlp", callbacks = cb,
 cb.unfreeze.starting_weights = select_invert(
   select_name(c("0.weight", "3.weight", "6.weight", "6.bias"))
 ),
 cb.unfreeze.unfreeze = data.table(
   epoch = c(2, 5),
   weights = list(select_name("0.weight"), select_name(c("3.weight", "6.weight")))
 ),
 epochs = 6, batch_size = 150, neurons = c(1, 1, 1)
)

mlp$train(task)

Context for Torch Learner

Description

Context for training a torch learner. This is the - mostly read-only - information callbacks have access to through the argument ctx. For more information on callbacks, see CallbackSet.

Public fields

Active bindings

Methods

Public methods

• ContextTorch$new()

• ContextTorch$clone()

ContextTorch$new()

Creates a new instance of this R6 class.

Usage

ContextTorch$new(
  learner,
  task_train,
  task_valid = NULL,
  loader_train,
  loader_valid = NULL,
  measures_train = NULL,
  measures_valid = NULL,
  network,
  optimizer,
  loss_fn,
  total_epochs,
  prediction_encoder,
  eval_freq = 1L,
  device
)

Arguments

ContextTorch$clone()

The objects of this class are cloneable with this method.

Usage

ContextTorch$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, t_clbk(), torch_callback()

FT-Transformer

Description

Feature-Tokenizer Transformer for tabular data that can either work on lazy_tensor inputs or on standard tabular features.

Some differences from the paper implementation: no attention compression, no option to have prenormalization in the first layer.

If training is unstable, consider a combination of standardizing features (e.g. using po("scale")), using an adaptive optimizer (e.g. Adam), reducing the learning rate, and using a learning rate scheduler (see CallbackSetLRScheduler for options).

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.ft_transformer", ...)
lrn("regr.ft_transformer", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “logical”, “integer”, “numeric”, “factor”, “ordered”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Parameters from LearnerTorch and PipeOpTorchFTTransformerBlock, as well as:

• n_blocks :: integer(1)
  The number of transformer blocks.

• d_token :: integer(1)
  The dimension of the embedding.

• cardinalities :: integer(1)
  The number of categories for each categorical feature. This only needs to be specified when working with lazy_tensor inputs.

• init_token :: character(1)
  The initialization method for the embedding weights. Either "uniform" or "normal". "Uniform" by default.

• ingress_tokens :: named list() or NULL
  A list of TorchIngressTokens. Only required when using lazy tensor features. The names are either "num.input" or "categ.input", and the values are lazy tensor ingress tokens constructed by, e.g. ⁠ingress_ltnsr(<num_feat_name>)⁠.

Super classes

mlr3::Learner -> LearnerTorch -> LearnerTorchFTTransformer

Methods

Public methods

• LearnerTorchFTTransformer$new()

• LearnerTorchFTTransformer$clone()

LearnerTorchFTTransformer$new()

Creates a new instance of this R6 class.

Usage

LearnerTorchFTTransformer$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

LearnerTorchFTTransformer$clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchFTTransformer$clone(deep = FALSE)

Arguments

References

Gorishniy Y, Rubachev I, Khrulkov V, Babenko A (2021). “Revisiting Deep Learning for Tabular Data.” arXiv, 2106.11959.

See Also

Other Learner: GraphLearnerTorch, as_learner_torch(), mlr_learners.mlp, mlr_learners.module, mlr_learners.realmlp, mlr_learners.tab_resnet, mlr_learners.tabm, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.ft_transformer")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu",
  n_blocks = 2, d_token = 32, ffn_d_hidden_multiplier = 4/3
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

Multi Layer Perceptron

Description

Fully connected feed forward network with dropout after each activation function. The features can either be a single lazy_tensor or one or more numeric columns (but not both).

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.mlp", ...)
lrn("regr.mlp", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “integer”, “numeric”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Parameters from LearnerTorch, as well as:

• activation :: nn_module
  The activation function. Is initialized to nn_relu.

• activation_args :: named list()
  A named list with initialization arguments for the activation function. This is intialized to an empty list.

• neurons :: integer()
  The number of neurons per hidden layer. By default there is no hidden layer. Setting this to c(10, 20) would have a the first hidden layer with 10 neurons and the second with 20.

• n_layers :: integer()
  The number of layers. This parameter must only be set when neurons has length 1.

• p :: numeric(1)
  The dropout probability. Is initialized to 0.1.

• shape :: integer() or NULL
  The input shape of length 2, e.g. c(NA, 5). Only needs to be present when there is a lazy tensor input with unknown shape (NULL). Otherwise the input shape is inferred from the number of numeric features.

Super classes

mlr3::Learner -> LearnerTorch -> LearnerTorchMLP

Methods

Public methods

• LearnerTorchMLP$new()

• LearnerTorchMLP$clone()

LearnerTorchMLP$new()

Creates a new instance of this R6 class.

Usage

LearnerTorchMLP$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

LearnerTorchMLP$clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchMLP$clone(deep = FALSE)

Arguments

References

Gorishniy Y, Rubachev I, Khrulkov V, Babenko A (2021). “Revisiting Deep Learning for Tabular Data.” arXiv, 2106.11959.

See Also

Other Learner: GraphLearnerTorch, as_learner_torch(), mlr_learners.ft_transformer, mlr_learners.module, mlr_learners.realmlp, mlr_learners.tab_resnet, mlr_learners.tabm, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.mlp")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu",
  neurons = 10
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

Learner Torch Module

Description

Wraps a torch::nn_module into a LearnerTorch, so an architecture written directly in torch can be trained, resampled and tuned like any other mlr3 learner without going through a Graph of PipeOpTorch operators. This is the shortest path from an existing torch model to mlr3, and the natural choice for an architecture whose forward pass is easier to write as code than to assemble from operators. The module receives the task's features as its arguments, which ingress_tokens maps.

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.module", ...)
lrn("regr.module", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “logical”, “integer”, “numeric”, “character”, “factor”, “ordered”, “POSIXct”, “Date”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Super classes

mlr3::Learner -> LearnerTorch -> LearnerTorchModule

Methods

Public methods

• LearnerTorchModule$new()

• LearnerTorchModule$clone()

LearnerTorchModule$new()

Creates a new instance of this R6 class.

Usage

LearnerTorchModule$new(
  module_generator = NULL,
  param_set = NULL,
  ingress_tokens = NULL,
  task_type,
  properties = NULL,
  optimizer = NULL,
  loss = NULL,
  callbacks = list(),
  packages = character(0),
  feature_types = NULL,
  predict_types = NULL,
  target_batchgetter = NULL
)

Arguments

LearnerTorchModule$clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchModule$clone(deep = FALSE)

Arguments

See Also

Other Learner: GraphLearnerTorch, as_learner_torch(), mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.realmlp, mlr_learners.tab_resnet, mlr_learners.tabm, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

nn_one_layer = nn_module("nn_one_layer",
  initialize = function(task, size_hidden) {
    self$first = nn_linear(task$n_features, size_hidden)
    self$second = nn_linear(size_hidden, output_dim_for(task))
  },
  # argument x corresponds to the ingress token x
  forward = function(x) {
    x = self$first(x)
    x = nnf_relu(x)
    self$second(x)
  }
)
learner = lrn("classif.module",
  module_generator = nn_one_layer,
  ingress_tokens = list(x = ingress_num()),
  epochs = 10,
  size_hidden = 20,
  batch_size = 16
)
task = tsk("iris")
learner$train(task)
learner$network

RealMLP

Description

RealMLP is an MLP for tabular data with tuned defaults (RealMLP-TD), introduced by Holzmüller (2024). The learner implements the preprocessing, architecture and training recipe of the reference implementation.

It is a special learner, as it implements it's own learning rate scheduling per parameter group, as well as custom weight decay handling. Configuring a custom learning reate scheduler as a callback or setting the weight decay of an optimizer will therefore lead to unexpected results.

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.realmlp", ...)
lrn("regr.realmlp", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “logical”, “integer”, “numeric”, “factor”, “ordered”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Changed defaults of LearnerTorch:

• epochs = 256

• batch_size = 256

• batch_size_predict = 1024

• drop_last = TRUE and the optimizer is adam with

• betas = c(0.9, 0.95)

• lr = 0.04 (classification) or lr = 0.2 (regression).

Parameters from LearnerTorch, as well as:

• n_hidden_layers :: integer(1)
  The number of hidden layers. Default is 3.

• hidden_width :: integer(1)
  The width of the hidden layers. Default is 256.

• act :: character(1)
  The activation function, one of "selu", "mish", "relu", "silu", "gelu" and "elu". Default is "selu" for classification and "mish" for regression.

• use_parametric_act :: logical(1)
  Whether to use parametric activations x + \alpha (f(x) - x) with a learned \alpha per neuron, initialized with one. Default is TRUE.

• p_drop :: numeric(1)
  The dropout probability. Default is 0.15.

• p_drop_sched :: character(1)
  The schedule of the dropout probability. Default is "flat_cos".

• wd :: numeric(1)
  The decoupled weight decay. Biases are not decayed. Default is 0.02.

• wd_sched :: character(1)
  The schedule of the weight decay. Default is "flat_cos".

• lr_sched :: character(1)
  The schedule of the learning rate. Default is "coslog4".

• bias_lr_factor, act_lr_factor, scale_lr_factor, plr_lr_factor :: numeric(1)
  The learning rate factors of the biases (default 0.1), the parametric activations (default 0.1), the scaling layer (default 6) and the periodic embeddings (default 0.1). All other parameters have the factor 1.

• add_front_scale :: logical(1)
  Whether the first layer starts with a learned elementwise scaling. Default is TRUE.

• num_emb_type :: character(1)
  The embedding of the numerical features, one of:

  • "pbld" (default): \mathrm{concat}(W_2 \cos(2 \pi w_1 x + b_1) + b_2, x).

  • "pblrd": as "pbld", with a ReLU after the second linear layer.

  • "pl": W_2 \mathrm{concat}(\cos(2 \pi w_1 x), \sin(2 \pi w_1 x)) + b_2.

  • "plr": as "pl", with a ReLU after the second linear layer.

  • "none": no embedding.

• plr_sigma :: numeric(1)
  The standard deviation of the initialization of the frequencies w_1. Default is 0.1.

• plr_hidden_1 :: integer(1)
  The number of frequencies ("pbld", "pblrd") or twice the number of frequencies ("pl", "plr", must be even) per feature. Default is 16.

• plr_hidden_2 :: integer(1)
  The embedding size per feature, including the appended input of "pbld" and "pblrd". Default is 4.

• max_one_hot_cat_size :: integer(1)
  Categorical features with at most max_one_hot_cat_size - 1 levels are one-hot encoded, all others are embedded. Default is 9.

• embedding_size :: integer(1)
  The embedding size of the categorical features. Default is 8.

Only for classification:

• ls_eps :: numeric(1)
  The label smoothing parameter. Requires the cross_entropy loss. The smoothing term ignores the loss's class_weight and ignore_index. Default is 0.1.

Only for regression:

• normalize_output :: logical(1)
  Whether to standardize the target during training. Default is TRUE.

• clamp_output :: logical(1)
  Whether to clamp predictions to the range of the training target. Default is TRUE.

The schedules, as functions of the fraction t of training steps already taken, are:

• "coslog4": (1 - \cos(2 \pi \log_2(1 + 15 t))) / 2, four cycles of increasing length.

• "flat_cos": one for t < 0.5, followed by a cosine decay to zero.

• "cos": a cosine decay from one to zero.

• "linear": 1 - t.

• "constant": one.

Super classes

mlr3::Learner -> LearnerTorch -> LearnerTorchRealMLP

Methods

Public methods

• LearnerTorchRealMLP$new()

• LearnerTorchRealMLP$clone()

LearnerTorchRealMLP$new()

Creates a new instance of this R6 class.

Usage

LearnerTorchRealMLP$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

LearnerTorchRealMLP$clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchRealMLP$clone(deep = FALSE)

Arguments

References

Holzmüller D, Grinsztajn L, Steinwart I (2024). “Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data.” In Advances in Neural Information Processing Systems (NeurIPS), volume 37. 2407.04491.

See Also

Other Learner: GraphLearnerTorch, as_learner_torch(), mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.module, mlr_learners.tab_resnet, mlr_learners.tabm, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.realmlp")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu",
  n_hidden_layers = 2, hidden_width = 32
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

Tabular ResNet

Description

A residual network for tabular data, following Gorishniy (2021).

The features are first projected to dimension d_block by a linear layer, followed by n_blocks residual blocks. Each block computes x + Dropout(Linear(Dropout(ReLU(Linear(BatchNorm(x)))))), where the two dropout ratios are dropout1 and dropout2. The prediction is Linear(ReLU(BatchNorm(x))) of the output x of the last block.

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.tab_resnet", ...)
lrn("regr.tab_resnet", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “integer”, “numeric”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Parameters from LearnerTorch, as well as:

• n_blocks :: integer(1)
  The number of blocks.

• d_block :: integer(1)
  The input and output dimension of a block.

• d_hidden :: integer(1)
  The latent dimension of a block.

• d_hidden_multiplier :: numeric(1)
  Alternative way to specify the latent dimension as d_block * d_hidden_multiplier.

• dropout1 :: numeric(1)
  First dropout ratio.

• dropout2 :: numeric(1)
  Second dropout ratio.

• shape :: integer() or NULL
  Shape of the input tensor. Only needs to be provided if the input is a lazy tensor with unknown shape.

Super classes

mlr3::Learner -> LearnerTorch -> LearnerTorchTabResNet

Methods

Public methods

• LearnerTorchTabResNet$new()

• LearnerTorchTabResNet$clone()

LearnerTorchTabResNet$new()

Creates a new instance of this R6 class.

Usage

LearnerTorchTabResNet$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

LearnerTorchTabResNet$clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchTabResNet$clone(deep = FALSE)

Arguments

References

Gorishniy Y, Rubachev I, Khrulkov V, Babenko A (2021). “Revisiting Deep Learning for Tabular Data.” arXiv, 2106.11959.

See Also

Other Learner: GraphLearnerTorch, as_learner_torch(), mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.module, mlr_learners.realmlp, mlr_learners.tabm, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.tab_resnet")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu",
  n_blocks = 2, d_block = 10, d_hidden = 20, dropout1 = 0.3, dropout2 = 0.3
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

TabM

Description

TabM is an MLP-based tabular deep learning model that represents an ensemble of k MLPs: the k submodels are trained in parallel on the same batches and share most of their weights. The network produces k predictions per observation; the learner averages the predicted probabilities (classification) or the predicted values (regression) over the k submodels, and its loss function trains all k submodels jointly.

Numerical features are used as-is, or – if the num_embeddings parameter is set – embedded feature-wise first. Categorical features are one-hot encoded.

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.tabm", ...)
lrn("regr.tabm", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “logical”, “integer”, “numeric”, “factor”, “ordered”

• Required Packages: mlr3, mlr3torch, torch