Package {nestedscales}

Title:	Nested Scales and Multi-Dimensional Indexes for Modeling
Version:	0.2.0
URL:	https://optimal2050.github.io/nestedscales/r/, https://github.com/optimal2050/nestedscales
BugReports:	https://github.com/optimal2050/nestedscales/issues
Description:	Provides multi-level discrete representations of data dimensions for modeling, data processing and visualization. A nested scale records its finest units (atoms) and their membership in coarser levels (frames), including nested, cross-cutting and partially covered relationships. Recasts tabular data between levels using explicit, direction-aware rules for sums, means, standard deviations, copying and within-parent shares. Also supports labeling, subsetting, combining scales into multi-dimensional indexes, clustering units and storing data with its scale metadata. Members may carry attributes (a calorific value, an emission factor) and aliases, resolved along the hierarchy and applied row by row, and a scale is exchanged as a language-neutral definition file: a 'JSON' header with inline, 'Arrow' or 'CSV' tables ('YAML' accepted), with a catalogue of the definitions installed packages ship. Forms the dimension-agnostic core of 'timescales' https://github.com/optimal2050/timescales (calendars) and 'geoscales' https://github.com/optimal2050/geoscales (regions), designed for optimization and simulation models.
License:	Apache License (≥ 2)
Encoding:	UTF-8
Depends:	R (≥ 4.2)
Imports:	dplyr, jsonlite, rlang, S7, stats, utils
Suggests:	arrow, cluster, data.table, dtplyr, ggplot2, knitr, pkgdown, rmarkdown, testthat (≥ 3.0.0), tibble, withr, yaml
Config/testthat/edition:	3
Config/Needs/website:	pkgdown, rmarkdown, ggplot2, optimal2050/timescales, optimal2050/geoscales, optimal2050/energypal
VignetteBuilder:	knitr
Config/roxygen2/version:	8.1.1
Collate:	'scale-class.R' 'aliases.R' 'product-class.R' 'array.R' 'attributes.R' 'backend.R' 'definition.R' 'catalog.R' 'cluster.R' 'contiguity.R' 'convert.R' 'dataset.R' 'distance.R' 'navigate.R' 'filter.R' 'frames.R' 'join.R' 'map.R' 'nestedscales-package.R' 'recast.R' 'product-recast.R' 'reconcile.R' 'rules.R' 'scale-construct.R' 'shipped-scales.R' 'sweep.R' 'utils.R' 'zzz.R'
NeedsCompilation:	no
Packaged:	2026-10-22 01:32:14 UTC; admin
Author:	Oleg Lugovoy [aut, cre]
Maintainer:	Oleg Lugovoy <olugovoy@optimalsolution.dev>
Repository:	CRAN
Date/Publication:	2026-11-03 16:00:16 UTC

nestedscales: Nested Scales and Multi-Dimensional Indexes for Modeling

Description

A multi-level discrete representation of a data dimension, for modeling, data processing and visualization. A nested scale is a flat table of atoms plus the frames that group them; every frame nests in the atoms, and frames may nest in or overlay each other. The package defines the transition rules between levels (sum, weighted mean, mean, copy, sd, share) and the operations on them: conversion of data between resolutions, labelling, subsetting, combination of several scales into a product index, and construction of new frames by clustering.

Details

Time and space live in dimension packages built on this one, designed for optimization and simulation models where labeled time slices and regions are the index sets: timescales' Calendar and geoscales' Geoscale are subclasses of NestedScale, so every verb here accepts them. Any other dimension – industries, income brackets, temperature regimes, technology vintages – is declared directly with scale_from_leaftable(). vignette("nestedscales", package = "nestedscales") introduces the functions; the package website includes a roadmap of what is available and what is planned.

A clustering is treated as scale construction: cluster_scale() returns the scale with a new coarser frame inserted above the one clustered, so recasting to clusters, within-cluster shares, coverage and products all work with no special casing. scale_distance() and its registry supply the metrics, cluster_contiguous() the order- and adjacency-constrained variants, and cluster_sweep() reports quality across k without choosing one for you.

Author(s)

Maintainer: Oleg Lugovoy olugovoy@optimalsolution.dev (ORCID)

Authors:

• Oleg Lugovoy olugovoy@optimalsolution.dev (ORCID)

See Also

Useful links:

• https://optimal2050.github.io/nestedscales/r/

• https://github.com/optimal2050/nestedscales

• Report bugs at https://github.com/optimal2050/nestedscales/issues

NestedScale (S7 class)

Description

A nested discrete partition: a flat table of atoms plus the ordered hierarchy of frames that groups them. The dimension is not named – time and space are the subclasses Calendar (timescales) and Geoscale (geoscales); anything else (industries, income brackets, temperature regimes, technology vintages) is declared directly.

Usage

NestedScale(
  leaftable,
  frames,
  members,
  key = "unit",
  meta = list(),
  attributes = NULL,
  aliases = NULL
)

Arguments

Details

Construct with scale_from_leaftable().

Value

A NestedScale object.

See Also

scale_from_leaftable(), scale_frames(), scale_units()

Examples

scale_example()

Kinds of attribute

Description

Every attribute name carries a kind, which convert_scale() consults:

Usage

SCALE_ATTRIBUTE_KINDS

Format

A character vector of length 2.

Details

property

A physical property of the member, an invertible ratio: a calorific value in GJ/t converts tonnes to gigajoules and gigajoules back to tonnes; a density, a yield.

factor

A one-way multiplier: an emission factor in kg/TJ turns energy into CO2, and there is no physical sense in which CO2 turns back into energy; a loss factor, a tax rate.

Value

A character vector of the two attribute kinds accepted in the kind column of attach_attributes_scale(): "property" and "factor".

Examples

SCALE_ATTRIBUTE_KINDS

Built-in distances

Description

euclidean, manhattan, maximum

Straight to stats::dist(). Compare units by the LEVEL of their profiles.

correlation

1 - cor(t(x)), Pearson. Compares SHAPE: two units that rise and fall together are close however different their magnitudes, which is what you usually want when grouping load or generation profiles.

spearman

The same on ranks – shape, robust to outliers.

cosine

⁠1 - cosine similarity⁠. Shape again, but sensitive to sign and not centred, so a unit that is uniformly positive is close to every other such unit.

Usage

SCALE_DISTANCES

Format

A character vector.

Value

A character vector of the six built-in distance names accepted by the method argument of scale_distance(); list_scale_distances() adds any registered with register_scale_distance().

Examples

SCALE_DISTANCES

Supported aggregation rules

Description

Each rule defines behaviour in both directions. Direction is taken from the frame ranks, so aggregation and disaggregation are one operation:

Usage

SCALE_RULES

Format

A character vector of length 7.

Details

sum

Up: sum. Down: split proportionally to the weight. For extensive quantities (capacity, demand, area, population).

weighted_mean

Up: weight-weighted mean. Down: copy unchanged. For intensive quantities (efficiency, price, capacity factor).

mean

Up: unweighted mean. Down: copy unchanged.

copy

Up: the common value, erroring if it is not constant. Down: copy unchanged. For scale-invariant scalars.

sd

Up: standard deviation over the atoms (aggregation only; going down it degenerates to NA for single-atom groups).

share

Share within parent: each source unit's value divided by the total over its parent group. Unlike every other rule the result stays keyed at the source frame – the recast target (or ⁠parent=⁠) names the parent – so it cannot be mixed with other rules in one call (the two share rules mix freely, being one computation). For building distribution keys and normalised profiles; requires from to nest within the parent.

logshare

The same computation as share – the values ARE shares – but figures draw it on a fixed log10 percent scale (0.01%..100%), where share gets a fixed linear 0..1 scale.

Value

A character vector of the seven rule names accepted by the rule argument of recast_scale() and register_scale_rule().

Examples

SCALE_RULES

A product of several scales (S7 class)

Description

Combines two or more NestedScale objects into one multi-dimensional index – time x space, time x space x industry, and so on. The atoms of the product are tuples of the component atoms, and they stay LAZY: nothing is materialised at construction, and product_atoms() is the guarded way to realise them.

Usage

ScaleProduct(axes, keys, joint_weights = NULL, meta = list())

Arguments

Details

Construct with scale_product().

Value

A ScaleProduct object.

See Also

scale_product(), product_atoms(), product_size()

Examples

scale_product(a = scale_example(), b = scale_example2())

Subset a NestedScale

Description

x[frame, unit] is filter_scale().

Usage

## S3 method for class 'NestedScale'
x[i, j, ...]

## S3 method for class ''nestedscales::NestedScale''
x[i, j, ...]

Arguments

Value

A NestedScale.

Examples

scale_example()["sector", "P"]

Data on scales as a dense array

Description

Reshapes a long table into an array whose dimensions are the scale's axes, in the order the product declares them, with the units as dimnames.

Usage

as_scale_array(data, x, value = NULL, frames = NULL, fill = NA, limit = 1e+07)

as_scale_table(a, x, value = "value", frames = NULL, drop_na = TRUE)

Arguments

Value

An array with one dimension per axis.

as_scale_table() returns a data.frame: one column per axis key, plus the value column.

Examples

p <- scale_product(a = scale_example(), b = scale_example2())
d <- merge(
  data.frame(unit = c("U1", "U2")),
  data.frame(period = c("p1", "p2"))
)
d$v <- 1:4
as_scale_array(d, p, value = "v")
as_scale_table(as_scale_array(d, p, value = "v"), p, value = "v")

Attach aliases to a scale

Description

Appends rows to the object's alias table: alternative labels by which a member (frame, unit) is known, tagged with the source the spelling comes from (a palette, a model's commodity codes, a dataset). The same alias may legitimately point at several members; scale_match() reports that as ambiguous unless the caller names the frame.

Usage

attach_aliases_scale(x, aliases, source = NULL)

Arguments

Value

The scale with the rows appended.

Examples

s <- attach_aliases_scale(
  scale_example(),
  data.frame(frame = "sector", unit = c("P", "S"),
             alias = c("primary", "secondary")),
  source = "labels"
)
scale_match(s, c("Primary", "G1", "nope"))

Attach physical attributes to a scale

Description

Appends rows to the object's attribute table. A row names the member it belongs to (frame, unit), the attribute (attribute, e.g. "lhv"), its kind (see SCALE_ATTRIBUTE_KINDS), its value, an optional lower/upper range, the physical unit_of_measure ("GJ/t", "kg/TJ"), the source the figure was taken from and an optional note. Values can be attached at any frame: a calorific value for a whole carrier is inherited by every fuel under it that has no value of its own (see scale_attributes()).

Usage

attach_attributes_scale(x, attributes, source = NULL, kind = NULL)

Arguments

Details

Existing rows are never overwritten: a row whose ⁠(frame, unit, attribute, source)⁠ already exists is an error. Attach a second source under its own source name and choose between them with ⁠source=⁠ when reading.

Value

The scale with the rows appended.

Examples

s <- scale_example()
s2 <- attach_attributes_scale(
  s,
  data.frame(frame = "class", unit = c("G1", "G2", "S1"),
             attribute = "density", value = c(10, 20, 5),
             unit_of_measure = "t/m3"),
  source = "survey", kind = "property"
)
scale_attributes(s2, "density", frame = "unit")

Backends: the table types the verbs accept

Description

The conversion verbs run on a data.frame, a tibble, a data.table, a dtplyr lazy table or an arrow Dataset/Table/query, and return results in the caller's type; lazy inputs stay lazy unless collected. These functions are that contract, exposed so a dimension package built on nestedscales can accept the same inputs as the core verbs.

Usage

backend_of(data, arg = "data")

backend_is_lazy(backend)

backend_schema(data, backend = NULL)

backend_lazy(data, backend = NULL)

backend_scan(data, backend = NULL, columns = NULL, filter = NULL)

backend_distinct(data, columns, backend = NULL)

backend_pull(table)

backend_restore(table, backend, collect = NULL)

backend_bind_rows(
  parts,
  use_dt = requireNamespace("data.table", quietly = TRUE)
)

Arguments

Details

• backend_of() names the backend of data, and errors on an unsupported input – the one place that message is written.

• backend_is_lazy() says whether a backend produces queries.

• backend_schema() describes the columns without scanning the data.

• backend_lazy() lifts data into a dplyr-compatible carrier for a pipeline (a data.table becomes a dtplyr step when dtplyr is installed).

• backend_scan() reads the needed columns and rows into memory.

• backend_distinct() is the distinct rows of some columns, in first-appearance order, read into memory.

• backend_pull() materialises a query or table as a data.frame.

• backend_restore() returns a pipeline result in the caller's type.

• backend_bind_rows() binds chunk results into one frame.

Value

backend_of() a single string naming the backend; backend_is_lazy() a logical; backend_schema() a zero-row data.frame; backend_lazy() data, wrapped where the backend needs it; backend_scan(), backend_distinct() and backend_pull() a data.frame; backend_restore() the result in the input's class; backend_bind_rows() one data frame, or NULL when every part is empty.

Examples

d <- data.frame(unit = c("U1", "U2", "U2"), v = 1:3)
backend_of(d)
backend_is_lazy(backend_of(d))
backend_schema(d)
backend_distinct(d, "unit")
backend_scan(d, columns = "v", filter = rlang::expr(v > 1))

Clear the registered crosswalks

Description

Mainly useful in tests.

Usage

clear_scale_maps(name = NULL)

Arguments

Value

Invisibly NULL.

Examples

clear_scale_maps()

Clear the rule registry

Description

Mainly useful in tests.

Usage

clear_scale_rules(name = NULL, scope = NULL)

Arguments

Value

Invisibly NULL.

Examples

register_scale_rule("tmp_param", "sum")
clear_scale_rules("tmp_param")

Cluster the units of a scale under a contiguity constraint

Description

Merges only units the constraint allows to be merged, so the resulting groups are contiguous in order (an ordered scale) or connected in space (an adjacency graph).

Usage

cluster_contiguous(
  data,
  x,
  k,
  frame = NULL,
  key = NULL,
  value = NULL,
  distance = "euclidean",
  scale_units = FALSE,
  adjacency = "order",
  circular = FALSE,
  linkage = "average",
  new_frame = "cluster",
  labels = "c"
)

Arguments

Details

The constraint can make k unreachable: an adjacency graph with more connected components than k cannot be merged below that number, and the function stops with the count rather than returning something that quietly violates the constraint.

Value

As cluster_scale(): the scale with the cluster frame added, and the assignment in the "clustering" attribute.

Examples

s <- scale_example()
d <- data.frame(
  unit = rep(scale_units(s), each = 4),
  t = rep(sprintf("t%d", 1:4), 7), v = as.numeric(seq_len(28))
)
cl <- cluster_contiguous(d, s, k = 3)
attr(cl, "clustering")

The representative unit of each cluster

Description

The medoid is an ACTUAL unit, not an average of units, which is what makes it usable as a representative period or a representative region: a model built on medoids is built on things that really happened.

Usage

cluster_medoids(x, frame = NULL)

Arguments

Value

A data frame of cluster and its representative unit.

Examples

s <- scale_example()
d <- data.frame(
  unit = rep(scale_units(s), each = 4),
  t = rep(sprintf("t%d", 1:4), 7), v = as.numeric(seq_len(28))
)
cluster_medoids(cluster_scale(d, s, k = 3))

Cluster the units of a scale into a new frame

Description

Groups the units at frame by the similarity of their profiles and returns the scale with those groups added as a new, coarser frame.

Usage

cluster_scale(
  data,
  x,
  k,
  frame = NULL,
  key = NULL,
  value = NULL,
  method = c("pam", "hclust", "kmeans"),
  distance = "euclidean",
  scale_units = FALSE,
  new_frame = "cluster",
  labels = "c",
  hclust_method = "ward.D2",
  seed = NULL,
  ...
)

Arguments

Details

The new frame goes directly above the clustered one, which is the only position that is always true: the clusters are coarser than frame by construction, but they need not nest inside any existing coarser frame, and nestedscales does not require them to. Use scale_nests() to ask whether a given pair happens to nest.

Value

The scale, with new_frame inserted directly above frame. The assignment is also returned as the "clustering" attribute: a data frame of unit, cluster and (for medoid-based methods) whether the unit is its cluster's medoid.

Examples

s <- scale_example()
set.seed(1)
d <- data.frame(
  unit = rep(scale_units(s), each = 4),
  t = rep(sprintf("t%d", 1:4), 7),
  v = as.numeric(seq_len(28))
)
cl <- cluster_scale(d, s, k = 3)
scale_frames(cl)
head(attr(cl, "clustering"))

Cluster quality across a range of k

Description

Cluster quality across a range of k

Usage

cluster_sweep(
  data,
  x,
  ks,
  frame = NULL,
  key = NULL,
  value = NULL,
  distance = "euclidean",
  scale_units = FALSE,
  FUN = cluster_scale,
  ...
)

Arguments

Value

A data frame with one row per k: k, within (total within-cluster sum of distances to the cluster medoid), silhouette (average, NA at k = 1), and smallest (the size of the smallest cluster, which is often what rules a k out).

Examples

s <- scale_example()
d <- data.frame(
  unit = rep(scale_units(s), each = 4),
  t = rep(sprintf("t%d", 1:4), 7), v = as.numeric(seq_len(28))
)
cluster_sweep(d, s, ks = 2:4)

Convert unit-keyed quantities with a member attribute

Description

Multiplies (or divides) the numeric value columns of data by the value of attribute resolved for each row's unit – a net calorific value to turn mass into energy, an emission factor to turn energy into CO2, a density to turn volume into mass. Values are resolved along the hierarchy exactly as scale_attributes() does, so a factor stored on a coarser member applies to every atom under it that has no factor of its own.

Usage

convert_scale(
  data,
  x,
  attribute,
  from_unit,
  to_unit,
  key = NULL,
  frame = NULL,
  values = NULL,
  source = NULL,
  na_action = c("error", "keep"),
  collect = NULL
)

Arguments

Details

Direction comes from the attribute's unit of measure, read as "<numerator>/<denominator>": with lhv in "GJ/t", ⁠from_unit = "t", to_unit = "GJ"⁠ multiplies, and ⁠from_unit = "GJ", to_unit = "t"⁠ divides. The inverse is allowed only when the attribute's kind is "property"; a "factor" is one-way. No unit-prefix arithmetic is done: a column in kilotonnes must be turned into tonnes before converting with a GJ/t factor.

A unit for which no value resolves is an error by default – a silently dropped row is a wrong total – unless na_action = "keep", which leaves NA in its rows.

Value

data with the value columns converted, plus two columns named after the attribute: ⁠<attribute>⁠ (the factor applied) and ⁠<attribute>.origin_frame⁠ (the frame the factor was taken from), in the input's class.

Examples

s <- attach_attributes_scale(
  scale_example(),
  data.frame(frame = c("sector", "unit"), unit = c("P", "U5"),
             attribute = "lhv", value = c(20, 30),
             unit_of_measure = "GJ/t"),
  source = "example", kind = "property"
)
mass <- data.frame(unit = c("U1", "U2", "U5"), t = c(100, 200, 50))
energy <- convert_scale(mass, s, attribute = "lhv",
                        from_unit = "t", to_unit = "GJ")
energy
# back to tonnes: a property is invertible
convert_scale(energy[c("unit", "t")], s, attribute = "lhv",
              from_unit = "GJ", to_unit = "t")

Subset a product componentwise

Description

Applies filter_scale() to one axis and returns a new product.

Usage

filter_product(x, axis, frame, unit, ...)

Arguments

Value

A ScaleProduct.

Examples

p <- scale_product(a = scale_example(), b = scale_example2())
filter_product(p, "a", "sector", "P")

Subset a NestedScale by unit

Description

Keeps only the atoms belonging to unit at frame, and rebuilds the member vocabularies accordingly.

Usage

filter_scale(x, frame, unit, drop_empty_frames = FALSE)

Arguments

Details

A genuine subset is a SAMPLE and is book-kept as one: meta$coverage records, per weight column, the kept fraction of the ROOT parent's total (so filters compose against the original object), meta$parent_totals stores those root totals (making the coverage claim verifiable by the validator), meta$parent_name records the parent, and meta$name is mangled to "parent[frame:codes]" so a sample never impersonates its parent in the crosswalk registry or in join_scale() column names. A filter that keeps every atom is a true no-op. Read the fraction back with scale_coverage().

Value

A NestedScale.

Examples

s <- scale_example()
p <- filter_scale(s, "sector", "P")
p
scale_coverage(p)

Look up a registered rule

Description

A scoped lookup prefers the entry registered for that scope and falls back to an unscoped one.

Usage

get_scale_rule(param, scope = NULL)

Arguments

Value

A list with elements rule, weight and scope, or NULL if param has not been registered.

Examples

register_scale_rule("demand", "sum")
get_scale_rule("demand")
get_scale_rule("not_registered")
clear_scale_rules("demand")

Validate frame names

Description

A frame name must be a non-empty, non-NA string that is a syntactically valid R name (so it can be used as a data.frame column without quoting).

Usage

is_valid_frame(x)

Arguments

Value

A logical vector the same length as x.

Examples

is_valid_frame(c("industry", "reg32", "", "2bad"))

Attach every axis's labels to product-indexed data

Description

join_scale() applied to each axis in turn, keyed by that axis's column. Because every attached column is named after its scale, the result carries all the axes' labels side by side.

Usage

join_product(data, x, axes = NULL, ...)

Arguments

Value

data with each axis's labels attached, in its own class.

Examples

p <- scale_product(a = scale_example(), b = scale_example2())
d <- data.frame(unit = c("U1", "U3"), period = c("p1", "p2"), v = 1:2)
join_product(d, p, attach = TRUE)

Attach scale labels and memberships to a dataset

Description

Adds a label column named after the scale, and optionally the codes of coarser frames ("<name>.<frame>") and the unit's share and weight ("<name>.share", "<name>.weight").

Usage

join_scale(
  data,
  x,
  key = NULL,
  frame = NULL,
  attach = NULL,
  meta = FALSE,
  weight = NULL,
  as_factor = TRUE,
  collect = NULL,
  diagnostics = c("auto", "on", "off")
)

Arguments

Details

Existing columns are never overwritten – a collision is an error.

Value

data with the attached columns, in its own class.

Examples

s <- scale_example()
d <- data.frame(class = c("G1", "G2", "S1"), v = 1:3)
join_scale(d, s, attach = TRUE)
join_scale(d, s, attach = "sector")
join_scale(d, s, meta = TRUE)

List registered rules

Description

List registered rules

Usage

list_scale_rules(scope = NULL)

Arguments

Value

A data.frame with columns param, rule, weight and scope.

Examples

register_scale_rule("invcost", "weighted_mean", weight = "size")
list_scale_rules()
clear_scale_rules("invcost")

Open a scale-aware dataset

Description

Open a scale-aware dataset

Usage

open_scale_dataset(path)

## S3 method for class 'scale_dataset'
print(x, ...)

Arguments

Value

A scale_dataset: a list with data (a lazily-read arrow Dataset) and scale (the NestedScale or ScaleProduct it is indexed by). Feed them to the verbs as recast_scale(ds$data, ds$scale, ...).

Examples

# see write_scale_dataset()

Realise the atoms of a product

Description

The cross product of the component leaftables: one row per tuple of component atoms, with each axis's key column named by product_keys() and its frame columns prefixed "<axis>.".

Usage

product_atoms(x, limit = 1e+06)

Arguments

Details

This is guarded on purpose. An hourly calendar crossed with NUTS3 is ~12.9M rows; the whole point of a product is that its atoms exist implicitly, and conversion never needs them realised (see recast_scale()). Ask for them only when you actually want the grid.

Value

A data.frame with product_size(x)$total rows.

Examples

p <- scale_product(a = scale_example(), b = scale_example2())
head(product_atoms(p))

Coverage of each axis

Description

Sampling is per-axis and independent, so the coverage of the product is the product of its axes' coverages – prod(product_coverage(p)).

Usage

product_coverage(x, weight = NULL)

Arguments

Value

A named numeric, one entry per axis.

Examples

p <- scale_product(a = scale_example(), b = scale_example2())
product_coverage(p)
prod(product_coverage(p))

Frames of each axis

Description

Frames of each axis

Usage

product_frames(x)

Arguments

Value

A named list of character vectors, each coarsest-first.

Examples

product_frames(scale_product(a = scale_example(), b = scale_example2()))

The data key column of each axis

Description

Data on a product carries one of these columns per axis.

Usage

product_keys(x)

Arguments

Value

A named character vector, axis -> column name.

Examples

product_keys(scale_product(a = scale_example(), b = scale_example2()))

Size of a product

Description

The per-axis atom counts and their product – the number of rows a fully dense realisation would have.

Usage

product_size(x)

Arguments

Value

A list with axes (named integer, atoms per axis) and total (double, their product).

Examples

product_size(scale_product(a = scale_example(), b = scale_example2()))

Weight columns of each axis

Description

A product has no weights of its own: the weight of a product atom is the product of its components' weights, which is derived when needed and never stored.

Usage

product_weights(x)

Arguments

Value

A named list of character vectors.

Examples

product_weights(scale_product(a = scale_example(), b = scale_example2()))

Collapse a NestedScale to a coarser frame

Description

Returns a new NestedScale whose atom layer is frame, dropping every finer frame. Weights are summed over the collapsed atoms.

Usage

prune_scale(x, frame)

Arguments

Details

The result is renamed "name@frame" with the parent recorded in meta$parent_name; every other meta field is preserved. Atoms with no code at frame are dropped, and that loss is reflected in meta$coverage. The concrete class is kept: a subclass comes back as itself, its attribute and alias rows at the surviving frames kept, and a positional payload merged through scale_payload_collapse().

Value

A NestedScale.

Examples

prune_scale(scale_example(), "class")

Recast data through a scale

Description

The bare pipeline verb, dispatching on the scale so one entry point serves a NestedScale, a ScaleProduct, and the dimension subclasses in timescales and geoscales, which register their own methods against this generic. recast_scale() and recast_product() are the explicit workers.

Usage

recast(x, scale, ...)

Arguments

Details

The scale is scale; the source frame is source and the target target, as in recast_scale(), so a call reads the same through the generic and through the worker.

Value

The recast data, in the input's class.

Examples

s <- scale_example()
d <- data.frame(
  unit = c("U1", "U2", "U3", "U4", "U5", "U6"),
  capacity = c(1, 2, 3, 4, 5, 6)
)
recast(d, s, target = "sector", rule = "sum")

Recast values through a given crosswalk

Description

The engine behind recast_scale(), for a crosswalk built elsewhere: by a dimension package whose conversion cannot be read off one scale's leaftable, such as timescales mapping one calendar onto another through a datetime grid. The crosswalk says how much of each source code falls in each target code; the rules then aggregate exactly as in recast_scale(), on any supported backend.

Usage

recast_crosswalk(
  data,
  map,
  source,
  target,
  key = source,
  values = NULL,
  rule = NULL,
  ids = NULL,
  by = character(),
  targets = NULL,
  missing_sources = c("na", "ignore"),
  unit = "unit",
  collect = NULL
)

Arguments

Details

The crosswalk is a data frame with one row per (source, target) pair:

• the source and target codes, in the columns named by source and target;

• n_from, the number of atoms (grid points) in the source code, and n_overlap, the number of them that fall in the target;

• optionally w, the weight of the pair for "weighted_mean" – by default n_overlap;

• optionally w_from, the source's total weight. With it, "sum" splits a source across its targets by w / w_from; without it, by n_overlap / n_from, an equal split over atoms;

• any columns named in by.

A target whose sources are only partly present in the data, per identifier combination, comes back NA unless missing_sources = "ignore".

Value

The recast data in its own class, with columns c(to, ids, values). Lazy in, lazy out.

Examples

# Two months onto one quarter, with the months' day counts as atoms
map <- data.frame(
  month = c("m01", "m02"), quarter = "Q1",
  n_from = c(31, 28), n_overlap = c(31, 28)
)
d <- data.frame(
  month = c("m01", "m02"), energy = c(310, 280),
  price = c(10, 20)
)
recast_crosswalk(d, map,
  source = "month", target = "quarter",
  rule = c(energy = "sum", price = "weighted_mean")
)

Recast data indexed by a product of scales

Description

Converts one or more axes of a product in a single call. Axes named in target are converted; axes left out pass through untouched.

Usage

recast_product(
  data,
  x,
  target,
  values = NULL,
  rules = NULL,
  weights = NULL,
  na_action = c("drop", "error", "keep"),
  collect = NULL
)

Arguments

Details

Execution is a sequence of ordinary single-axis recast_scale() calls – see the file comment in R/product-recast.R for why that is exactly equivalent to a joint recast for every supported rule, and why the joint crosswalk is never built. The passes are ordered so the axis that reduces the data most runs first.

A per-axis "sd" is refused because it is order-dependent; a scalar "sd" means the pooled sd over all product atoms in a target block, which is well defined and is computed without materialising the product. "share" is not defined across a product, because it changes which axis the result is keyed by.

Value

The recast data in the input's class, keyed by the converted axes' target units.

Examples

p <- scale_product(a = scale_example(), b = scale_example2())
d <- merge(
  data.frame(unit = c("U1", "U2", "U3", "U4", "U5", "U6")),
  data.frame(period = c("p1", "p2", "p3", "p4"))
)
d$cap <- seq_len(nrow(d))

# aggregate both axes at once
recast_product(d, p, target = list(a = "sector", b = "era"), rules = "sum")

# different rules per axis
recast_product(d, p,
  target = list(a = "sector", b = "era"),
  rules = list(cap = c(a = "sum", b = "weighted_mean"))
)

Recast values from one resolution to another

Description

The central conversion verb: takes a table keyed by unit code at frame source and returns one keyed at target – a frame name of the same NestedScale, or ANOTHER NestedScale (whose atom layer is the target, matched on shared atom keys). Handles both aggregation (fine to coarse) and disaggregation (coarse to fine) with one rule per value column; frames that cross-cut work too, because the route always goes through the atom layer. A crosswalk registered with register_scale_map() short-circuits the derivation.

Usage

recast_scale(
  data,
  x,
  source = NULL,
  target,
  key = NULL,
  values = NULL,
  rule = NULL,
  weight = NULL,
  na_action = c("drop", "error", "keep"),
  parent = NULL,
  collect = NULL,
  diagnostics = c("auto", "on", "off"),
  missing_sources = c("na", "ignore"),
  batch = NULL,
  batch_by = NULL,
  ...
)

Arguments

Details

Columns of the data that are neither the key nor a value column are treated as identifiers (panel columns – a year, a technology) and preserved as grouping columns, so panel data recasts correctly in one call. This is what makes mixed pipelines across two dimensions work: a recast in one dimension carries the other dimension's key through untouched. Columns named like the scale's own frames are treated as unit attributes and dropped.

"sum" splits each source value across its unit's atoms proportionally to the weight before summing up, so totals are conserved. "weighted_mean" weights by the atom weights; "mean" is the plain atom-count mean – the two differ exactly when atom weights differ. "copy" requires a constant value per target unit; "sd" is aggregation-only.

"share" inverts the output contract: the result stays keyed at source, and each value becomes that unit's share of the total over its parent group. source must nest within the parent. It cannot be combined with other rules in one call, and ⁠weight=⁠ is ignored – the observed values themselves are the weights.

Value

The recast table in the input's class, with columns c(to, identifiers, values).

Backends

The data may be a data.frame, tibble, data.table, dtplyr lazy table, or an arrow Dataset/Table/query. The result comes back in the input's class; lazy inputs return the uncollected query unless collect = TRUE. Lazy results contain the observed target units only – the full-vocabulary completion (and its NA rows) applies when the result is materialised.

Examples

s <- scale_example()

# Extensive quantity, fine -> coarse: totals are preserved
d <- data.frame(
  unit = c("U1", "U2", "U3", "U4", "U5", "U6"),
  capacity = c(1, 2, 3, 4, 5, 6)
)
recast_scale(d, s, source = "unit", target = "sector", rule = "sum")

# Coarse -> fine: split proportionally to size
y <- data.frame(sector = c("P", "S"), capacity = c(10, 20))
recast_scale(y, s,
  source = "sector", target = "class",
  rule = "sum", weight = "size"
)

# Share within parent: result stays at the atoms, sums to 1 per sector
recast_scale(
  d, s,
  source = "unit", target = "sector",
  rule = "share",
  na_action = "keep"
)

Recast unit data down to the atom layer, and back

Description

The two public halves of the from -> atoms -> to route: recast_to_atoms() projects unit-keyed data DOWN to the atom layer (one row per atom), and recast_from_atoms() aggregates atom-keyed data UP into a frame's units. Their composition is recast_scale(), and because the atom rows are keyed by the atom IDs, recast_from_atoms(recast_to_atoms(data, a), b, target) recasts across two scales that share atom keys.

Usage

recast_to_atoms(
  data,
  x,
  source = NULL,
  key = NULL,
  values = NULL,
  rule = NULL,
  weight = NULL,
  attach_weight = TRUE,
  collect = NULL
)

recast_from_atoms(
  data,
  x,
  target,
  key = NULL,
  values = NULL,
  rule = NULL,
  weight = NULL,
  na_rm = FALSE,
  na_action = c("drop", "error", "keep"),
  collect = NULL
)

Arguments

Details

Going down, extensive columns (rule "sum") are split across a unit's atoms proportionally to the chosen weight so totals conserve; intensive columns are repeated. A weight column is attached by default so the return trip's "weighted_mean" reproduces the source weighting exactly.

Value

recast_to_atoms(): one row per (atom x identifier combination). recast_from_atoms(): one row per (unit x identifier combination). Both in the input's class; lazy in, lazy out.

Examples

s <- scale_example()
d <- data.frame(sector = c("P", "S"), capacity = c(10, 20))
atoms <- recast_to_atoms(d, s,
  source = "sector", rule = "sum",
  weight = "size"
)
head(atoms)
recast_from_atoms(
  atoms, s,
  target = "class",
  rule = "sum",
  na_action = "keep"
)

Check fine data against an independent parent total

Description

Aggregates data from source to target and compares the result with totals, the parent figure from another source. Reports the gap by default; can also close it.

Usage

reconcile_scale(
  data,
  x,
  source,
  target,
  totals,
  key = NULL,
  values = NULL,
  rule = "sum",
  weight = NULL,
  balance = c("none", "residual", "proportional"),
  tolerance = 1e-09
)

Arguments

Value

With balance = "none", a long gap table: the target column, any identifiers, value (which column), aggregated, target, gap (target - aggregated), rel_gap and ok. With a balancing method, the adjusted data keyed as the input was, carrying that same table as its "reconciliation" attribute.

Examples

lf <- data.frame(
  country = c("DE", "DE", "DE"),
  unit = c("DE1", "DE2", "DE_XR"),
  pop = c(40, 30, 0), stringsAsFactors = FALSE
)
s <- scale_from_leaftable(lf,
  frames = c("country", "unit"), key = "unit",
  weights = "pop", name = "geo",
  residuals = list(unit = "DE_XR")
)
fine <- data.frame(unit = c("DE1", "DE2", "DE_XR"), gdp = c(400, 300, 0))
published <- data.frame(country = "DE", gdp = 750)

reconcile_scale(fine, s, source = "unit", target = "country", published)

reconcile_scale(fine, s,
  source = "unit", target = "country", published,
  balance = "residual"
)

Register a distance

Description

The built-ins cover level and shape; anything else – dynamic time warping, a domain-specific dissimilarity, a precomputed matrix – is registered here and then usable by name everywhere in the package.

Usage

register_scale_distance(name, fn)

get_scale_distance(name)

list_scale_distances()

clear_scale_distances(name = NULL)

Arguments

Value

Invisibly, name.

Examples

register_scale_distance("first_diff", function(m) {
  stats::dist(t(apply(m, 1, diff)))
})
"first_diff" %in% list_scale_distances()
clear_scale_distances("first_diff")

Register / look up a direct crosswalk

Description

A registered map short-circuits the atom-layer derivation in scale_map() or scale_map_between() (and thereby recast_scale()) for one pair of resolutions – for cases where the exact correspondence is known (hand-audited crosswalks, official concordance tables).

Usage

register_scale_map(x, source, target, map)

register_scale_map_between(source, target, map)

get_scale_map(x, source, target)

get_scale_map_between(source, target)

list_scale_maps()

Arguments

Details

register_scale_map() and get_scale_map() handle a pair of frames of one scale; the map is scoped to that scale, so the same frame pair in two different scales does not collide. register_scale_map_between() and get_scale_map_between() handle a pair of scales.

Value

Invisibly, the registry key. The get_ functions return the registered map (or NULL); list_scale_maps() a data.frame of registry keys.

Examples

s <- scale_example()
fake <- data.frame(
  class = "G1", group = "GC", n_from = 1L,
  n_overlap = 1L, w = 1, w_from = 1
)
register_scale_map(s, "class", "group", fake)
list_scale_maps()
get_scale_map(s, "class", "group")
register_scale_map(s, "class", "group", NULL) # remove
clear_scale_maps()

Register how a parameter should be recast

Description

Records the rule (and optionally the weight) to use for a named value column, so callers of recast_scale() need not repeat it. Downstream packages register their own parameter maps at load time.

Usage

register_scale_rule(param, rule, weight = NULL, scope = NULL)

Arguments

Value

Invisibly, the registered entry.

Examples

register_scale_rule("capacity", "sum")
register_scale_rule("eff", "weighted_mean", weight = "count")
get_scale_rule("eff")
clear_scale_rules(c("capacity", "eff"))

An alias property for a subclass

Description

Builds a getter/setter property that reads and writes an inherited property under a different name – how timescales' Calendar keeps ⁠@timeframes⁠ and geoscales' Geoscale keeps ⁠@geoframes⁠ working over the inherited ⁠@frames⁠, so no existing code has to change when they become subclasses.

Usage

scale_alias_property(name)

Arguments

Details

The setter is a no-op on NULL. S7's generated constructor writes every dynamic property once with NULL before the real values are set, so a setter that forwards NULL makes the class impossible to construct.

Value

An S7 property.

Examples

Aliased <- S7::new_class("Aliased",
  parent = scale_class(),
  properties = list(levels = scale_alias_property("frames"))
)
a <- Aliased(
  leaftable = scale_leaftable(scale_example()),
  frames = scale_frames(scale_example()),
  members = S7::prop(scale_example(), "members")
)
a@levels

Read the alias table

Description

Read the alias table

Usage

scale_aliases(x, frame = NULL, source = NULL)

Arguments

Value

A data.frame with columns frame, unit, alias, source.

Examples

s <- attach_aliases_scale(
  scale_example(),
  data.frame(frame = "sector", unit = "P", alias = "primary"),
  source = "labels"
)
scale_aliases(s)
scale_aliases(s, frame = "class")

Ancestry between all frame pairs

Description

Every ⁠(coarser, finer)⁠ code pair that shares at least one atom, for all frame pairs.

Usage

scale_ancestry(x)

Arguments

Details

Computed atom-mediated, directly from the leaftable – deliberately not as a transitive closure of scale_family(). Closing a hierarchy that cross-cuts manufactures false relationships: in the example scale, group GB straddles both sectors, so closing sector -> class -> group -> unit would wrongly report sector P as an ancestor of unit U5, which lies in sector S.

For frames that do not nest this relation is overlap, not containment – test a given pair with scale_nests().

Frame columns are retained because codes are not unique across frames: in the example, "G1" exists at both class and group, so a bare ⁠(parent, child)⁠ pair would read as a self-loop.

Value

A data.frame with columns parent_frame, parent, child_frame, child.

Examples

head(scale_ancestry(scale_example()))

The level at which a scale enumerates its atoms

Description

The finest frame when one code of it sits on each leaftable row, and the key column otherwise. A NestedScale built from a nested hierarchy is the first kind; a timescales Calendar is the second, because its timeslices are combinations of its frames rather than codes of any one of them.

Usage

scale_atom_level(x)

Arguments

Details

Needed by the dimension packages: it is the frame to pass to join_scale() or recast_scale() for data keyed at the atoms.

Value

A single string – a frame name, or the key.

Examples

scale_atom_level(scale_example())

Atom-level correspondence between two frames

Description

The seam a dimension package overrides when its atom layer is generated rather than enumerated. Returns one row per atom (or per atom and extra identifier, e.g. per grid instant and year).

Usage

scale_atom_pairs(x, ...)

Arguments

Value

A data.frame with columns source, target, w, plus any extra identifier columns the dimension carries. The extra columns must be named in the by argument of the caller.

Examples

head(scale_atom_pairs(scale_example(), "class", "group"))

Read attributes, resolved along the hierarchy

Description

With frame = NULL, returns the stored attribute table (filtered by attribute and source). With a frame, returns one row per member of that frame per attribute, resolved: a member's own value when it has one, otherwise the value of its nearest ancestor that has one, walking coarser frames one at a time. The row reports where the value came from in origin_frame/origin_unit; a member with no value anywhere gets an NA row.

Usage

scale_attributes(
  x,
  attribute = NULL,
  frame = NULL,
  source = NULL,
  inherit = TRUE
)

Arguments

Details

Two situations are errors rather than guesses:

• the walk reaches a frame where the member has several ancestors (the frames cross-cut) and at least one of them holds the attribute – "inheritance is ambiguous";

• several sources publish the attribute for the same member and source is NULL – pass ⁠source=⁠ to choose.

Value

A data.frame with the attribute columns (frame, unit, attribute, kind, value, lower, upper, unit_of_measure, source, note), plus origin_frame and origin_unit when frame is given.

Examples

s <- attach_attributes_scale(
  scale_example(),
  data.frame(frame = c("sector", "class"), unit = c("P", "S1"),
             attribute = "density", value = c(10, 5),
             unit_of_measure = "t/m3"),
  source = "survey", kind = "property"
)
scale_attributes(s)                           # stored rows
scale_attributes(s, "density", frame = "unit") # resolved per atom

Axes of a product

Description

Axes of a product

Usage

scale_axes(x, axis = NULL)

Arguments

Value

A named list of NestedScale objects, or one NestedScale when axis is given.

Examples

p <- scale_product(a = scale_example(), b = scale_example2())
names(scale_axes(p))
scale_axes(p, "a")

Shipped scale definitions

Description

Packages ship scale definitions as files (see write_scale()) and register the directory holding them; scale_catalog() lists what every registered directory holds and read_scale() takes an id from it. nestedscales registers its own ⁠inst/scales/⁠ on load, so read_scale("carriers") works with nothing else attached.

Usage

register_scale_dir(path, package = NULL)

list_scale_dirs()

clear_scale_dirs()

scale_catalog()

Arguments

Details

register_scale_dir() is what a dimension package calls in its .onLoad(): register_scale_dir(system.file("scales", package = pkgname), package = pkgname).

Value

scale_catalog() a data.frame with one row per definition: id, name, kind (the class, without its package prefix), key, frames (joined by /), n_atoms, package, path. register_scale_dir() the registry key, invisibly; list_scale_dirs() a data.frame with package and path; clear_scale_dirs() NULL, invisibly.

Examples

scale_catalog()

The scale classes, for subclassing and dispatch

Description

The classes are not exported under their own names (the package's surface is its functions), so these return them. Needed to build a subclass (S7::new_class(..., parent = scale_class()), which is what scale_alias_property() is for) or to register an S7 method against one. To merely ask what an object is, use scale_is().

Usage

scale_class()

scale_product_class()

Value

The S7 class object.

Examples

scale_class()
S7::S7_inherits(scale_example(), scale_class())

Coverage of a sampled NestedScale

Description

The fraction of the root parent's weight that survives in this object. A scale that was never subset covers all of itself, so every weight reports 1.

Usage

scale_coverage(x, weight = NULL)

Arguments

Value

A named numeric over the declared weights, or a single unnamed numeric when weight is given.

Examples

scale_coverage(scale_example())

Do two frames form a product?

Description

TRUE when every unit of a combines with the same set of units of b, as the months and hours of a ⁠MONTH x HOUR⁠ calendar do: the frames overlay rather than nest, and the atoms are their combinations. The complement of scale_nests() for the other common shape; a pair can be neither.

Usage

scale_crosses(x, a, b)

Arguments

Value

A logical scalar. When FALSE, the attribute "offenders" names the units of a whose set of b units differs from the first one's.

Examples

s <- scale_example()
scale_crosses(s, "class", "group")

grid <- expand.grid(m = c("m1", "m2"), h = c("h1", "h2", "h3"),
                    stringsAsFactors = FALSE)
grid$unit <- paste(grid$m, grid$h, sep = "_")
mh <- scale_from_leaftable(grid, frames = c("m", "h"), key = "unit")
scale_crosses(mh, "m", "h")
scale_nests(mh, "m", "h")

Describe a stored dataset without opening it fully

Description

Describe a stored dataset without opening it fully

Usage

scale_dataset_info(path)

Arguments

Value

A list with kind, axes, keys, files and bytes.

Examples

# see write_scale_dataset()

Distance between the units of a scale

Description

Reshapes long, unit-keyed data into a unit-by-observation matrix and measures the distance between its rows.

Usage

scale_distance(
  data,
  x,
  frame = NULL,
  key = NULL,
  value = NULL,
  method = "euclidean",
  scale_units = FALSE
)

Arguments

Value

A stats::dist object over the frame's units, in member order.

Examples

s <- scale_example()
d <- data.frame(
  unit = rep(scale_units(s), each = 3),
  t = rep(sprintf("t%d", 1:3), 7), v = as.numeric(seq_len(21))
)
round(scale_distance(d, s, method = "correlation"), 2)

A small example NestedScale

Description

A synthetic 3-frame hierarchy used in examples and tests. It deliberately reproduces three awkward features of real classification tables:

Usage

scale_example()

Details

• a code ("G1") reused at more than one frame, so bare codes are ambiguous and every lookup must name its frame;

• a non-nesting pair of frames – group "GB" draws units from two different classes, so class and group do not form a tree;

• a unit ("OTH") with no code at any coarser frame (partial coverage).

Value

A NestedScale with 7 atoms and frames sector/class/group/unit.

Examples

s <- scale_example()
s
scale_nests(s, "class", "group") # FALSE - they cross-cut

A second small example NestedScale

Description

A different dimension from scale_example(), for the examples that need two axes. Two frames, four atoms, its own key column (period) and a weight – deliberately well behaved, so the awkwardness in an example stays on the other axis.

Usage

scale_example2()

Value

A NestedScale with 4 atoms and frames era/period.

Examples

scale_example2()
scale_product(a = scale_example(), b = scale_example2())

Immediate parent-child table between two frames

Description

Immediate parent-child table between two frames

Usage

scale_family(x, parent = NULL, child = NULL)

Arguments

Value

A data.frame with columns parent_frame, parent, child_frame, child. Atoms unassigned at either frame are omitted.

Examples

s <- scale_example()
scale_family(s, "group", "unit")

Frames of a NestedScale

Description

The hierarchy names, ordered coarsest first. The last entry is the atom frame whenever one frame enumerates the atoms; when none does – a Calendar's atoms are combinations of its frames – the key column is the atom level instead.

Usage

scale_frames(x, finest = FALSE)

## S3 method for class 'NestedScale'
names(x)

Arguments

Value

A character vector of frame names, or a single name when finest = TRUE.

Examples

s <- scale_example()
scale_frames(s)
scale_frames(s, finest = TRUE)

Build a NestedScale from a flat table of atoms

Description

The general constructor. Takes a wide data.frame with one row per atom (the finest unit) and one column per frame, and returns a NestedScale.

Usage

scale_from_leaftable(
  leaftable,
  frames,
  key = NULL,
  weights = NULL,
  default_weight = NULL,
  members = NULL,
  name = "",
  desc = "",
  attributes = NULL,
  aliases = NULL,
  ...
)

Arguments

Details

Blank strings ("") in frame columns are normalised to NA, meaning "this atom has no code at this frame" – partial coverage is normal in real classification tables.

Value

A NestedScale.

Examples

df <- data.frame(
  section = c("A", "A", "A", "B"),
  group   = c("A1", "A1", "A2", "B1"),
  unit    = c("A11", "A12", "A21", "B11"),
  gva     = c(100, 200, 300, 400)
)
scale_from_leaftable(df, frames = c("section", "group", "unit"))

Is this a scale, or a product of scales?

Description

The classes themselves are not exported (the package's surface is its functions), so these are how calling code asks. Build a scale with scale_from_leaftable() and a product with scale_product().

Usage

scale_is(x)

scale_is_product(x)

Arguments

Details

Named with the object prefix rather than is_scale(), which ggplot2 already exports.

Value

A single TRUE or FALSE. A product is NOT a scale: it holds several and materialises no atoms of its own.

Examples

scale_is(scale_example())
scale_is(data.frame(unit = "U1"))
scale_is_product(scale_product(a = scale_example(), b = scale_example2()))

Are the units of a frame the same size?

Description

TRUE when every unit of frame holds the same number of atoms, or the same total of a weight column: the dimension-free form of a "regular" calendar (equal durations) or an equal-area regionalisation. Atoms with no code at frame are ignored.

Usage

scale_is_uniform(x, frame, weight = NULL, tolerance = 1e-09)

Arguments

Value

A logical scalar. When FALSE, the attribute "sizes" holds the per-unit counts or totals.

Examples

s <- scale_example()
scale_is_uniform(s, "class")                  # two atoms each
scale_is_uniform(s, "class", weight = "size") # but not the same size

The atom key column name

Description

The atom key column name

Usage

scale_key(x)

Arguments

Value

A single string.

Examples

scale_key(scale_example())

The leaftable of a NestedScale

Description

The one-row-per-atom table the scale is built on, as a plain data.frame – the exported accessor to prefer over reaching for x@leaftable. as.data.frame() and ggplot2::fortify() on a NestedScale are equivalent, so ⁠ggplot(s) + geom_*()⁠ pipelines work directly.

Usage

scale_leaftable(x)

## S3 method for class 'NestedScale'
as.data.frame(x, row.names = NULL, optional = FALSE, ...)

## S3 method for class ''nestedscales::NestedScale''
as.data.frame(x, row.names = NULL, optional = FALSE, ...)

Arguments

Value

A data.frame: one row per atom, with the frame columns plus any weight columns.

Examples

head(scale_leaftable(scale_example()))
head(as.data.frame(scale_example()))

Crosswalk between two frames of a scale

Description

Materialises the from -> atoms -> to route as a table: one row per pair of overlapping units with

Usage

scale_map(x, source, target, weight = NULL, by = character(), ...)

Arguments

Details

• n_from – atoms in the source unit (its full set, before any target coverage is considered),

• n_overlap – atoms the pair shares,

• w – the weight of the overlap (summed atom weights, chosen weight column), the quantity "weighted_mean" aggregation uses,

• w_from – the full weight of the source unit; w / w_from is the split share "sum" disaggregation uses.

The two label columns are named by the frames; rows with an NA target label are atoms target does not cover. A crosswalk registered with register_scale_map() is returned as-is instead of being derived.

Value

A data.frame with columns ⁠<from>⁠, ⁠<to>⁠ (NA = uncovered by target), n_from, n_overlap, w, w_from, plus any by columns.

See Also

scale_map_between() for the map between two scales.

Examples

s <- scale_example()
scale_map(s, "class", "group")
scale_map(s, "sector", "class", weight = "count")

Crosswalk between two scales through their shared atoms

Description

The counterpart of scale_map() for two scales: atoms are matched on their keys, so a unit of source overlaps a unit of target where they contain the same atoms. Same columns as scale_map(), with the two label columns named after the scales. Atoms of source absent from target get an NA target (with a warning). A crosswalk registered with register_scale_map_between() is returned as-is instead of being derived.

Usage

scale_map_between(source, target, weight = NULL)

Arguments

Value

A data.frame with columns ⁠<from name>⁠, ⁠<to name>⁠, n_from, n_overlap, w, w_from.

Examples

a <- scale_example()
b <- scale_from_leaftable(
  data.frame(
    big = c("X", "X", "Y", "Y", "Y", "Y", "Z"),
    unit = c("U1", "U2", "U3", "U4", "U5", "U6", "OTH")
  ),
  frames = c("big", "unit"), name = "other"
)
scale_map_between(a, b)

Match labels to members, strictly

Description

Resolves each label to a ⁠(frame, unit)⁠ member in two stages, stopping at the first that hits: an exact member name (case-insensitive), then an alias (case-insensitive). A label that hits several members – the same name at two frames, or an alias shared by two members – is reported as "ambiguous" with the candidates, unless ⁠frame=⁠ narrows the search to one frame. A label that hits nothing is "none". Nothing is guessed: no fuzzy matching, no substring matching.

Usage

scale_match(x, labels, frame = NULL, source = NULL)

Arguments

Value

A data.frame with one row per label: label, frame, unit, method ("exact", "alias", "ambiguous", "none") and candidates (the frame/unit pairs of an ambiguous hit, joined by "; ", else NA).

Examples

s <- scale_example()
scale_match(s, c("P", "G1", "u1", "nope"))   # G1 is a class AND a group
scale_match(s, "G1", frame = "group")

Navigate a NestedScale's hierarchy

Description

Codes related to unit at another frame, found through the atoms. scale_children()/scale_parents() step one frame down/up by default; scale_descendants()/scale_ancestors() report every finer/coarser frame as a frame-tagged table.

Usage

scale_children(x, frame, unit, target = NULL)

scale_parents(x, frame, unit, target = NULL)

scale_descendants(x, frame, unit, target = NULL)

scale_ancestors(x, frame, unit, target = NULL)

Arguments

Value

A character vector (children/parents) or a data.frame with columns frame and unit (descendants/ancestors).

Examples

s <- scale_example()
scale_children(s, "sector", "P")
scale_parents(s, "unit", "U5")
scale_descendants(s, "sector", "P")
scale_ancestors(s, "unit", "U5")

Do two frames nest?

Description

Tests whether every code at the finer frame falls entirely within a single code at the coarser frame. Real hierarchies often fail this: a spatial code can straddle two parents, and a reporting classification can assign one detailed class to several aggregates.

Usage

scale_nests(x, parent, child)

Arguments

Details

Nesting is not required by NestedScale – conversion routes through the atom layer and works either way. This function is a diagnostic.

Value

TRUE or FALSE. When FALSE, the offending child codes are attached as the "offenders" attribute.

Examples

s <- scale_example()
scale_nests(s, "class", "group") # FALSE - they cross-cut

Collapse a subclass's per-atom payload

Description

The companion of scale_payload_slice() for the rebuilds that MERGE atoms rather than keep a subset of them – prune_scale(), which makes a coarser frame the new atom layer. Called with map, an integer vector with one entry per row of the OLD leaftable giving the index of the NEW row that atom was merged into, or NA for an atom that was dropped.

Usage

scale_payload_collapse(x, ...)

Arguments

Details

The NestedScale default returns x unchanged, which is right for a payload keyed by member codes (the core's own attribute and alias tables are re-keyed by the engine itself). A subclass whose payload is positional – one geometry per leaftable row – must override it and merge its payload by map; if it does not, its own validator rejects the length mismatch, so the failure is loud rather than a silently misaligned payload.

Value

x, with any payload merged to the new rows.

Examples

identical(
  scale_payload_collapse(scale_example(), c(1L, 1L, 2L, 2L, 3L, 3L, NA)),
  scale_example()
)

Write and read a subclass's payload in a definition file

Description

The seam a dimension package uses to carry what the core does not know about – geoscales' geometry – in a scale definition written by write_scale() and read by read_scale().

Usage

scale_write_payload(x, ...)

scale_read_payload(x, ...)

register_scale_class(class)

list_scale_classes()

Arguments

Details

scale_write_payload(x, dir, header) is called after the core tables are written, with the bundle directory (NULL for a single-file definition, which can hold inline data only) and the header so far; it writes its files and returns the header with its entries added under payload. scale_read_payload(x, dir, header) is called with the CORE scale rebuilt from the header and returns the subclass object. The NestedScale defaults return the header and x unchanged.

read_scale() finds the method through the class recorded in the header, so a dimension package registers its class once, in .onLoad(), with register_scale_class().

Value

scale_write_payload() the header; scale_read_payload() a scale; register_scale_class() the registry key, invisibly; list_scale_classes() a character vector of registered class names.

Examples

list_scale_classes()

Subset a subclass's per-atom payload

Description

The seam a dimension package overrides when it carries per-atom data alongside the leaftable – geoscales' Geoscale and its geometry. Called whenever the engine rebuilds an object from a row subset, with the kept row indices. The NestedScale default has no payload and returns x unchanged.

Usage

scale_payload_slice(x, ...)

Arguments

Value

x, with any payload subset to the kept rows.

Examples

identical(
  scale_payload_slice(scale_example(), 1:3),
  scale_example()
)

Combine scales into a product index

Description

Combine scales into a product index

Usage

scale_product(..., name = "", desc = "", keys = NULL, joint_weights = NULL)

Arguments

Value

A ScaleProduct.

Examples

p <- scale_product(a = scale_example(), b = scale_example2())
p
product_size(p)

Frame rank

Description

Position of a frame in the hierarchy: 1 is the coarsest.

Usage

scale_rank(x, frame)

Arguments

Value

An integer vector of ranks; NA for names that are not frames of x.

Examples

s <- scale_example()
scale_rank(s, c("group", "unit"))

Units that are residuals

Description

A residual carries a real value but is not a destination for allocation: non-regionalized GDP, taxes that belong to no industry, an "n.e.c." bucket. It aggregates upward like any other unit – that is the point, it is what makes the parent total reconcile – but recast_scale() never splits a coarse figure INTO one.

Usage

scale_residuals(x, frame = NULL)

Arguments

Details

Declare them when building the scale, per frame, because codes are not unique across frames: scale_from_leaftable(..., residuals = list(unit = "DE_XR")).

Value

A character vector of unit codes, or the named list over frames. Empty when the scale declares none.

Examples

s <- scale_example()
scale_residuals(s) # none declared

lf <- scale_leaftable(s)
r <- scale_from_leaftable(
  lf,
  frames = scale_frames(s), key = "unit",
  weights = scale_weights(s), name = "with_residual",
  residuals = list(unit = "OTH")
)
scale_residuals(r, "unit")

Weight shares within a frame

Description

Normalised weights, either of the whole object or within each parent group.

Usage

scale_share(x, frame, weight = NULL, within = NULL)

Arguments

Value

A data.frame with a code column named frame, the weight, and share. When within is given, a column of that name carries the parent code.

Examples

s <- scale_example()
scale_share(s, "class", weight = "size")
scale_share(s, "class", weight = "size", within = "sector")

Units of a NestedScale

Description

The code vocabulary at one frame, in canonical order.

Usage

scale_units(x, frame = NULL)

Arguments

Value

A character vector of unit codes.

Examples

s <- scale_example()
scale_units(s)
scale_units(s, "group")

The dimension vocabulary of a scale

Description

Engine messages are written once and interpolated with the words of the dimension at hand, so a Calendar flowing through the shared engine still errors in the language of time ("timeframe", "timeslice") and a Geoscale in the language of space ("geoframe", "region"). Subclasses supply their own method; the NestedScale default is the neutral vocabulary.

Usage

scale_vocab(x, ...)

Arguments

Value

A named list with elements object, frame, frames, unit, units, atoms, and optionally register_rule and register_map: the calls that error messages suggest for registering a value column's rule or an explicit crosswalk, when the dimension package offers its own ("register_geoscale_rule()").

Examples

scale_vocab(scale_example())

Weight columns of a NestedScale

Description

Weight columns of a NestedScale

Usage

scale_weights(x)

Arguments

Value

A character vector of weight column names (possibly empty).

Examples

scale_weights(scale_example())

Shipped definitions: carriers and the IPCC 2006 fuel defaults

Description

nestedscales ships one scale definition, carriers, under ⁠inst/scales/⁠, and the IPCC tables it draws on under ⁠inst/extdata/⁠. read_scale("carriers") reads the scale; scale_catalog() lists it with every other registered definition.

carriers

A scale of 71 fuels in three nested frames, ⁠group (8) > carrier (17) > fuel (71)⁠, key fuel, taken from the carriers palette of the energypal package (its inst/extdata/palettes/carriers.yml, which cites the IPCC 2006 Guidelines and the IEA World Energy Balances for its classification). energypal remains the editable source of the names, hierarchy, labels and aliases; the definition is a snapshot built by data-raw/carriers.R, which stops when the two disagree.

The three carriers energypal declares without subtypes (Electricity, Heat, Other) become their own fuel, so every carrier is reachable from the atoms. Five names appear both as a group and as the carrier inside it (Nuclear, Electricity, Heat, Storage, Other), and Hydrogen is both a carrier (under Synthetic) and a fuel (under Storage): every lookup names its frame.

Attributes (source "IPCC2006", 26 rows): the default net calorific value lhv (kind property, GJ/t) and CO2 emission factor ef_co2 (kind factor, kg/TJ) of the IPCC 2006 Guidelines, Vol. 2 Ch. 1, Tables 1.2 and 1.4, attached to the 13 fuels whose definition matches an IPCC fuel exactly. Fuels without a matching IPCC entry carry no value; scale_attributes() reports them as NA rather than inventing one.

Aliases (136 rows): energypal's alternative spellings (source "energypal") and a set of model commodity codes such as GAS, LIG, ELC (source "commodity"). scale_match() resolves them.

meta$labels carries energypal's short and long labels per member; meta$energypal_version and meta$palette_version record the snapshot. Colours are not stored: join them from energypal by name.

IPCC 2006 default calorific values and CO2 emission factors

system.file("extdata", "ipcc2006_fuel_defaults.csv", package = "nestedscales") holds every row of Table 1.2 (default net calorific values, TJ/Gg, with the lower and upper limits of the 95% confidence interval) and Table 1.4 (default CO2 emission factors for combustion, kg/TJ, with the 95% interval) of the IPCC 2006 Guidelines, Vol. 2 (Energy), Chapter 1, in long form, together with how each IPCC fuel maps onto carriers. TJ/Gg is numerically identical to GJ/t, the unit the lhv attribute uses. One row per IPCC fuel and attribute:

ipcc_fuel

The fuel's English description in the Guidelines.

attribute

"lhv" (Table 1.2) or "ef_co2" (Table 1.4).

kind

"property" or "factor", see SCALE_ATTRIBUTE_KINDS.

value, lower, upper

The default and its 95% interval.

unit_of_measure

"GJ/t" or "kg/TJ".

table

"1.2" or "1.4".

source

"IPCC2006".

frame, unit

The carriers member the row maps to (NA when not mapped).

mapping_status

"mapped" (an exact or definitional correspondence), "review" (a plausible correspondence awaiting a ruling; not attached) or "unmapped" (no counterpart).

primary

Whether this IPCC fuel is the one carriers carries for that member. Only mapped rows with primary = TRUE enter the scale's attributes.

Source

energypal carriers.yml; IPCC (2006), 2006 IPCC Guidelines for National Greenhouse Gas Inventories, Volume 2 (Energy), Chapter 1, Tables 1.2 and 1.4, https://www.ipcc-nggip.iges.or.jp/public/2006gl/. Transcribed in ⁠data-raw/ipcc2006/⁠.

See Also

read_scale(), scale_catalog(), convert_scale(); vignette("attributes").

Examples

carriers <- read_scale("carriers")
carriers
scale_units(carriers, "carrier")
scale_attributes(carriers, "lhv", frame = "fuel")
scale_match(carriers, c("GAS", "LIG", "Hydrogen"))

ipcc <- read.csv(system.file("extdata", "ipcc2006_fuel_defaults.csv",
                             package = "nestedscales"))
table(ipcc$attribute, ipcc$mapping_status)

Summarize a NestedScale

Description

Complements print() with the quantitative view: per-weight totals and coverage, and the adjacent-frame nesting table. Returns a "summary_Scale" object (a list) with its own print method.

Usage

## S3 method for class 'NestedScale'
summary(object, ...)

## S3 method for class ''nestedscales::NestedScale''
summary(object, ...)

## S3 method for class 'summary_Scale'
print(x, ...)

Arguments

Value

summary() returns a list of class "summary_Scale": name, desc, frames (named member counts), unassigned (named NA-atom counts), n_atoms, weights, weight_totals, default_weight, coverage (see scale_coverage()), sampled, parent_name, nesting (adjacent-pair table with offender counts, see scale_nests()), attributes (a data.frame of row counts by attribute, frame and source), aliases (counts by source), source, vocab.

Examples

summary(scale_example())

Summarize a product

Description

Summarize a product

Usage

## S3 method for class 'ScaleProduct'
summary(object, ...)

## S3 method for class ''nestedscales::ScaleProduct''
summary(object, ...)

## S3 method for class 'summary_ScaleProduct'
print(x, ...)

Arguments

Value

A list of class "summary_ScaleProduct": name, desc, axes (per-axis summary rows), total, coverage, joint_coverage, joint_weights.

Examples

summary(scale_product(a = scale_example(), b = scale_example2()))

Write a scale as a definition file, and read one back

Description

A definition is a header (JSON, the stored form; YAML on request) plus tables. A small scale fits in one file with every table inline; a large one is a bundle directory holding scale.json and its tables as Arrow IPC files (or CSV when arrow is not installed). Either form is language-neutral: the header is plain JSON and the tables are Arrow or CSV with their column types recorded.

Usage

write_scale(
  x,
  path,
  format = c("json", "yaml"),
  tables = c("auto", "inline", "arrow", "csv"),
  overwrite = FALSE
)

read_scale(path, package = NULL, class = c("stored", "core"))

Arguments

Details

path ending in .json, .yaml or .yml writes a single file; any other path is a bundle directory. A bundle is only ever written into an empty directory or over a bundle written before (overwrite = TRUE); nothing else in a directory is touched.

read_scale() takes a file, a bundle directory, or the id of a shipped definition (see scale_catalog()). The scale comes back as the class it was written from when that class is registered (see register_scale_class()); class = "core" asks for the plain NestedScale instead, dropping any payload.

Value

write_scale() returns path, invisibly. read_scale() returns the scale.

Round trips

An arrow bundle reads back identical. A JSON or YAML header carries numbers at 15 significant digits, so an inline table reads back equal (all.equal()), not necessarily identical, for arbitrary doubles.

Examples

s <- scale_example()
f <- file.path(tempdir(), "example.json")
write_scale(s, f, overwrite = TRUE)
back <- read_scale(f)
identical(scale_frames(back), scale_frames(s))
all.equal(scale_leaftable(back), scale_leaftable(s))

# a bundle, with the tables as files
if (requireNamespace("arrow", quietly = TRUE)) {
  d <- file.path(tempdir(), "example.scale")
  write_scale(s, d, tables = "arrow", overwrite = TRUE)
  list.files(d)
  identical(scale_leaftable(read_scale(d)), scale_leaftable(s))
}

Write data and the scales that index it to one folder

Description

The data is written as a hive-partitioned Arrow IPC dataset and the scales travel with it, so open_scale_dataset() gives back both and the result can be recast without re-declaring anything. Each scale is stored as the definition write_scale() writes, so its attributes and aliases travel too.

Usage

write_scale_dataset(
  data,
  x,
  path,
  partitioning = NULL,
  compression = "zstd",
  overwrite = FALSE
)

Arguments

Value

path, invisibly.

Examples

p <- scale_product(a = scale_example(), b = scale_example2())
d <- merge(
  data.frame(unit = c("U1", "U2")),
  data.frame(period = c("p1", "p2"))
)
d$v <- 1:4
dir <- file.path(tempdir(), "store")
if (requireNamespace("arrow", quietly = TRUE)) {
  write_scale_dataset(d, p, dir, overwrite = TRUE)
  ds <- open_scale_dataset(dir)
  ds
}