Package {ICCDesign}

ICCDesign: Intraclass Correlation Coefficient Analysis and Study Planning

Description

A comprehensive, user-friendly R package for intraclass correlation coefficient (ICC) analysis and sample size planning. Implements the full McGraw & Wong (1996) framework supporting all 10 ICC types, with an intuitive 4-question decision system that eliminates the need for users to memorize ICC type codes.

The package provides both a command-line interface for advanced users and a fully interactive Shiny web application for point-and-click analysis. It also includes automated reliability evaluation based on Koo & Li (2016) criteria, publication-ready report generation, and rigorous sample size and power calculation functions.

Key Features

• Full support for all 10 ICC types from McGraw & Wong (1996)

• Intuitive 4-question decision framework for ICC type selection

• Automated reliability rating and interpretation

• Publication-ready text, Markdown, and HTML reports

• Sample size calculation based on confidence interval lower bound or width

• Power analysis for existing study designs

• Interactive Shiny web application with data upload and visualization

• Comprehensive data validation and error handling

Main Functions

icc_calc

Top-level function for complete ICC analysis

icc_sample_size

Unified interface for sample size and power calculation

run_icc_app

Launch the interactive Shiny application

icc_preprocess_data

Data preprocessing and validation utility

Author(s)

Maintainer: Ziyu Liu 1755454769@qq.com

Authors:

• Ziyu Liu 1755454769@qq.com

• Ruilin Ma

• Chenge Gao

• Yundan Zhang

References

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30-46.

Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155-163.

See Also

Useful links:

• https://github.com/KlariZhang/ICCDesign

• Report bugs at https://github.com/KlariZhang/ICCDesign/issues

Calculate Intraclass Correlation Coefficient (ICC)

Description

Top-level main function for complete ICC analysis. Users only need to provide raw data and answer 4 design questions. The function automatically handles data preprocessing, parameter validation, ICC type mapping, core calculation, reliability evaluation, and report generation.

Usage

icc_calc(
  data,
  same_raters,
  rater_effect = NULL,
  rating_type,
  agreement_type = NULL,
  alpha = 0.05,
  rho0 = NULL,
  interaction = TRUE,
  na.rm = TRUE,
  verbose = TRUE
)

Arguments

Value

A named list containing:

data_summary

List. Data preprocessing summary.

icc_result

List. Full ICC calculation results.

evaluation

List. Reliability evaluation results.

report

Character. Standardized report text.

warning_msg

Character or NULL. Warning message.

tip_msg

Character or NULL. Tip message.

References

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30-46.

Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155-163.

Examples

# Example 0: Use built-in example dataset
data(icc_data)
result <- icc_calc(icc_data, same_raters = TRUE, rater_effect = "random",
                   rating_type = "single", agreement_type = "absolute")
# Example 1: One-way random effects, single rating (ICC(1,1))
data <- matrix(rnorm(100), nrow = 20, ncol = 5)
result <- icc_calc(data, same_raters = FALSE, rating_type = "single")

# Example 2: Two-way random effects, average rating, absolute agreement (ICC(2,k))
result <- icc_calc(data, same_raters = TRUE, rater_effect = "random",
                   rating_type = "average", agreement_type = "absolute")

# Example 3: Two-way mixed effects, single rating, consistency (ICC(3,1))
result <- icc_calc(data, same_raters = TRUE, rater_effect = "fixed",
                   rating_type = "single", agreement_type = "consistency")

# Example 4: Special scenario - automatic mapping with tip
# Random effects + consistency (automatically mapped to ICC(3,1))
result <- icc_calc(data, same_raters = TRUE, rater_effect = "random",
                   rating_type = "single", agreement_type = "consistency")

# Example 5: Special scenario - not recommended combination with warning
# Fixed effects + absolute agreement (NOT RECOMMENDED)
result <- icc_calc(data, same_raters = TRUE, rater_effect = "fixed",
                   rating_type = "average", agreement_type = "absolute")

# Example 6: Advanced parameters - custom alpha and non-zero test
result <- icc_calc(data, same_raters = TRUE, rater_effect = "random",
                   rating_type = "single", agreement_type = "absolute",
                   alpha = 0.1, rho0 = 0.6, verbose = FALSE)

# Example 7: Extract specific results
result$icc_result$point_est  # ICC point estimate
result$evaluation$rating_en  # Reliability rating
result$report               # Full text report

Calculate ICC(1,1)

Description

Calculates the Intraclass Correlation Coefficient (ICC) for a one-way random effects model using a single rater/measurement, focusing on absolute agreement.

Usage

icc_calc_1_1(data_matrix, alpha = 0.05, rho0 = NULL, interaction = TRUE)

Arguments

Value

A standardized list containing ICC results, see package documentation for details.

References

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30-46.

Calculate ICC(1,k)

Description

Calculates the Intraclass Correlation Coefficient (ICC) for a one-way random effects model using the average of k raters/measurements, focusing on absolute agreement.

Usage

icc_calc_1_k(data_matrix, alpha = 0.05, rho0 = NULL, interaction = TRUE)

Arguments

Value

A standardized list containing ICC results.

References

McGraw, K. O., & Wong, S. P. (1996).

Calculate ICC(2,1)

Description

Calculates the Intraclass Correlation Coefficient (ICC) for a two-way random effects model using a single rater, focusing on absolute agreement.

Usage

icc_calc_2_1(data_matrix, alpha = 0.05, rho0 = NULL, interaction = TRUE)

Arguments

Value

A standardized list containing ICC results.

References

McGraw, K. O., & Wong, S. P. (1996).

Calculate ICC(2,1,consistency) [Supplementary]

Description

Calculates ICC for two-way random effects, single rating, consistency. Note: In practice, this maps to ICC(3,1) (mixed effects model).

Usage

icc_calc_2_1_consistency(
  data_matrix,
  alpha = 0.05,
  rho0 = NULL,
  interaction = TRUE
)

Arguments

Value

A standardized list containing ICC results.

References

McGraw, K. O., & Wong, S. P. (1996).

Calculate ICC(2,k)

Description

Calculates the Intraclass Correlation Coefficient (ICC) for a two-way random effects model using the average of k raters, focusing on absolute agreement.

Usage

icc_calc_2_k(data_matrix, alpha = 0.05, rho0 = NULL, interaction = TRUE)

Arguments

Value

A standardized list containing ICC results.

References

McGraw, K. O., & Wong, S. P. (1996).

Calculate ICC(2,k,consistency) [Supplementary]

Description

Calculates ICC for two-way random effects, average of k ratings, consistency. Note: In practice, this maps to ICC(3,k) (mixed effects model).

Usage

icc_calc_2_k_consistency(
  data_matrix,
  alpha = 0.05,
  rho0 = NULL,
  interaction = TRUE
)

Arguments

Value

A standardized list containing ICC results.

References

McGraw, K. O., & Wong, S. P. (1996).

Calculate ICC(3,1)

Description

Calculates the Intraclass Correlation Coefficient (ICC) for a two-way mixed effects model using a single rater, focusing on consistency.

Usage

icc_calc_3_1(data_matrix, alpha = 0.05, rho0 = NULL, interaction = TRUE)

Arguments

Value

A standardized list containing ICC results.

References

McGraw, K. O., & Wong, S. P. (1996).

Calculate ICC(3,1,absolute) [Supplementary]

Description

Calculates ICC for two-way mixed effects, single rating, absolute agreement. Note: This combination is rarely used and not recommended.

Usage

icc_calc_3_1_absolute(
  data_matrix,
  alpha = 0.05,
  rho0 = NULL,
  interaction = TRUE
)

Arguments

Value

A standardized list containing ICC results.

References

McGraw, K. O., & Wong, S. P. (1996).

Calculate ICC(3,k)

Description

Calculates the Intraclass Correlation Coefficient (ICC) for a two-way mixed effects model using the average of k raters, focusing on consistency.

Usage

icc_calc_3_k(data_matrix, alpha = 0.05, rho0 = NULL, interaction = TRUE)

Arguments

Value

A standardized list containing ICC results.

References

McGraw, K. O., & Wong, S. P. (1996).

Calculate ICC(3,k,absolute) [Supplementary]

Description

Calculates ICC for two-way mixed effects, average of k ratings, absolute agreement. Note: This combination is rarely used and not recommended.

Usage

icc_calc_3_k_absolute(
  data_matrix,
  alpha = 0.05,
  rho0 = NULL,
  interaction = TRUE
)

Arguments

Value

A standardized list containing ICC results.

References

McGraw, K. O., & Wong, S. P. (1996).

Calculate ANOVA for ICC Models

Description

Unified function to compute mean squares and degrees of freedom for one-way random and two-way random/mixed ANOVA models. Serves as the foundational calculation for all ICC types, eliminating redundant code.

Usage

icc_calc_anova(data_matrix, model_type, interaction = TRUE)

Arguments

Value

A named list containing ANOVA results:

MSR

Numeric. Mean square for subjects (rows).

MSC

Numeric. Mean square for raters (columns) (NULL for one-way model).

MSE

Numeric. Residual mean square (two-way model only).

MSW

Numeric. Within-group mean square (one-way model only).

df1

Integer. Degrees of freedom for subjects (n-1).

df2

Integer. Residual degrees of freedom.

df3

Integer. Degrees of freedom for raters (k-1) (NULL for one-way model).

n

Integer. Number of subjects.

k

Integer. Number of raters.

Hypothesis Testing for ICC

Description

Performs one-tailed F-test for ICC significance following McGraw & Wong (1996) Table 8. Supports both null hypothesis of zero ICC and custom non-zero null value.

Usage

icc_calc_f_test(anova_result, icc_type, rho0 = 0, alpha = 0.05)

Arguments

Value

Named list with test results:

H0

Character. Null hypothesis statement.

F_stat

Numeric. F test statistic.

df1

Integer. Numerator degrees of freedom.

df2

Integer. Denominator degrees of freedom.

p_value

Numeric. One-tailed p-value (H1: ICC > rho0).

References

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30-46.

Validate ICC Design Parameters

Description

Validates the legality of 4 core user-input design parameters, intercepts invalid inputs in advance, and generates standardized warnings and tips to avoid runtime errors during ICC calculation.

Usage

icc_check_design(
  same_raters,
  rater_effect = NULL,
  rating_type,
  agreement_type = NULL,
  k
)

Arguments

Value

A named list containing:

is_valid

Logical. Whether the design parameters are valid.

error_msg

Character or NULL. Error message for invalid parameters, NULL if valid.

warning_msg

Character or NULL. Warning message for non-recommended scenarios, NULL if none.

tip_msg

Character or NULL. Tip message for automatic mapping scenarios, NULL if none.

References

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30-46.

Example ICC Dataset

Description

A small, carefully constructed example dataset containing ratings from 4 raters on 5 subjects. Designed to demonstrate all package functionality with fast execution and predictable results.

Format

A numeric matrix with 5 rows (subjects) and 4 columns (raters). Row names are "Subject1" to "Subject5", column names are "Rater1" to "Rater4".

Details

This dataset can be used to calculate all 10 ICC types supported by the package, simply by changing the design parameters in icc_calc.

The dataset was simulated to have an approximate ICC of 0.8, which falls into the "Good" reliability category according to Koo & Li (2016).

Source

Simulated data for demonstration purposes.

Examples

data(icc_data)
head(icc_data)

Evaluate ICC Reliability

Description

Evaluates the reliability of an ICC result based on the 95 interval lower bound, following the criteria of Koo & Li (2016).

Usage

icc_evaluate(icc_result)

Arguments

Value

A named list with elements:

• icc_code: Character. ICC code.

• point_est: Numeric. ICC point estimate.

• ci_lower: Numeric. 95

• rating_en: Character. English reliability rating.

• explanation: Character. Interpretation text.

References

Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155-163.

Generate ICC Report

Description

Generates a standardized, publication-ready report of ICC results, supporting text, Markdown, and HTML formats.

Usage

icc_generate_report(icc_result, evaluation, format = "text")

Arguments

Value

Character. Formatted report text.

Map Design Parameters to ICC Type

Description

Maps user's 4 design questions to the corresponding ICC core calculation function, full name, and any necessary warnings or tips.

Usage

icc_map_design_to_icc(same_raters, rater_effect, rating_type, agreement_type)

Arguments

Value

A named list containing:

icc_func_name

Character. Name of the core ICC calculation function.

icc_full_name

Character. Full name of the ICC type.

icc_code

Character. ICC code (e.g., "1,1").

warning_msg

Character or NULL. Warning message for not recommended combinations.

tip_msg

Character or NULL. Tip message for automatically mapped combinations.

References

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30-46.

Power Calculation for ICC Study Design

Description

Power Calculation for ICC Study Design

Usage

icc_power(
  n,
  rho,
  rho0 = NULL,
  omega = NULL,
  k = 3,
  same_raters,
  rater_effect = NULL,
  rating_type,
  agreement_type = NULL,
  alpha = 0.05,
  method = c("lower", "width"),
  verbose = TRUE
)

Arguments

Value

Power.

Examples

# Note: It is recommended to use the unified interface icc_sample_size()
icc_power(n = 30, rho = 0.7, rho0 = 0.5, k = 3,
          same_raters = TRUE, rater_effect = "fixed",
          rating_type = "single", agreement_type = "consistency")

Preprocess and Validate Data for ICC Analysis

Description

Performs standardized data cleaning, format conversion, and legality validation for raw input data. Provides a unified valid data input for all ICC calculation functions to avoid repetitive validation code.

Usage

icc_preprocess_data(data, na.rm = TRUE)

Arguments

Value

A named list containing:

data_matrix

Numeric matrix. Standardized numeric matrix (no missing values if na.rm = TRUE).

n

Integer. Number of valid subjects (rows).

k

Integer. Number of raters/repeated measurements (columns).

warning_msg

Character or NULL. Warning message for missing values, NULL if no missing values.

error_msg

Character or NULL. Error message for invalid data, NULL if data is valid.

Examples

# Preprocess the built-in example dataset
data(icc_data)
processed <- icc_preprocess_data(icc_data)
str(processed)

Unified ICC Sample Size & Power Interface

Description

Unified ICC Sample Size & Power Interface

Usage

icc_sample_size(method = c("lower", "width", "power"), ...)

Arguments

Value

Sample size or power.

Examples

# Method 1: Sample size based on lower confidence limit (most recommended)
# Ensure 95% CI lower bound >= 0.75 (good reliability)
n1 <- icc_sample_size(
  method = "lower",
  rho = 0.85,
  rating_target = "good",
  k = 3,
  same_raters = TRUE,
  rater_effect = "random",
  rating_type = "single",
  agreement_type = "absolute"
)

# Method 2: Sample size based on confidence interval width
# Ensure 95% CI half-width <= 0.1
n2 <- icc_sample_size(
  method = "width",
  rho = 0.7,
  omega = 0.1,
  k = 3,
  same_raters = FALSE,
  rating_type = "average"
)

# Method 3: Power calculation for existing study design
power <- icc_sample_size(
  method = "power",
  n = 30,
  rho = 0.7,
  rho0 = 0.5,
  k = 3,
  same_raters = TRUE,
  rater_effect = "fixed",
  rating_type = "single",
  agreement_type = "consistency"
)

Sample Size for ICC based on Lower Confidence Limit

Description

Sample Size for ICC based on Lower Confidence Limit

Usage

icc_sample_size_lower(
  rho,
  rho0 = NULL,
  k = 3,
  same_raters,
  rater_effect = NULL,
  rating_type,
  agreement_type = NULL,
  alpha = 0.05,
  assurance = 0.8,
  rating_target = NULL,
  verbose = TRUE
)

Arguments

Value

Required sample size.

Examples

# Note: It is recommended to use the unified interface icc_sample_size()
icc_sample_size_lower(rho = 0.8, rho0 = 0.6, k = 3, same_raters = FALSE, rating_type = "single")

Sample Size for ICC based on Confidence Interval Width

Description

Sample Size for ICC based on Confidence Interval Width

Usage

icc_sample_size_width(
  rho,
  omega,
  k = 3,
  same_raters,
  rater_effect = NULL,
  rating_type,
  agreement_type = NULL,
  alpha = 0.05,
  assurance = 0.8,
  verbose = TRUE
)

Arguments

Value

Required sample size.

Examples

# Note: It is recommended to use the unified interface icc_sample_size()
icc_sample_size_width(rho = 0.7, omega = 0.1, k = 3, same_raters = FALSE, rating_type = "average")

Build the ICCDesign Shiny Application

Description

Build the ICCDesign Shiny Application

Usage

icc_shiny_app()

Value

A Shiny application object.

Calculate ICC Confidence Intervals

Description

Computes confidence intervals for all 10 ICC types using exact F-distribution method from McGraw & Wong (1996) Table 7. Implements Satterthwaite degrees of freedom correction for Type A ICC models (ICC(2,1), ICC(2,k) and their variants).

Usage

icc_tool_ci(anova_result, icc_type, point_est, alpha = 0.05)

Arguments

Value

A named list with CI results:

ci_level

Numeric. Confidence level (1 - alpha).

ci_lower

Numeric. Lower bound of CI (truncated to 0).

ci_upper

Numeric. Upper bound of CI (truncated to 1).

df_corrected

Numeric or NULL. Satterthwaite-corrected df (Type A only).

References

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30-46.

Calculate ICC Point Estimate

Description

Unified low-level function to compute point estimates for all 10 ICC types based on the standard formulas from McGraw & Wong (1996) Table 4 and 5. Serves as the single source of truth for ICC point calculations.

Usage

icc_tool_point(anova_result, icc_type)

Arguments

Value

Numeric. Point estimate of the intraclass correlation coefficient.

References

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30-46.

Launch the ICCDesign Shiny Application

Description

Starts an interactive Shiny application for ICC analysis, reliability reporting, and sample size or power planning.

Usage

run_icc_app(
  host = "127.0.0.1",
  port = NULL,
  launch.browser = interactive(),
  ...
)

Arguments

Value

Runs the Shiny application.

Examples

if (interactive()) {
# Launch the interactive Shiny application
run_icc_app()
}