---
title: "Sequence Data Validation and Preparation"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Sequence Data Validation and Preparation}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
library(gp3sequences)
```

## Why preparation is explicit

Ordered categorical data can contain missing states, duplicated positions,
unsorted rows, consecutive repeats, zero durations, unknown states, and
inconsistent metadata. Silent repair can change the analytical object.
`gp3sequences` therefore separates non-modifying audit and validation from
policy-driven preparation.

## A deliberately problematic synthetic input

The example includes an unsorted sequence, a duplicated position, a missing
state, a consecutive repeat, a zero duration, an unexpected state, and an
unused factor level. Participant and group metadata remain constant within
each sequence.

```{r problematic-data}
problem_data <- data.frame(
  sequence_id = c("s2", "s1", "s1", "s1", "s1", "s2", "s2", "s2", "s2"),
  sequence_order = c(2, 2, 1, 2, 3, 1, 2, 3, 4),
  state = factor(
    c("search", "search", "home", "search", NA,
      "home", "home", "product", "other"),
    levels = c("home", "search", "product", "checkout", "other", "unused")
  ),
  duration = c(120, 100, 90, 110, 80, 100, 0, 150, 130),
  participant_id = c("p2", "p1", "p1", "p1", "p1", "p2", "p2", "p2", "p2"),
  group = c("interface_b", "interface_a", "interface_a", "interface_a",
            "interface_a", "interface_b", "interface_b", "interface_b",
            "interface_b"),
  stringsAsFactors = FALSE
)

expected_states <- c("home", "search", "product", "checkout")
problem_data
```

## Audit without modification

`audit_sequence_data()` reports one row per issue using stable issue codes and
severity values. It does not repair the data.

```{r audit}
audit <- audit_sequence_data(
  problem_data,
  sequence_id_col = "sequence_id",
  order_col = "sequence_order",
  state_col = "state",
  duration_col = "duration",
  metadata_cols = c("participant_id", "group"),
  expected_states = expected_states
)

audit
as.data.frame(table(audit$severity), stringsAsFactors = FALSE)
as.data.frame(table(audit$issue_code), stringsAsFactors = FALSE)
```

## Compact validation contract

A review-level issue does not automatically invalidate an input. Error-level
issues must be resolved through source correction or an explicit supported
policy.

```{r validation}
validation <- validate_sequence_data(
  problem_data,
  sequence_id_col = "sequence_id",
  order_col = "sequence_order",
  state_col = "state",
  duration_col = "duration",
  metadata_cols = c("participant_id", "group"),
  expected_states = expected_states
)

validation[c("valid", "status", "n_errors", "n_reviews", "n_info")]
validation$mapping
```

## Apply explicit preparation policies

This example deliberately chooses to:

- drop rows with missing states;
- retain the first row at duplicated positions;
- collapse consecutive repeated states;
- drop zero-duration rows;
- drop states absent from the declared state set;
- drop unused factor levels.

These are analytical choices, not universal defaults.

```{r prepare}
prepared <- prepare_sequence_data(
  problem_data,
  sequence_id_col = "sequence_id",
  order_col = "sequence_order",
  state_col = "state",
  duration_col = "duration",
  metadata_cols = c("participant_id", "group"),
  expected_states = expected_states,
  missing_state_policy = "drop",
  duplicate_position_policy = "first",
  repeated_state_policy = "collapse",
  zero_duration_policy = "drop",
  unknown_state_policy = "drop",
  unused_state_levels = "drop"
)

prepared$status
prepared$decisions
prepared$data
prepared$audit
```

## Revalidate the canonical result

The prepared table uses stable canonical columns while preserving unmapped
metadata and original-row provenance.

```{r revalidate}
revalidation <- validate_sequence_data(
  prepared$data,
  sequence_id_col = "sequence_id",
  order_col = "sequence_order",
  state_col = "state",
  duration_col = "duration",
  metadata_cols = c("participant_id", "group"),
  expected_states = expected_states
)

revalidation[c("valid", "status", "n_errors", "n_reviews", "n_info")]
prepared$mapping
prepared$state_levels
```

## Errors that require source correction

Some conditions are intentionally not repaired automatically. Examples include
missing sequence identifiers, missing or non-finite order values, negative or
non-finite durations, absent mapped columns, duplicated column names, invalid
column types, and metadata that varies within a sequence. These conditions
require correction or an explicit redefinition of the sequence unit.

## Reporting recommendations

A reproducible report should record the input mapping, expected states, every
preparation policy, the audit table, the decision log, original and prepared
row counts, and the final state levels. These records describe data handling;
they do not validate a substantive interpretation of the resulting sequence
patterns.