Getting started with ihsMW

The ihsMW package is a dedicated toolkit designed to clean, harmonise, and aggregate household survey data from the Malawi Integrated Household Survey (IHS) series. It supports IHS2 (2004/05), IHS3 (2010/11), IHS4 (2016/17), IHS5 (2019/20) and IHS6 (2024/25).

For a complete standalone manual covering every function, worked end-to-end workflows, troubleshooting and FAQs, see the ihsMW User Guide, also available as a PDF and as R Markdown source.

1. Installation

You can install the stable release of ihsMW from CRAN:

install.packages("ihsMW")

Or install the development version from GitHub:

# Install using pak
pak::pak("vituk123/ihsMW")

# Or using remotes
remotes::install_github("vituk123/ihsMW")

2. The IHS Landscape

The Malawi National Statistical Office (NSO) conducts the Integrated Household Survey (IHS) periodically to track poverty, household expenditure, agriculture, and other socio-economic indicators. The primary rounds include:

Due to licensing restrictions, the raw microdata cannot be redistributed directly within R packages. Researchers must first register and manually download the survey data in Stata (.dta) format from the World Bank Microdata Library.

Once downloaded, place the files in a structured folder hierarchy on your local machine.

3. Loading and Harmonising

Each round of the IHS uses different variable names for the same question. For example, household size is recorded under different column names depending on the round. ihsMW uses a comprehensive crosswalk to harmonise these variable names.

To load and harmonise a raw survey file:

library(ihsMW)
library(haven)

# Load the raw Stata file
raw_data <- read_dta("path/to/IHS5/hh_mod_a_filt.dta")

# Harmonise variables to standard names
harmonised_data <- ihs_harmonise(raw_data, round = "IHS5")

4. Searching Variables

To find variables mapped in the crosswalk, use ihs_search(). You can search by keywords or labels:

# Search for consumption-related variables
ihs_search("consumption")

# Search for age within a specific round
ihs_search("age", round = "IHS5")

To view a summary of the crosswalk coverage and flag variables needing review, use ihs_crosswalk_check():

ihs_crosswalk_check()

5. Data Cleaning

ihsMW provides tools to clean standard survey anomalies, handle missing value codes, and winsorize extreme values:

# Convert standard survey missing codes (-99, -98, etc.) to NA
df_clean <- ihs_standardize_missing(harmonised_data)

# Winsorize outliers (e.g. food expenditure) stratified by urban/rural.
# This adds a `food_exp_w` column and leaves `food_exp` untouched.
df_winsor <- ihs_winsorize(df_clean, vars = "food_exp", by = "urban")

# Run the master cleaning wrapper which applies both steps and logs changes
df_cleaned <- ihs_clean(
  data = harmonised_data,
  winsorize_vars = c("food_exp", "nonfood_exp"),
  winsorize_by = "urban"
)

# Every transformation is recorded on the result
str(attr(df_cleaned, "ihs_audit"))

6. Unit Conversion

Agricultural modules in the IHS allow households to report harvest quantities in non-standard units (e.g., pails, basins, ox-carts, bags) rather than standard kilograms. ihsMW bundles official NSO conversion factors to convert these quantities to standard kilograms:

library(ihsMW)
#> ihsMW: Dedicated offline cleaning suite loaded.

# A `region` column, if present, is detected automatically and used to pick
# region-specific factors. There is no `region_col` argument.
crop_data <- data.frame(
  crop_code = c(1, 2),
  unit_code = c(3, 4),
  quantity  = c(10, 5),
  region    = c(1, 2)
)

crop_data_kg <- ihs_convert_units(
  data     = crop_data,
  crop_col = "crop_code",
  unit_col = "unit_code",
  qty_col  = "quantity"
)

crop_data_kg
#>   crop_code unit_code quantity region quantity_kg
#> 1         1         3       10      1         945
#> 2         2         4        5      2          25

Combinations with no official factor return NA rather than a guessed number, and the function tells you which ones failed.

7. Aggregation

To aggregate member-level or agricultural plot-level data up to the household level, use ihs_aggregate(). It takes a single grouping column and infers a sensible aggregation for every other column:

member_data <- data.frame(
  case_id           = c("A", "A", "B", "B"),
  years_education   = c(8, 12, 4, 6),
  completed_primary = c(1, 1, 0, 1)
)

ihs_aggregate(member_data, group_col = "case_id")
#> i Aggregating data by `case_id`...
#> # A tibble: 2 x 3
#>   case_id years_education completed_primary
#>   <chr>             <dbl>             <dbl>
#> 1 A                    20                 1
#> 2 B                    10                 1

Continuous columns are summed, 0/1 columns become a logical OR, and text columns collapse to their single distinct value. Select your columns before calling if summing is not what you want.