Package: crawlee
Title: Tidy Interface for Reproducible Web Crawling
Version: 0.1.1
Authors@R: c(
    person("Andre", "Leite", email = "leite@castlab.org", role = c("aut", "cre")),
    person("Marcos", "Wasiliew", email = "marcos.wasiliew@gmail.com", role = "aut",
           comment = c(ORCID = "0009-0004-4694-3159")),
    person("Hugo", "Vasconcelos", email = "hugo.vasconcelos@ufpe.br", role = "aut"),
    person("Carlos", "Amorim", email = "carlos.agaf@ufpe.br", role = "aut"),
    person("Diogo", "Bezerra", email = "diogo.bezerra@ufpe.br", role = "aut"),
    person("Júlia", "Nascimento Barreto", email = "juliabarreto@gd.seplag.pe.gov.br", role = "aut",
           comment = c(ORCID = "0009-0004-2851-7770")))
Description: A tidy, pipe-friendly toolkit for reproducible web crawling
    and structured data collection, inspired by the architecture of the
    'Crawlee' library. Provides a unified crawler with a deduplicating,
    resumable request queue, content-type aware handlers, structured
    storage backends and rich console logging via 'cli'. Supports crawling
    HTML pages, sitemaps, RSS and Atom feeds and PDF documents, with optional
    headless-browser rendering and helpers for retrieval-augmented generation.
License: MIT + file LICENSE
URL: https://github.com/StrategicProjects/crawlee,
        https://strategicprojects.github.io/crawlee/
BugReports: https://github.com/StrategicProjects/crawlee/issues
Depends: R (>= 4.1.0)
Imports: cli, httr2, R6, rlang, rvest, tibble, vctrs, xml2
Suggests: arrow, chromote, DBI, dplyr, duckdb, httptest2, jsonlite,
        knitr, later, nanoparquet, pdftools, promises, rmarkdown,
        testthat (>= 3.0.0)
VignetteBuilder: knitr
Config/testthat/edition: 3
Encoding: UTF-8
Language: en-US
RoxygenNote: 8.0.0
NeedsCompilation: no
Packaged: 2026-09-29 12:36:26 UTC; leite
Author: Andre Leite [aut, cre],
  Marcos Wasiliew [aut] (ORCID: <https://orcid.org/0009-0004-4694-3159>),
  Hugo Vasconcelos [aut],
  Carlos Amorim [aut],
  Diogo Bezerra [aut],
  Júlia Nascimento Barreto [aut] (ORCID:
    <https://orcid.org/0009-0004-2851-7770>)
Maintainer: Andre Leite <leite@castlab.org>
Repository: CRAN
Date/Publication: 2026-09-30 21:20:55 UTC
