Changelog

submitpack 0.0.1

The release that states its own boundary. v0.0.0.9000 declared no scope or limits anywhere — README, vignette, NEWS or roxygen — and the DESCRIPTION claimed the assembled folder contained “Define-XML”, which the package has never generated.

Scope, stated

  • README, vignette, package-level docs and every exported function now carry an explicit Scope and Limits. In short: submitpack writes a folder of SAS v5 transport files, splits oversized ones, gates the write on nineteen transport-layer checks, and manifests what landed. It is not a conformance checker — no controlled terminology, no referential integrity, no required-variable rules, no ISO 8601 checks, no Pinnacle 21 rule set.
  • DESCRIPTION corrected. It no longer claims to write Define-XML. write_submission(define_xml = ) is a caller-supplied hook: a file path to copy, or a function(spec, path) to call after the transport files are written. That contract is unchanged from v0.0.0.9000 and is deliberate — submitpack does not depend on any Define-XML generator.
  • define21 added to Suggests with one integration test proving the hook accepts define21::write_define(). The test skips if define21 is absent and converts any error inside define21 into a skip, so submitpack’s suite cannot go red because of it. Nothing in Imports changed.

New checks

Nine check identifiers added to the ten that existed:

  • dataset_name_chars, variable_name_chars (error) — names outside ^[A-Za-z_][A-Za-z0-9_]*$: a leading digit, a space, a hyphen, or a non-ASCII letter.
  • control_character (error) — a C0 control byte (0x01-0x1F) or DEL in a label or a value, which corrupts the fixed-width record layout.
  • split_variable_set, split_variable_order, split_length_consistency, split_label_consistency, split_part_name_length (error) and split_part_naming (warning) — consistency across the parts of a split dataset. check_submission() now accepts a dataset as a list of data frames or as paths to already-written .xpt files, and compares the parts.
  • file_size (warning) — the estimated written size of a part against max_file_mb, a new argument on check_submission() and write_submission(), defaulting to 5000 MB (the 5 GB per-file ceiling FDA and PMDA both publish). submitpack does not track guidance revisions; set the number yourself, or NULL to skip.

length_too_short now counts bytes, not characters, and says so in the message: a 4-character UTF-8 value needing 5 bytes overflows a length-4 variable and is caught.

Internals

  • Added an XPT v5 NAMESTR reader. haven::read_xpt() trims the space padding, so it cannot tell you what length a file declared; the split-consistency checks read the declared lengths and labels out of the transport file’s own variable descriptors.
  • Added an exact XPT v5 size calculator, asserted byte-for-byte against real CDISCPILOT01 transport files.

Fixtures and tests

  • inst/extdata now ships real CDISCPILOT01 (LZZT) material: dataset and variable metadata extracted from the study’s own SDTM and ADaM define.xml (32 datasets, 728 variables), plus the real DM (306 x 25) and ADSL (254 x 48) data. Writing DM from that metadata reproduces the study’s own dm.xpt at exactly 110,800 bytes.
  • Tests now assert on the recomputed content of the assembled folder — file names, the declared lengths, labels and order read back out of the transport file, the manifest’s checksums and record counts — rather than on the folder existing.

Known limits carried forward

  • file_size is arithmetic on the spec, not a measurement; a spec that does not match the data gives a size that does not match the file.
  • Split parts written by write_submission(split_mb = ) are consistent by construction, so the split checks exist for parts written elsewhere.
  • submission_manifest() lists one directory level and checksums with MD5. Checksums are not reproducible across runs: a v5 transport file stamps its own creation timestamp into the header.
  • Only "character" and "numeric" variable types; no date semantics.
  • Labels read back from a transport file are the 40 bytes a v5 NAMESTR holds, so split_label_consistency compares the first 40 bytes only.

submitpack 0.0.0.9000

Initial development version.

  • Spec engine: submission_spec(), new_submission_spec(), validate_spec() (aggregating validation) with an S3 submission_spec class. The variables table uses {xportr}’s own column names so it can be passed straight through.
  • check_submission(): pre-write agency-constraint gate covering dataset/ variable name length, dataset/variable label length, non-ASCII characters in data or metadata, a declared length shorter than the longest observed value, and duplicate names (in the spec and in the real data frame). Returns a tibble of violations with severity and an actionable message.
  • write_submission(): runs the {xportr} pipeline (type, length, label, format, order, then write) per dataset, lays out the submission folder, splits oversized transport files into numbered parts via split_mb, aborts before writing on any error-severity violation, and supports an optional Define-XML hook (file path to copy, or a function(spec, path)).
  • submission_manifest(): inventories a submission folder from disk (file name, size, MD5 checksum, record count, variable count) — independent of what write_submission() thinks it wrote.