getting-started.Rmdxportr writes an excellent transport file. A submission
is a folder of them, correctly named and split, with a manifest
proving what shipped. submitpack makes that one call.
submitpack writes a folder of SAS v5 transport files, splits anything over the per-file ceiling, gates the write on nineteen transport-layer checks, and manifests what landed on disk. Three things it deliberately does not do:
define_xml is a hook — a file path to copy into the folder,
or a function(spec, path) to call. define21,
or any other generator, plugs in there. submitpack neither imports nor
requires one.--SEQ uniqueness,
value-level metadata and the rest of the Pinnacle 21 rule set are
not checked. Zero violations means clean against the
list in section 2, and nothing more.submission_spec() takes a datasets table
(name, label, class) and a variables table (name, label,
type, length, format, order per dataset). The variables
columns are exactly the columns xportr expects, so this
table is handed straight to the xportr pipeline with no
reshaping.
library(submitpack)
datasets <- data.frame(
dataset = "DM", label = "Demographics", class = "SDTM"
)
variables <- data.frame(
dataset = "DM",
variable = c("USUBJID", "AGE", "SEX"),
label = c("Unique Subject Identifier", "Age", "Sex"),
type = c("character", "numeric", "character"),
length = c(20, 8, 1),
format = NA_character_,
order = 1:3
)
spec <- submission_spec(datasets, variables, study = list(study_id = "CDISCPILOT01"))
spec
#>
#> ── submitpack specification ────────────────────────────────────────────────────
#> ℹ Study: CDISCPILOT01 | datasets: 1 | variables: 3
#> ℹ [DM] Demographics (SDTM: 3 variable(s))Every problem in the spec is reported at once:
bad_variables <- variables
bad_variables$type[2] <- "logical"
submission_spec(datasets, bad_variables)
#> Error in `validate_spec()`:
#> ! Submission specification failed validation:
#> ✖ Variable(s) with `type` not in {character, numeric}: AGEcheck_submission() runs the checks that would otherwise
get a submission rejected. The complete list, and nothing beyond it:
| Check | Severity |
|---|---|
dataset_name_length,
variable_name_length
|
error |
dataset_name_chars,
variable_name_chars
|
error |
dataset_label_length,
variable_label_length
|
error |
duplicate_dataset_name,
duplicate_variable_name
|
error |
length_too_short (counted in
bytes) |
error |
control_character |
error |
non_ascii_metadata, non_ascii_data
|
warning |
split_variable_set,
split_variable_order
|
error |
split_length_consistency,
split_label_consistency
|
error |
split_part_name_length |
error |
split_part_naming |
warning |
file_size (against max_file_mb, default
5000) |
warning |
?check_submission explains each one and spells out what
is not checked.
dm <- data.frame(
USUBJID = c("CDISC-001", "CDISC-002"),
AGE = c(63, 35),
SEX = c("M", "F")
)
check_submission(list(DM = dm), spec)
#> # A tibble: 0 × 5
#> # ℹ 5 variables: dataset <chr>, variable <chr>, check <chr>, severity <chr>,
#> # message <chr>An empty tibble means clean — this is what
write_submission() checks internally before it writes
anything.
out <- file.path(tempdir(), "submission")
man <- write_submission(list(DM = dm), spec, path = out)
#>
#> ── All variables in dataset are found in `metadata` ──
#>
#> ── All variables in dataset are ordered ──
#>
#> ✔ Wrote DM (2 record(s), 3 variable(s))
#> ✔ Submission written to /tmp/RtmpvxLVHt/submission (1 file(s))
man
#>
#> ── submitpack manifest ─────────────────────────────────────────────────────────
#> ℹ 1 file(s) in /tmp/RtmpvxLVHt/submission | 1,280 bytes total
#> # A tibble: 1 × 5
#> file size_bytes checksum records variables
#> <chr> <dbl> <chr> <int> <int>
#> 1 dm.xpt 1280 668ccac6f1f0dfd337799c1b6db663d9 2 3Each dataset runs the xportr pipeline in order — type,
length, label, format, order, then write — and lands as a lowercase
<dataset>.xpt. Pass split_mb to split
any file over that size into dm1.xpt, dm2.xpt,
…
write_submission(list(DM = dm), spec, path = out, split_mb = 5000)Parts written that way come from one data frame, so they agree with
each other by construction. Parts you did not write here —
legacy files, another vendor’s, a previous run — are exactly what the
split checks are for. Hand check_submission() the paths and
it reads the declared lengths and labels straight out of each file’s
variable descriptors:
one <- file.path(tempdir(), "part-demo")
write_submission(list(DM = dm[1, ]), spec, path = one)
#>
#> ── All variables in dataset are found in `metadata` ──
#>
#> ── All variables in dataset are ordered ──
#>
#> ✔ Wrote DM (1 record(s), 3 variable(s))
#> ✔ Submission written to /tmp/RtmpvxLVHt/part-demo (1 file(s))
wide <- spec
wide$variables$length[wide$variables$variable == "USUBJID"] <- 30
two <- file.path(tempdir(), "part-demo-2")
write_submission(list(DM = dm[2, ]), wide, path = two)
#>
#> ── All variables in dataset are found in `metadata` ──
#>
#> ── All variables in dataset are ordered ──
#>
#> ✔ Wrote DM (1 record(s), 3 variable(s))
#> ✔ Submission written to /tmp/RtmpvxLVHt/part-demo-2 (1 file(s))
parts <- file.path(tempdir(), c("dm1.xpt", "dm2.xpt"))
file.copy(file.path(one, "dm.xpt"), parts[1], overwrite = TRUE)
#> [1] TRUE
file.copy(file.path(two, "dm.xpt"), parts[2], overwrite = TRUE)
#> [1] TRUE
check_submission(list(DM = parts), spec)[, c("variable", "check", "message")]
#> # A tibble: 1 × 3
#> variable check message
#> <chr> <chr> <chr>
#> 1 USUBJID split_length_consistency Variable 'USUBJID' is length 20 in 'dm1' bu…submitpack does not build Define-XML. define_xml is a
plain argument, not a dependency: pass a path to a file you already have
and it is copied alongside the transport files, or pass a
function(spec, path) and it is called once the datasets are
written.
generator <- function(spec, path) {
writeLines("<ODM><!-- your generator's output --></ODM>",
file.path(path, "define.xml"))
}
write_submission(list(DM = dm), spec, path = out, define_xml = generator)
#>
#> ── All variables in dataset are found in `metadata` ──
#>
#> ── All variables in dataset are ordered ──
#>
#> ✔ Wrote DM (2 record(s), 3 variable(s))
#> ✔ Submission written to /tmp/RtmpvxLVHt/submission (2 file(s))define21::write_define() fits that signature; so does a
shell-out to any other tool. Nothing about the hook is validated beyond
it being a readable path or a function — if your generator writes
nothing, submitpack will not notice.
submission_manifest() reads the folder back off disk —
not memory — for file name, size, checksum, and (for .xpt
files) the record and variable counts read straight out of the transport
file.
submission_manifest(out)
#>
#> ── submitpack manifest ─────────────────────────────────────────────────────────
#> ℹ 2 file(s) in /tmp/RtmpvxLVHt/submission | 1,324 bytes total
#> # A tibble: 2 × 5
#> file size_bytes checksum records variables
#> <chr> <dbl> <chr> <int> <int>
#> 1 define.xml 44 678c24c180f4df4bcbb1577347e14de5 NA NA
#> 2 dm.xpt 1280 668ccac6f1f0dfd337799c1b6db663d9 2 3The listing is one level deep and the checksum is MD5 — an integrity check, not a tamper-proof signature. It is not reproducible either: a v5 transport file stamps its own creation date and time into the header, so writing the same data twice gives the same size and a different checksum.
The package ships the real CDISCPILOT01 (LZZT) metadata and data in
inst/extdata: 32 datasets and 728 variables extracted from
the study’s own SDTM and ADaM define.xml, plus the real
DM and ADSL.
ds <- read.csv(system.file("extdata", "lzzt_datasets.csv", package = "submitpack"))
vr <- read.csv(system.file("extdata", "lzzt_variables.csv.gz", package = "submitpack"))
vr$format <- NA_character_
real <- submission_spec(
ds[, c("dataset", "label", "class")],
vr[, c("dataset", "variable", "label", "type", "length", "format", "order")],
study = list(study_id = "CDISCPILOT01")
)
nrow(check_submission(list(), real)) # the real metadata is clean
#> [1] 0