getting-started.RmdOpen-source SDTM tooling handles the mechanical domains well. What is
still hand-written on every study is the part that depends on the trial
design: subject elements, EPOCH, subject visits, and supplemental
qualifiers. sdtmgap does those four, from metadata.
Everything below runs on CDISCPILOT01 — the “LZZT”
xanomeline Alzheimer’s study, the CDISC SDTM/ADaM pilot submission
package, published by the CDISC Data Exchange Standards Team and coded
to SDTMIG 3.1.2
(def:StandardVersion="3.1.2" in the study’s own
define.xml). It is a real, public, 306-subject study: 3
arms plus screen failures, 7 trial elements, 21 planned visits, 3559
collected visit records.
The datasets ship with the package, read from the study’s SAS transport files. Nothing here needs a data package or a network.
sdtmgap_example()
#> [1] "edge_dm" "edge_elements" "edge_lb" "edge_ta"
#> [5] "edge_te" "edge_vs" "lzzt_dm" "lzzt_ex"
#> [9] "lzzt_se" "lzzt_sv" "lzzt_ta" "lzzt_te"
#> [13] "lzzt_ti" "lzzt_ts" "lzzt_tv"Its trial design is small enough to print in full. Note
TEDUR — the planned element durations — and note that TA
has no row for the follow-up element FOLO, which will
matter later.
sdtmgap_example("lzzt_te")[, c("ETCD", "ELEMENT", "TEDUR")]
#> # A tibble: 7 × 3
#> ETCD ELEMENT TEDUR
#> <chr> <chr> <chr>
#> 1 FOLO Follow_up NA
#> 2 HIE High_End P2W
#> 3 HIM High_Middle P22W
#> 4 HIS High_Start P2W
#> 5 LO Low P26W
#> 6 PBO Placebo P26W
#> 7 SCRN Screen NA
sdtmgap_example("lzzt_ta")[, c("ARMCD", "TAETORD", "ETCD", "EPOCH")]
#> # A tibble: 8 × 4
#> ARMCD TAETORD ETCD EPOCH
#> <chr> <dbl> <chr> <chr>
#> 1 Pbo 1 SCRN Screening
#> 2 Pbo 2 PBO Treatment
#> 3 Xan_Hi 1 SCRN Screening
#> 4 Xan_Hi 2 HIS Treatment
#> 5 Xan_Hi 3 HIM Treatment
#> 6 Xan_Hi 4 HIE Treatment
#> 7 Xan_Lo 1 SCRN Screening
#> 8 Xan_Lo 2 LO TreatmentThe obstacle to deriving SE from real data is that exposure does not carry the element code. CDISCPILOT01’s EX is no exception:
ex <- sdtmgap_example("lzzt_ex")
names(ex)
#> [1] "STUDYID" "DOMAIN" "USUBJID" "EXSEQ" "EXTRT" "EXDOSE"
#> [7] "EXDOSU" "EXROUTE" "VISITNUM" "VISIT" "EXSTDTC" "EXENDTC"derive_etcd() reads the answer off Trial Arms. The
subject’s arm selects that arm’s EPOCH == "TREATMENT"
elements in TAETORD order; if the arm has one such element
every dosing record belongs to it, and if it has several, the subject’s
records take them in start-date order. The high-dose arm is titrated
across three elements, so this is not a formality:
dm <- sdtmgap_example("lzzt_dm")
ta <- sdtmgap_example("lzzt_ta")
ex <- derive_etcd(ex, dm, ta)
table(ex$ETCD)
#>
#> HIE HIM HIS LO PBO
#> 28 72 72 193 226
subset(ex, USUBJID == "01-701-1028", c(USUBJID, VISIT, EXDOSE, EXSTDTC, ETCD))
#> # A tibble: 3 × 5
#> USUBJID VISIT EXDOSE EXSTDTC ETCD
#> <chr> <chr> <dbl> <chr> <chr>
#> 1 01-701-1028 BASELINE 54 2013-07-19 HIS
#> 2 01-701-1028 WEEK 2 81 2013-08-02 HIM
#> 3 01-701-1028 WEEK 24 54 2014-01-07 HIEWhere ordering does not determine the element,
derive_etcd() aborts and names the subjects rather than
guessing. derive_se() calls it for you when
exposure arrives without ETCD.
The element source is one frame of “subject was in element X from
this date”: screening from the first collected visit, treatment from the
dosing records, follow-up from the reference end date to the end of
participation. No end dates except the last — SDTMIG says elements are
contiguous, and derive_se() uses that to fill them.
sv <- sdtmgap_example("lzzt_sv")
first_visit <- stats::aggregate(SVSTDTC ~ USUBJID, data = sv, FUN = min)
elements <- rbind(
data.frame(USUBJID = first_visit$USUBJID, ETCD = "SCRN",
SESTDTC = first_visit$SVSTDTC, SEENDTC = NA_character_),
with(ex, data.frame(USUBJID = USUBJID, ETCD = ETCD,
SESTDTC = EXSTDTC, SEENDTC = NA_character_)),
with(dm[!is.na(dm$RFENDTC) & !is.na(dm$RFPENDTC) & dm$RFPENDTC > dm$RFENDTC, ],
data.frame(USUBJID = USUBJID, ETCD = "FOLO",
SESTDTC = RFENDTC, SEENDTC = RFPENDTC))
)
elements$DOMAIN <- "SE"
se <- derive_se(
dm = dm,
exposure = elements,
te = sdtmgap_example("lzzt_te"),
ta = ta
)
subset(se, USUBJID == "01-701-1028")
#>
#> ── SDTM SE (5 records) ─────────────────────────────────────────────────────────
#> ℹ 306 subject(s) with elements, 889 element record(s).
#> ℹ 52 element(s) took EPOCH from the element alone (arm not in TA).
#> ℹ 229 element(s) got no EPOCH from TA (arm/element not in TA).
#> # A tibble: 5 × 10
#> STUDYID DOMAIN USUBJID SESEQ ETCD ELEMENT SESTDTC SEENDTC TAETORD EPOCH
#> <chr> <chr> <chr> <int> <chr> <chr> <chr> <chr> <dbl> <chr>
#> 1 CDISCPILOT01 SE 01-701-… 1 SCRN Screen 2013-0… 2013-0… 1 Scre…
#> 2 CDISCPILOT01 SE 01-701-… 2 HIS High_S… 2013-0… 2013-0… 2 Trea…
#> 3 CDISCPILOT01 SE 01-701-… 3 HIM High_M… 2013-0… 2014-0… 3 Trea…
#> 4 CDISCPILOT01 SE 01-701-… 4 HIE High_E… 2014-0… 2014-0… 4 Trea…
#> 5 CDISCPILOT01 SE 01-701-… 5 FOLO Follow… 2014-0… 2014-0… NA NAThe notes on the print method are the derivation audit: 306 subjects, 889 elements, 52 screening elements that took their EPOCH from the element alone because CDISCPILOT01’s TA defines no screen-failure arm, and 229 elements with no EPOCH at all.
Those 229 are every FOLO record. TE defines a follow-up
element; TA never places it in an epoch. sdtmgap leaves
EPOCH missing rather than filling in the
FOLLOW-UP a human would obviously write, because the
study’s own metadata does not say so:
table(se$ETCD, se$EPOCH, useNA = "ifany")
#>
#> Screening Treatment <NA>
#> FOLO 0 0 229
#> HIE 0 28 0
#> HIM 0 72 0
#> HIS 0 72 0
#> LO 0 96 0
#> PBO 0 86 0
#> SCRN 306 0 0derive_epoch() is an interval join per subject: it finds
the element whose SESTDTC–SEENDTC window
contains the record’s date and copies its EPOCH. Both ends
are inclusive; because contiguous elements share a boundary date, a
record landing exactly on one is assigned to the later
element.
sv_epoch <- derive_epoch(sv, se, dtc_var = "SVSTDTC")
table(sv_epoch$EPOCH, useNA = "ifany")
#>
#> Screening Treatment <NA>
#> 622 2826 111CDISCPILOT01 ships its own SE, built independently by the study team. Joining that to the same TA gives a reference EPOCH for every visit, so the only thing that differs is the derivation.
ref <- derive_epoch(
sv,
merge(sdtmgap_example("lzzt_se"), unique(ta[, c("ETCD", "EPOCH")]),
by = "ETCD", all.x = TRUE),
dtc_var = "SVSTDTC"
)$EPOCH
ours <- sv_epoch$EPOCH
table(reference = ref, sdtmgap = ours, useNA = "ifany")
#> sdtmgap
#> reference Screening Treatment <NA>
#> Screening 610 0 0
#> Treatment 0 2826 19
#> <NA> 12 0 923436 of the 3436 records that both sides place agree — 100%. Over all 3559 SV records, counting “both leave it missing” as agreement, 3528 agree (99.1%). No record is put in a different epoch. The two off-diagonal cells are disagreements about whether to assign, not about which:
Treatment
and sdtmgap leaves missing. The study’s treatment element
runs to the end of participation for subjects who discontinued early;
ours ends at the last dose and hands the rest to FOLO,
which TA gives no EPOCH.sdtmgap calls Screening — unscheduled visits
before the study’s screening element starts. Ours starts at the
subject’s first collected visit, so they are covered.At the finer element level agreement is lower — 3270 of 3544 (92.3%) — and that is the more honest number to quote for SE itself. Two systematic causes, both explainable:
sdtmgap starts it on the day the source
record says. Every high-dose subject therefore has one visit on the
switch day that the study calls HIM and we call
HIS.ARMCD); sdtmgap prefers the
actual arm (ACTARMCD). Twelve subjects
were randomised to Xan_Hi and actually treated as Xan_Lo, so their
elements differ by construction.SV is built from the collected data, not from any one domain: the visit window is the range across everything supplied. EX is the only visit-bearing collected domain the package ships for CDISCPILOT01, so compare what it gives against the study’s own SV.
sv_derived <- derive_sv(domains = list(ex = ex), dm = dm)
subset(sv_derived, USUBJID == "01-701-1028")
#>
#> ── SDTM SV (3 records) ─────────────────────────────────────────────────────────
#> ℹ 591 visit record(s) for 254 subject(s), from 591 input record(s) across 1 domain(s).
#> # A tibble: 3 × 7
#> STUDYID DOMAIN USUBJID VISITNUM VISIT SVSTDTC SVENDTC
#> <chr> <chr> <chr> <dbl> <chr> <chr> <chr>
#> 1 CDISCPILOT01 SV 01-701-1028 3 BASELINE 2013-07-19 2013-07-19
#> 2 CDISCPILOT01 SV 01-701-1028 4 WEEK 2 2013-08-02 2013-08-02
#> 3 CDISCPILOT01 SV 01-701-1028 12 WEEK 24 2014-01-07 2014-01-07
subset(sv, USUBJID == "01-701-1028" & VISITNUM %in% sv_derived$VISITNUM,
c(USUBJID, VISITNUM, SVSTDTC, SVENDTC))
#> # A tibble: 3 × 4
#> USUBJID VISITNUM SVSTDTC SVENDTC
#> <chr> <dbl> <chr> <chr>
#> 1 01-701-1028 3 2013-07-19 2013-07-19
#> 2 01-701-1028 4 2013-08-01 2013-08-01
#> 3 01-701-1028 12 2014-01-06 2014-01-06Feed it LB, VS and QS as well and the windows widen to whichever domain observed the subject earliest and latest that day — which is the point of building SV from collected data rather than from one domain.
make_supp(dm, c("ARMCD", "ACTARMCD"))
#>
#> ── SDTM SUPPDM (612 records) ───────────────────────────────────────────────────
#> ℹ 612 qualifier record(s) from 2 variable(s) on 306 subject(s).
#> ℹ 0 record(s) dropped for missing QVAL.
#> # A tibble: 612 × 10
#> STUDYID RDOMAIN USUBJID IDVAR IDVARVAL QNAM QLABEL QORIG QEVAL QVAL
#> * <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
#> 1 CDISCPILOT01 DM 01-701-10… "" "" ACTA… ACTAR… CRF "" Pbo
#> 2 CDISCPILOT01 DM 01-701-10… "" "" ARMCD ARMCD CRF "" Pbo
#> 3 CDISCPILOT01 DM 01-701-10… "" "" ACTA… ACTAR… CRF "" Pbo
#> 4 CDISCPILOT01 DM 01-701-10… "" "" ARMCD ARMCD CRF "" Pbo
#> 5 CDISCPILOT01 DM 01-701-10… "" "" ACTA… ACTAR… CRF "" Xan_…
#> 6 CDISCPILOT01 DM 01-701-10… "" "" ARMCD ARMCD CRF "" Xan_…
#> 7 CDISCPILOT01 DM 01-701-10… "" "" ACTA… ACTAR… CRF "" Xan_…
#> 8 CDISCPILOT01 DM 01-701-10… "" "" ARMCD ARMCD CRF "" Xan_…
#> 9 CDISCPILOT01 DM 01-701-10… "" "" ACTA… ACTAR… CRF "" Xan_…
#> 10 CDISCPILOT01 DM 01-701-10… "" "" ARMCD ARMCD CRF "" Xan_…
#> # ℹ 602 more rowsIDVAR is blank because DM has no --SEQ,
which is what SUPPDM requires. On a domain that has one it defaults to
the parent’s --SEQ:
make_supp(sdtmgap_example("edge_vs"),
data.frame(QNAM = "VSCLSIG",
QLABEL = "Clinically Significant",
QORIG = "CRF",
QEVAL = "INVESTIGATOR"))
#>
#> ── SDTM SUPPVS (3 records) ─────────────────────────────────────────────────────
#> ℹ 3 qualifier record(s) from 1 variable(s) on 3 subject(s).
#> ℹ 29 record(s) dropped for missing QVAL.
#> # A tibble: 3 × 10
#> STUDYID RDOMAIN USUBJID IDVAR IDVARVAL QNAM QLABEL QORIG QEVAL QVAL
#> * <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
#> 1 SDTMGAP01 VS SDTMGAP01-001 VSSEQ 4 VSCLS… Clini… CRF INVE… Y
#> 2 SDTMGAP01 VS SDTMGAP01-003 VSSEQ 1 VSCLS… Clini… CRF INVE… Y
#> 3 SDTMGAP01 VS SDTMGAP01-005 VSSEQ 4 VSCLS… Clini… CRF INVE… YMissing QVAL rows are dropped rather than emitted blank,
as the standard requires. Drop the moved columns from the parent
yourself:
Every function above goes through one internal helper that widens a
truncated ISO 8601 value to the interval of instants it could denote,
rather than erroring or coercing to NA:
| Collected | Earliest | Latest |
|---|---|---|
2026 |
2026-01-01T00:00:00 |
2026-12-31T23:59:59 |
2026-02 |
2026-02-01T00:00:00 |
2026-02-28T23:59:59 |
2024-02 |
2024-02-01T00:00:00 |
2024-02-29T23:59:59 |
2026-03-15 |
2026-03-15T00:00:00 |
2026-03-15T23:59:59 |
Records are located at their earliest possible instant; element
boundaries are widened outwards, so a partial date is never excluded
from an element it might belong to. The collected value is written
through to the output unchanged — sdtmgap never invents a
precision the CRF did not have.
TEDUR durations, the last resort for an element with no
other end date, take the composite ISO 8601 forms. Years and months are
calendar arithmetic clamped to the end of the target
month, then weeks and days at 7 and 1 day, then one day subtracted
because the start date is day 1 of the element:
SESTDTC |
TEDUR |
End | Why |
|---|---|---|---|
2026-01-15 |
P1M |
2026-02-14 |
month step lands on the 15th |
2026-01-31 |
P1M |
2026-02-27 |
clamped to 2026-02-28, not 2026-03-03
|
2024-01-31 |
P1M |
2024-02-28 |
leap year: clamps to the 29th |
2026-01-20 |
P1M15D |
2026-03-06 |
month first (02-20), then 15 days |
2026-02-18 |
P1DT12H |
NA |
a date cannot carry the 12 hours |
The edge_* datasets are a fictional 7-subject study that
exists only to hold the cases CDISCPILOT01 does not have, so they can be
tested at all: an element with no recorded end date, a subject with no
follow-up element, a screen failure with no elements, partial
"2026-01" dates, a visit landing exactly on a shared
element boundary, and a record dated before the study began.
edge_se <- derive_se(sdtmgap_example("edge_dm"), sdtmgap_example("edge_elements"),
sdtmgap_example("edge_te"), sdtmgap_example("edge_ta"))
derive_epoch(sdtmgap_example("edge_vs"), edge_se)[, c("USUBJID", "VISIT", "VSDTC", "EPOCH")]
#>
#> ── SDTM VS (32 records) ────────────────────────────────────────────────────────
#> ℹ EPOCH assigned from VSDTC for 29 of 32 record(s).
#> ℹ 3 record(s) fell in no element interval; EPOCH left missing.
#> ℹ 1 record(s) had a partial date, placed at its earliest instant.
#> # A tibble: 32 × 4
#> USUBJID VISIT VSDTC EPOCH
#> <chr> <chr> <chr> <chr>
#> 1 SDTMGAP01-001 UNSCHEDULED 2025-12-20 NA
#> 2 SDTMGAP01-001 SCREENING 2026-01-05 SCREENING
#> 3 SDTMGAP01-001 DAY 1 2026-01-19 TREATMENT
#> 4 SDTMGAP01-001 WEEK 2 2026-02-02 TREATMENT
#> 5 SDTMGAP01-001 WEEK 4 2026-02-15 TREATMENT
#> 6 SDTMGAP01-001 FOLLOW-UP 2026-03-01 FOLLOW-UP
#> 7 SDTMGAP01-002 SCREENING 2026-01-07 SCREENING
#> 8 SDTMGAP01-002 DAY 1 2026-01-21 TREATMENT
#> 9 SDTMGAP01-002 WEEK 2 2026-02-04 TREATMENT
#> 10 SDTMGAP01-002 WEEK 4 2026-02-17 TREATMENT
#> # ℹ 22 more rowsNothing is dropped. The unscheduled visit dated before the study, the
record with no date, and the screen failure’s visit all come back with
EPOCH missing, which is the honest answer and the one you
can query for.
Every function returns an sdtmgap_domain, a tibble
subclass. The print method adds the derivation notes; the
object itself is a plain data frame, so it goes straight into
xportr, admiral, datasetjson or
anything else without conversion.
is.data.frame(se)
#> [1] TRUE
class(as.data.frame(se))
#> [1] "data.frame"