getting-started.RmdA validation report tells you a rule failed. It does not tell you which line of your mapping code or which row of your specification caused it. This vignette walks the four steps that close that loop, on the example study that ships with the package.
ex <- readRDS(system.file("extdata", "example_study.rds", package = "p21bridge"))
names(ex$datasets)
#> [1] "DM" "AE"
head(ex$spec, 3)
#> dataset variable label type length codelist core
#> 1 DM STUDYID Study Identifier text 8 <NA> Req
#> 2 DM USUBJID Unique Subject Identifier text 12 <NA> Req
#> 3 DM SUBJID Subject Identifier text 4 <NA> ReqReports arrive as .xlsx from Pinnacle 21 and as
.csv from everyone who exported one. Column naming drifts
between versions - Rule ID, RULE_ID,
Check ID. read_findings() normalises all of
it, so nothing downstream has to care. It replaces the
PROC IMPORT plus per-vintage RENAME block.
findings <- read_findings(system.file("extdata", "findings.csv", package = "p21bridge"))
findings
#>
#> ── p21bridge findings ──────────────────────────────────────────────────────────
#> ℹ 12 findings from findings.csv
#> ℹ Severity: Error 7 | Warning 3 | Notice 2
#> ℹ Rule class: SDTM data 8 | Controlled terminology 2 | Define-XML 1 | ODM / define metadata 1
#> # A tibble: 12 × 8
#> rule_id severity domain variable message count rule_class category
#> <chr> <chr> <chr> <chr> <chr> <int> <chr> <chr>
#> 1 SD0056 Error DM RFENDTC SDTM Required var… 1 SDTM data Metadata
#> 2 SD1310 Warning DM USUBJID Variable length i… 5 SDTM data Length
#> 3 CT2001 Error DM SEX SEX value not fou… 1 Controlle… Termino…
#> 4 CT2001 Error AE AESEV AESEV value not f… 2 Controlle… Termino…
#> 5 SD1011 Error DM RFSTDTC Invalid ISO 8601 … 1 SDTM data Format
#> 6 SD1117 Warning AE USUBJID Duplicate records 2 SDTM data Consist…
#> 7 SD0064 Error AE USUBJID Subject is not pr… 1 SDTM data Cross-r…
#> 8 SD0002 Error AE AESTDTC NULL value in var… 1 SDTM data Presence
#> 9 SD1234 Notice VS VSORRES Result value is n… 3 SDTM data Consist…
#> 10 DD0011 Warning DM AGE Variable label mi… 1 Define-XML Metadata
#> 11 SD1310 Error ADSL TRT01P Variable length i… 4 SDTM data Length
#> 12 OD0018 Notice NA NA Define-XML is mis… 1 ODM / def… MetadataThe same report as .xlsx, with different column names
and the findings on a sheet called Details, normalises to
the same table:
xl <- read_findings(system.file("extdata", "findings.xlsx", package = "p21bridge"))
identical(xl$rule_id, findings$rule_id)
#> [1] TRUErule_class comes from the rule-id namespace, so you can
triage SDTM data issues separately from define.xml metadata issues:
rule_class(c("SD0002", "AD0018", "CT2002", "DD0011", "OD0018", "XX9999"))
#> [1] "SDTM data" "ADaM data" "Controlled terminology"
#> [4] "Define-XML" "ODM / define metadata" "Other"trace_findings() needs the datasets the report was run
against, and optionally the specification. Matching is case-insensitive
on dataset and variable name.
traced <- trace_findings(findings, ex$datasets, spec = ex$spec)
traced[, c("rule_id", "domain", "variable", "resolution", "variable_max_nchar", "spec_length")]
#> # A tibble: 12 × 6
#> rule_id domain variable resolution variable_max_nchar spec_length
#> <chr> <chr> <chr> <chr> <int> <int>
#> 1 SD0056 DM RFENDTC dataset+spec NA 19
#> 2 SD1310 DM USUBJID dataset+variable+spec 20 12
#> 3 CT2001 DM SEX dataset+variable+spec 6 1
#> 4 CT2001 AE AESEV dataset+variable+spec 8 10
#> 5 SD1011 DM RFSTDTC dataset+variable+spec 10 19
#> 6 SD1117 AE USUBJID dataset+variable+spec 20 20
#> 7 SD0064 AE USUBJID dataset+variable+spec 20 20
#> 8 SD0002 AE AESTDTC dataset+variable+spec 10 19
#> 9 SD1234 VS VSORRES unresolved: dataset '… NA NA
#> 10 DD0011 DM AGE dataset+variable+spec 2 3
#> 11 SD1310 ADSL TRT01P unresolved: dataset '… NA NA
#> 12 OD0018 NA NA unresolved: finding r… NA NAresolution records how far the trace got. The
DM.USUBJID finding reached
dataset+variable+spec: the longest observed
USUBJID is 20 characters while the spec declares 12 - the
finding, explained, without opening the data.
Findings that cannot be traced are kept and flagged. They are never silently dropped, because a finding you cannot explain is exactly the one that bites at submission:
traced[!traced$resolved, c("rule_id", "domain", "resolution")]
#> # A tibble: 3 × 3
#> rule_id domain resolution
#> <chr> <chr> <chr>
#> 1 SD1234 VS unresolved: dataset 'VS' not supplied
#> 2 SD1310 ADSL unresolved: dataset 'ADSL' not supplied
#> 3 OD0018 NA unresolved: finding reports no domainThe package ships the Pinnacle 21 rule catalogue - 537 rules keyed by Pinnacle 21 id, with the Publisher (FDA) id, the canonical text, the category and the severity - so you never have to go and look one up:
rule_info(c("SD0064", "CT2001", "SD1310"))[, c("rule_id", "in_catalogue", "severity", "message")]
#> # A tibble: 3 × 4
#> rule_id in_catalogue severity message
#> <chr> <lgl> <chr> <chr>
#> 1 SD0064 TRUE Error Subject is not present in DM domain
#> 2 CT2001 TRUE Error Variable value not found in non-extensible code…
#> 3 SD1310 FALSE NA NA
finding_summary(traced)
#>
#> ── p21bridge finding summary ───────────────────────────────────────────────────
#> ℹ 12 findings covering 23 reported records
#> ! 3 findings could not be traced to a supplied dataset
#>
#> ── By severity ──
#>
#> # A tibble: 3 × 4
#> level findings records errors
#> <chr> <int> <int> <int>
#> 1 Error 7 11 7
#> 2 Warning 3 8 0
#> 3 Notice 2 4 0
#> ── By domain ──
#> # A tibble: 5 × 4
#> level findings records errors
#> <chr> <int> <int> <int>
#> 1 DM 5 9 3
#> 2 AE 4 6 3
#> 3 <missing> 1 1 0
#> 4 ADSL 1 4 1
#> 5 VS 1 3 0
#> ── By rule_class ──
#> # A tibble: 4 × 4
#> level findings records errors
#> <chr> <int> <int> <int>
#> 1 SDTM data 8 18 5
#> 2 Controlled terminology 2 3 2
#> 3 Define-XML 1 1 0
#> 4 ODM / define metadata 1 1 0suggest_fix() recomputes each problem from the data and
proposes a remedy. Seven finding classes are recognised: missing
required variable, null value in a required variable, variable length,
controlled terminology, ISO 8601 dates, key uniqueness and referential
integrity.
Each finding is classified by looking its rule id up in the catalogue
first, and only then by matching the report’s message text -
class_source records which path fired, so a classification
is always attributable:
fixes <- suggest_fix(traced)
fixes[, c("rule_id", "fix_class", "class_source", "deterministic")]
#> # A tibble: 12 × 4
#> rule_id fix_class class_source deterministic
#> <chr> <chr> <chr> <lgl>
#> 1 SD0056 missing_variable catalogue TRUE
#> 2 SD1310 variable_length message_regex TRUE
#> 3 CT2001 ct_codelist catalogue FALSE
#> 4 CT2001 ct_codelist catalogue TRUE
#> 5 SD1011 iso8601_date catalogue TRUE
#> 6 SD1117 key_not_unique catalogue TRUE
#> 7 SD0064 referential_integrity catalogue TRUE
#> 8 SD0002 required_value_null catalogue FALSE
#> 9 SD1234 human_judgement message_regex FALSE
#> 10 DD0011 human_judgement message_regex FALSE
#> 11 SD1310 variable_length message_regex FALSE
#> 12 OD0018 human_judgement message_regex FALSESD1310 is not a rule the shipped catalogue has (it
stands in for a rule newer than the catalogue): it is classified from
its message text, exactly as p21bridge 0.0.0.9000 classified
everything.
The suggestion is specific and carries the evidence it was derived from:
iso <- fixes[fixes$rule_id == "SD1011", ]
iso$suggestion
#> [1] "Convert the 1 non-ISO value(s) of DM.RFSTDTC (format '%d%b%Y') to ISO 8601, leaving conforming values untouched."
iso$evidence
#> [1] "1 of 5 populated value(s) are not ISO 8601: '12MAR2021'. Parsed with format '%d%b%Y' -> 2021-03-12."
iso$r_expression
#> [1] "DM <- dplyr::mutate(DM, RFSTDTC = ifelse(grepl(\"^[0-9]{4}(-[0-9]{2}(-[0-9]{2}(T[0-9]{2}(:[0-9]{2}(:[0-9]{2})?)?)?)?)?$\", RFSTDTC), RFSTDTC, format(as.Date(RFSTDTC, format = \"%d%b%Y\"), \"%Y-%m-%d\")))"The expression is real code. Run it and the finding clears - the conforming values are left untouched:
A finding whose remedy depends on study knowledge gets a label, not a guess:
hj <- fixes[fixes$fix_class == "human_judgement", c("rule_id", "suggestion")]
hj$suggestion
#> [1] "No deterministic remedy for this finding class - programmer review required. The message needs a study-specific decision before it can be actioned."
#> [2] "No deterministic remedy for this finding class - programmer review required. Reconcile the define.xml metadata against the dataset by hand."
#> [3] "No deterministic remedy for this finding class - programmer review required. Reconcile the define/ODM metadata against the dataset by hand."The same applies inside a deterministic class. DM.SEX
holds "Female", which is not in the F;M;U
codelist the specification declares and does not become one by
upper-casing - so the value and the codelist are reported, and no
expression is emitted:
sex <- fixes[which(fixes$domain == "DM" & fixes$variable == "SEX"), ]
sex$suggestion
#> [1] "Recode DM.SEX: 'Female' not in the spec codelist ('F', 'M', 'U'). The target term is a study decision, so no expression is emitted."
sex$r_expression
#> [1] NAWhereas AE.AESEV differs from its codelist only by case
and padding, which is mechanical:
aesev <- fixes[which(fixes$domain == "AE" & fixes$variable == "AESEV"), ]
aesev$r_expression
#> [1] "AE <- dplyr::mutate(AE, AESEV = toupper(trimws(AESEV)))"
aesev$evidence
#> [1] "Non-conforming value(s): 'mild', 'SEVERE '. Checked against CDISC CT 2026-03-27 codelist 'Severity/Intensity Scale for Adverse Events' (AESEV, non-extensible): 'MILD', 'MODERATE', 'SEVERE'."Note where that codelist came from: the example specification leaves
AESEV without one, so the check fell back to the CDISC
controlled terminology shipped with the package. You can use that
dictionary directly:
ct_release()
#> [1] "2026-03-27"
ct_check(c("MILD", "mild", "Grade 2"), "AESEV")
#> # A tibble: 3 × 6
#> value conforms codelist_name codelist_value codelist_code extensible
#> <chr> <lgl> <chr> <chr> <chr> <lgl>
#> 1 MILD TRUE Severity/Intensity S… AESEV C66769 FALSE
#> 2 mild FALSE Severity/Intensity S… AESEV C66769 FALSE
#> 3 Grade 2 FALSE Severity/Intensity S… AESEV C66769 FALSEEverything returned is a plain tibble underneath, so it goes straight
into dplyr, into a listing, or into your existing tracking
spreadsheet:
dplyr::count(tibble::as_tibble(fixes), fix_class, deterministic)
#> # A tibble: 10 × 3
#> fix_class deterministic n
#> <chr> <lgl> <int>
#> 1 ct_codelist FALSE 1
#> 2 ct_codelist TRUE 1
#> 3 human_judgement FALSE 3
#> 4 iso8601_date TRUE 1
#> 5 key_not_unique TRUE 1
#> 6 missing_variable TRUE 1
#> 7 referential_integrity TRUE 1
#> 8 required_value_null FALSE 1
#> 9 variable_length FALSE 1
#> 10 variable_length TRUE 1