Build Define-XML 2.1 in pure R, and check it against the datasets it describes.
Define-XML is usually produced by a commercial tool or by a round trip through Excel, and then it drifts: a Length that no longer covers the longest value actually present, a codelist term used in the data but never declared, a variable dropped from the data but left in the metadata. Nothing in that workflow reads the data. define21 does.
data frames -> define_spec() -> write_define() -> define.xml
|
+-------> check_define(spec, data) -> tibble of discrepanciesThe metadata is eight plain data frames: datasets, variables, codelists, methods, comments, plus value_levels, where_clauses and external_codelists for value level metadata and dictionary codelists.
install.packages(
"https://clincoder.cloud/define21/define21_0.0.0.9000.tar.gz",
repos = NULL, type = "source"
)
library(define21)
ex <- define21_example() # a small SDTM spec (DM, AE, VS) and its datasets
spec <- ex$spec
write_define(spec, "define.xml") # Define-XML 2.1, validated against the CDISC schema
check_define(spec, ex$data)
#> -- Define-XML consistency check --
#> v 0 errors
#> ! 2 warning(s)
#> i codelist_external: 2 # AEDECOD/AEBODSYS are MedDRA; no terms to check
# Now let the data drift away from the metadata, as it always does.
dm <- ex$data$DM
dm$USUBJID[1] <- "STUDY01-A-VERY-LONG-SUBJECT-IDENTIFIER" # wider than Length=20
dm$SEX[2] <- "U" # not in codelist SEX
dm$ARM <- NULL # variable dropped
res <- check_define(spec, list(DM = dm, AE = ex$data$AE, VS = ex$data$VS))
res$message[res$severity == "error"]
#> [1] "Variable 'ARM' is declared for dataset 'DM' but is not a column of it."
#> [2] "Variable 'DM.USUBJID' declares Length 20 but the longest value is 38
#> characters ('STUDY01-A-VERY-LONG-SUBJECT-IDENTIFIER'). Set Length to at least 38."
#> [3] "Variable 'DM.SEX' holds 1 value(s) absent from codelist 'SEX': 'U'."
# Value level metadata is checked the same way, on the rows its where clause
# selects. mmHg is a legal VSSTRESU for the variable, but not for PULSE.
vs <- ex$data$VS
vs$VSSTRESU[vs$VSTESTCD == "PULSE"] <- "mmHg"
subset(check_define(spec, list(DM = ex$data$DM, AE = ex$data$AE, VS = vs)), !is.na(value))$message
#> [1] "Value 'PULSE' of variable 'VS.VSSTRESU' holds 1 value(s) absent from
#> codelist 'VSRESU_HR': 'mmHg'."
read_define("define.xml") # parse a define back into a spec object| Function | Purpose |
|---|---|
define_spec() |
Validate plain data frames of metadata into a define21_spec. Every problem is reported at once. |
write_define() |
Emit a Define-XML 2.1 document with xml2. |
read_define() |
Parse a Define-XML 2.1 file back into a define21_spec. |
check_define() |
Compare the metadata with the real datasets; return a tibble of discrepancies. |
define_types() |
The permitted Define-XML 2.1 DataType values. |
define21_example() |
The example specification and datasets used throughout. |
ODM root (ODM 1.3.2 + def 2.1 namespaces, def:DefineVersion="2.1.0"), Study/GlobalVariables, MetaDataVersion, def:Standards, ItemGroupDef (with Purpose, def:Structure, def:Class, def:ArchiveLocationID and its def:leaf), ItemRef (OrderNumber, Mandatory, KeySequence, Role, MethodOID), ItemDef (DataType, Length, SASFieldName, def:DisplayFormat, def:Origin, CodeListRef), CodeList with CodeListItem/Decode or EnumeratedItem, MethodDef with FormalExpression, and def:CommentDef.
Value level metadata (new in this version): def:ValueListDef referenced from a variable’s ItemDef by def:ValueListRef, each of its ItemRefs carrying a def:WhereClauseRef into a def:WhereClauseDef built from RangeCheck/CheckValue. Written and read, and checked against the data by check_define() on the rows the where clause selects.
External codelists (new in this version): a CodeList holding one ExternalCodeList with Dictionary, Version, href and ref - MedDRA, WHODrug, a CDISC CT release, ISO 3166.
Everything below is a deliberate boundary, not an oversight.
Not emitted and not read:
def:AnnotatedCRF, def:SupplementalDoc, def:DocumentRef, def:PDFPageRef. The only def:leaf written is the dataset archive location.Alias (the nci:ExtCodeID codes on codelists and terms), SignificantDigits, ItemDef-level RangeCheck, MeasurementUnitRef, def:IsNonStandard, def:HasNoData, def:ExtendedValue.def:Standard. spec$standard holds one, so a document listing an IG standard plus its CT standards keeps only the first, and def:StandardOID on a CodeList is not preserved.FormalExpression per MethodDef; only the first is kept.Shapes this package refuses rather than mangles:
ItemDef shared by several ItemGroupDefs is attributed to the first that references it, and writing back expands it into one ItemDef per dataset. The OIDs change; the metadata does not.ItemRef/@OrderNumber within one ItemGroupDef or one def:ValueListDef is rejected by define_spec(), because the schema forbids it (UC-IGD-2, UC-VLD-2). Some real documents contain it; they are themselves schema-invalid.RangeCheck expressed with FormalExpression instead of CheckValue is rejected, not guessed at.check_define() reports codelist_external rather than passing it silently.check_define() reports where_clause_not_evaluable.OrderNumber order, so a document that lists them in some other order round trips with the same content in sorted order./opt/cosa/library-xml/schema/cdisc-define-2.1/. The shipped example, with value level metadata and external codelists, validates TRUE; the same document with one corrupted DataType - at variable level and at value level - validates FALSE.ExternalCodeList): read, written back, and compared element by element. Every construct listed under “What is supported” is preserved exactly; the re-emitted document validates against the XSD; reading it again gives an identical specification.This replaces the %define macro suites that read a spec workbook with PROC IMPORT and write XML with DATA _NULL_ / put. Two differences worth the switch: the spec is validated before a single tag is written, and check_define() reads the actual datasets, which a put-statement generator never does.