Assess the risk of installed R packages — assess

Runs the riskmetric pipeline (pkg_ref() -> pkg_assess() -> pkg_score()) against the installed copy of each package and returns one tidy row per package: the overall riskmetric score plus every individual metric riskmetric returned. A package riskmetric cannot reference at all (not installed, unreadable metadata) is still given a row, with NA metrics and the reason in error, so a single bad package never aborts the run.

assess_risk(
  pkgs,
  lib = .libPaths(),
  assessments = riskmetric::all_assessments()
)

# S3 method for class 'valpack_risk'
print(x, ...)

Arguments

pkgs: Character vector of installed package names.
lib: Library paths to search, passed to riskmetric::pkg_ref().
assessments: List of riskmetric assessment functions. Defaults to riskmetric::all_assessments().
x: A valpack_risk object.
...: Ignored.

Value

A tibble of class valpack_risk with columns package, version, score, error, and one numeric column per riskmetric metric. The assessments attribute holds the names of the assessment set used; two valpack_risk tibbles are comparable only if that attribute matches.

Details

pkg_score is riskmetric's weighted roll-up on a 0-1 scale where higher is better (it is a quality score, despite the package name). Metrics that riskmetric could not evaluate come back as NA rather than 0.

The default assessments includes metrics that query CRAN over the network (downloads, reverse dependencies, CRAN check results). Pass a subset of riskmetric::all_assessments() for a fast, offline assessment — that is what the example below does.

Subset scores are not comparable to full scores

riskmetric weights its roll-up across the assessments it is given, so the same package at the same version scores differently under different assessment sets. Measured on this machine (R 4.5.2, riskmetric 0.2.7):

package	2 metrics	5 metrics	all 19
`tibble`	0.0000	0.1480	0.3478
`jsonlite`	0.0000	0.1036	0.3615
`whisker`	0.5000	0.2642	0.4600

The gap is large — tibble is 0.1480 on a five-metric subset and 0.3478 on the full set, a factor of 2.4 — and it is not signed: whisker scores higher on two metrics (0.5000) than on all nineteen (0.4600). So "a subset deflates the score" is not a safe rule of thumb either; the two numbers are simply not on the same scale. Compare subset to identical subset only.

To make that checkable rather than a matter of memory, the returned tibble carries an assessments attribute naming the set used, print() shows it, and evidence_bundle() writes it onto the risk page.

Examples

# Offline subset of assessments keeps this fast and network-free.
quick <- riskmetric::all_assessments()[c(
  "assess_has_news", "assess_has_vignettes", "assess_has_examples",
  "assess_exported_namespace", "assess_has_maintainer"
)]
risk <- assess_risk(c("tibble", "jsonlite"), assessments = quick)
risk
#> 
#> ── valpack risk assessment ─────────────────────────────────────────────────────
#> ℹ 2 packages assessed | 2 scored | riskmetric 0.2.7
#> ℹ 5 assessments: assess_has_news, assess_has_vignettes, assess_has_examples, assess_exported_namespace, assess_has_maintainer
#> ! Scores are comparable only to scores from this same assessment set.
#> ℹ score range 0.104 - 0.148 (higher is better)
#> # A tibble: 2 × 9
#>   package  version score error has_news has_vignettes has_examples
#>   <chr>    <chr>   <dbl> <chr>    <dbl>         <dbl>        <dbl>
#> 1 tibble   3.3.1   0.148 NA           1             1        0.870
#> 2 jsonlite 2.0.0   0.104 NA           1             1        1    
#> # ℹ 2 more variables: exported_namespace <dbl>, has_maintainer <dbl>
risk$score
#> [1] 0.1480046 0.1036422