Maintain the modular SCE

Environments need to be reproducible, and this responsibility starts with IT qualification of the infrastructure and the operating system and software installation. But reproducibility does not stop on the IT side. Analyses run by biostatisticians need to be reproducible as well. Posit Workbench will help by directly offering or providing support for tools like renv, projects, version control, etc.

RenV | Posit Projects | Version Control

Do you need to validate your environment?

Absolutely. As a medical body, a regulatory agency needs to know that the code you ran a couple of days ago can be reproduced when asked. Organisations can integrate the setup Posit Team into their Computerized System Validation (CSV) process that will lead to a fully validated environment. CSV starts with user requirements gathering, includes IT Qualification of the infrastructure, operating system and software installation and will end with testing against the user requirements that will give the system the validation status.

Testing resources for packages

While everyone talks about package validation, the actual process can be more accurately described as package testing. Strictly speaking, you only can ever validate a workflow or business process but you never validate a software as such. However, when validating a workflow, it is imperative to ensure that the software used is producing correct results, for example and this can be achieved via package testing. There are many tools out there to help with automating those tests with some being listed below.

When it comes to packages, there are two categories to keep in mind: public packages and internal packages. Some Pharma specific packages are supported and maintained by communities within the Pharma space while others are regularly maintained and tested by repositories such as CRAN and others. Each repository has their own criteria and selection process to allow packages into that respective repository and as such the fact that a package is part of a given repository can provide insights into how trustworthy this package is.

Testing Basics | testthat | {shinyValidator} | Shinymeta

While in principle a package would need to be tested for each of the functions it offers and for the full parameter space it allows, the testing effort very quickly can become a problem both in terms of time and resources. As a consequence, the typical approach used when testing (and later on validating) packages is risk-based validation.

The overall approach is documented in a validation strategy and validation plan. Both describe a consistent approach to package testing where clearly defined criteria are being used to infer the risk arising from a given package. This risk can be assessed by mixing various metrics (e.g. download count, open issues on github, number of reverse dependencies, …) For the risk assessment the Validation Hub has developed tools that simplify this assessment. Based on the calculated risk a specific testing strategy is informed (as defined in the validation plan). While undertaking such a rigorous risk assessment and testing can quickly become very time-consuming if done in-house, there are companies like Atorus that can help you here via their OpenVal product.

OpenVal | R Validation Hub