R packages and Shiny for FDA clinical trial submissions

Imagine you’re tasked with making a critical decision.

Some people really want your ruling to go one way, others would much prefer it go another.

Your choice is important, potentially benefiting or harming a huge number of people.

Fortunately, your decision will be informed by numerous experiments and rigorous analysis. You’ve helped define the conditions under which that work is done, organized around the goal of getting the best information you can to make your decision. You’ll actually get a lot of high-quality data.

However, now imagine all that data arrives at your office in the form of oodles of static .pdf & .rtf files. A single, crucial table may be split into hundreds of pages. Also, imagine that too many people perceive your organization as having an informal bias preferring proprietary software to do this work. Your first job will be to wade through all this digitized paper.

In many ways, this is the world in which the Food and Drug Administration (FDA) operates. Not too long ago, clinical trials submission data for new therapeutics would appear in the form of boxes of paper, practically submitted by the forklift. Once the FDA requests revisions, sponsors may then need to resubmit hundreds of new tables and figures in reply.

No one argues that national health regulators should have a less exacting process when making these kinds of important decisions. But as Janet Woodcock, Principal Deputy Commissioner of the FDA, has written, the reliance on ‘Digitized Paper’ is slowing drug development, and many echo her sentiment.

In similar decision settings, decision-makers have transitioned to graphic and interactive tools for data analysis. These tools offer ways to visualize and interact with large amounts of data effectively. Interactivity allows decision-makers to pose initially unforeseen questions, no forklift return trips required.

And relying on open-source tooling to do this work provides advantages, promoting transparency, efficiency, accessibility, and fairness. Supporting open-source software helps ensure that unintentional or perceived bias towards one tool or another doesn’t promote vendor lock-in among sponsors. Open source means all code and analysis, including how all underlying statistical methods are implemented, is available for inspection. Regulators don’t need to rely on the word of a software vendor. The final analysis may be attached to a clear, step-by-step mapping from the original study data.

These practices have the opportunity to be more effective at processing the kinds of data and analyses that appear in clinical trial submissions. And they can help the FDA to more quickly and accurately assess the safety and efficacy of new medical products.

We celebrate the work of the R Consortium’s R Submissions Working Group

Posit, as a Public Benefit Corporation, is a strong supporter of open-source software in scientific research and technical communication. Practices like these are prevalent throughout scientific research and business, and we advocate for greater adoption in regulatory submissions.

However, the nature of decisions made by analysts in business and academia differs from the critical, life-or-death decisions that FDA analysts are responsible for informing. While interactive visualizations and open-source software may be desirable for many reasons, their introduction creates new difficulties. Interactive features challenge reproducibility. The ability to filter and drill down into data creates opportunities to form incorrect opinions based on a misuse of a tool. Submission sponsors may be reluctant to rely on new tooling if they are concerned regulators are less comfortable with it.

How do regulators and the industry adopt these practices responsibly?

We celebrate the work of the R Consortium’s R Submissions Working Group in supporting this change.

This cross-industry pharma working group includes members from FDA, in addition to Argenx, Atorus, Bayer, Beigene, BD, Biogen, Eli Lilly, Emdserono, EMD, EQRx, GlaxoSmithKline, Janssen, Johnson & Johnson, Merck, Moderna, MSD, Novartis, Novo Nordisk, Pfizer, Phastar, Posit PBC, Procogia, Regeneron, Roche/Genentech, Serono, Syneos and the University of Michigan.

Since 2021, this group has demonstrated and documented open-source R-based submissions, given the operational and infrastructure constraints under which the FDA functions. Additionally, industry and regulators collaborate to pilot new technologies, like Shiny for interactive visualization, that may prove useful to the FDA.

They are responsible for two successful pilot submissions in the past year.

• Pilot 1 concluded in March 2022. The group submitted an R package and responded to FDA comments. To our knowledge, this was the first publicly available regulatory submission using an open-source language and reproducible package structure. See the announcement and the repository.
• Pilot 2 was successfully accepted by the FDA in December 2022, and includes a Shiny application. To our knowledge, this is the first publicly available submission package that includes a Shiny component. See the announcement and repository.

Both submissions were based on publicly available data and done as a trial, allowing both to be fully open to the public. They include submission repositories where all code, documentation, revisions, and communications between the sponsor and the regulator are available. Anyone can review and learn from these before they make their own submission.

These certainly are not the first FDA submissions to incorporate open-source software, visualizations, or Shiny, but this work is groundbreaking. It goes a long way in demonstrating how to use open-source tools and Shiny applications in clinical trial submissions and helping permit other sponsors to adopt these practices. The group shows how collaboration between regulators and industry can have a transformative impact on how each side approaches submissions.

We also hope this work helps dismiss the myth that the FDA has any bias towards one software tool or another or is resistant to change.

Learn more about R packages and Shiny for FDA clinical trial submissions

• If you are keen to learn from the R Submissions Working Group’s example, we encourage you to review their trial repositories and follow their activity at the R Consortium. If you would like to participate in this effort, please reach out to the group here.
• The working group emerged from R in Pharma. In August 2018, pharmaceutical companies gathered at Harvard University to discuss R and open-source tooling for drug development at R in Pharma. Since then, clinical reporting has been a major focus of these conferences, with workshops on clinical submissions, and keynotes by representatives from the FDA.
• At Posit, we have a dedicated Pharma team to help organizations migrate and utilize open source for drug development. To learn more about our support for life sciences, please see our dedicated Pharma page where you can book a call with our team.