Empowering R and Python Developers: Databricks and Posit Announce New Integrations
This is a cross-post of the announcement on the Databricks blog.
Today, Databricks and Posit are thrilled to announce a strategic partnership to provide R and Python developers with an unmatched experience when working with Databricks. To meet this goal, our product and engineering teams at Posit are collaborating to deliver an enhanced integration between Posit Workbench and Databricks, support for Spark Connect in R, and the hosting of Posit Workbench on the Databricks Marketplace. Let’s take a closer look:
Enhancing the Databricks developer experience with Posit Workbench
Posit Workbench is the standard for enterprise deployments of RStudio IDE, the fan favorite development experience for R developers everywhere. The new Databricks integration with Posit Workbench will simplify connecting to Databricks Workspaces, accessing data, running code, and interacting with other resources like Databricks Jobs or MLflow models. We are very excited to be able to bring Databricks to wherever you have Posit Workbench installed!
Sparklyr unlocks Spark Connect for R developers
Since Apache Spark 3.4, Spark Connect was introduced to decouple the client-server architecture, allowing IDEs and modern data applications to access Spark interactively with minimal setup. Now, Posit is adding support for Spark Connect in R via sparklyr, making it easier for users to access Spark clusters – including Databricks clusters via Databricks Connect. For many R developers, sparklyr is the preferred method to work with Apache Spark due to its unification with dplyr and the tidyverse ecosystem as well as its integration with the Connections pane in RStudio IDE. sparklyr will continue this tradition and improve parity with Python through wrapping the PySpark API, giving R developers more choices when it comes to writing Spark applications.
Posit Workbench in the Databricks Marketplace
Mutual customers often ask us if Posit products can be hosted within their Databricks Workspace. Today, this is limited to RStudio Server or Posit Workbench installed directly on Databricks compute resources in the data plane. Bringing Posit Workbench to the Databricks Marketplace allows for a much deeper integration. Databricks Apps run directly on a customer’s Databricks instance, integrate with their data, and enable users to interact through single sign-on: combining the highest standards of security, privacy, compliance, and scalability with data never having to leave the customer’s account. Customers can enjoy their favorite developer tooling right on the shores of their Databricks Lakehouse.
The Future is Bright
As Databricks and Posit further strengthen their partnership, the data community stands to reap the benefits of these exciting new developments. The sparklyr package, improved integration of Posit Workbench with Databricks, and the addition of Posit Workbench in the Databricks Marketplace demonstrates our commitment to providing comprehensive solutions for our shared customers. We look forward to witnessing the transformative impact of these advancements on our customers’ data-driven journeys.
Please schedule a call to chat with our Sales and Product teams to learn more about these upcoming features, and subscribe to the blog for product update notifications.