Predicting Lending Rates with Databricks, tidymodels, and Posit Team
Machine learning algorithms are reshaping financial decision-making, changing how the industry manages financial risk.
Garrett Grolemund at Posit shows how to use both Posit and Databricks to apply machine learning methods to the consumer credit market, where accurately predicting lending rates is critical for customer acquisition.
*Please note that while the workflow focuses on a financial example, the general workflow will be useful to those using Databricks and R together across any industry.
During this workflow demo, you will learn how to:
- Connect to historical lending rate data stored in Databricks Delta Lake
- Tune and cross-validate a penalized linear regression (LASSO) that predicts interest rates
- Select variables with the penalized linear regression model (LASSO)
- Build an interactive Shiny app to provide a customer-facing user interface for our model
- Deploy the app to production on Posit Connect, and arrange for the app to access Databricks
Resources for the demo:
- GitHub repo for today’s materials: https://github.com/posit-dev/databricks-finance-app
- Accompanying Guide: https://pub.demo.posit.team/content/
- Q&A Recording: https://youtube.com/live/wNI3AhHP7uM
Additional follow-up links:
- GitHub Repo: https://github.com/posit-dev/databricks-finance-app
- Accompanying Guide: https://pub.demo.posit.team/content/
- While this demo uses ODBC package to connect to Databricks, you can also use sparklyr R package. Learn more about both here: https://docs.posit.co/ide/server-pro/user/rstudio-pro/guide/databricks.html
- Example using sparklyr instead of ODBC: https://posit.co/blog/reporting-on-nyc-taxi-data-with-rstudio-and-databricks/
- Posit Workbench provides additional features for managing Databricks Credentials, learn more here: https://docs.posit.co/ide/server-pro/user/posit-workbench/guide/databricks
- For more on the Posit x Databricks partnership: https://posit.co/solutions/databricks/
- Blog post on Edgar’s workshop on Databricks at conf: https://posit.co/blog/using-databricks-with-r-conf-workshop/
- Solutions article on ODBC and Databricks: https://solutions.posit.co/connections/db/databases/databricks/
- Want to chat more with Posit?
To talk with Posit about integrating Posit & Databricks: https://posit.co/schedule-a-call/
Had fun and want to join again? You can add the monthly recurring event to your calendar with this link: https://pos.it/team-demo