From Excel Spreadsheets to AI: Modernizing Dairy Science with Posit & Snowflake at DairyNZ

Five years ago, DairyNZ relied on Excel spreadsheets and a small team of statisticians for its analytics. They were limited in their ability to handle large datasets effectively on local laptops. Insights were siloed, reproducibility was a challenge, and farmers had to wait 12–24 months for the release of economic surveys. In a sector where fertilizer, feed, and fuel prices can spike within a season, such delays limit the value of data for real-world farm decisions.

At the same time, new technologies were generating a “metric ton” of high-frequency data. Wearable sensors for thousands of cows (think Apple watches for cows), weather stations, and IoT devices began streaming 60 observations per second, demanding scalable infrastructure and modern workflows. 

Traditional linear science—experiment, analysis, publish—could not keep up with the new scale and speed of data. DairyNZ needed “fit for purpose tools” that supported iterative, real-time analytics and made insights easily shareable with farmers and analysts alike. They needed to transition from having siloed R users to supporting an IT-backed data science team.

DairyNZ rebuilt its foundation for data science with Posit Team and Snowflake—establishing infrastructure that supports everything from high-frequency farm data to AI-enabled research.

Mark Neal, Head of Data Science at DairyNZ highlights, “When the smoke is starting to pour out of your laptop, it’s time to move that compute to the cloud. Our data analyses efficiently pull the relevant data from Snowflake into our Posit tools like Workbench with code in a secure and very low friction approach that scales beautifully.”

Snowflake Data Warehouse: Centralizes data from thousands of farms with a bronze/silver/gold quality framework, ensuring scalability, reproducibility, and trust. The combination of Posit and Snowflake has allowed for reduced IT management overhead, seamless connectivity as well as credential management, and persistent data storage. 

Posit Workbench & Posit Connect: Provide secure, cloud-based compute for large simulations and a centralized environment to publish dashboards, Shiny apps, and Quarto reports internally and externally.

Skills Development & Continuous Improvement: DairyNZ sought to create an effective, continuous improvement mindset and a culture that recognized employee learning. The company ran internal cohorts and external R training programs, as well as internal GitHub and Snowflake lessons, with spaced learning and summative assessments, ensuring scientists could confidently adopt modern tools (including AI-assisted workflows). They also have an internal R Community where they continue to learn from speakers and celebrate wins across the team.

Farmer-Facing Tools:

• • Econ Tracker: Updates quarterly (instead of annually), enabling farmers to budget, compare against peers, and order inventory up to six months in advance based on forecasted fertilizer and fuel prices.
  • Pasture Potential Tool: Interactive dashboards that let farmers measure pasture harvest performance, a key indicator of farm productivity.
  • Connected Farm Prototypes: Dashboards that help farmers predict and mitigate cow heat stress by combining wearable sensor data and weather forecast data.

Researcher-Facing Tools: Quarto and Shiny applications provide near real-time trial data, helping scientists track feed supply, milk production, and emissions reduction experiments as they happen.

By modernizing with Posit and Snowflake, DairyNZ has established the foundation for AI-driven animal science while delivering measurable impact across the value chain:

• Reports are available 4X faster: Once backward-looking, reports have transitioned into forward-thinking tools as reports that once took up to two years are now updated quarterly.
• Farmer profitability: Farmers now make informed decisions on input purchases, anticipate market shifts, and track profitability alongside sustainability goals.
• Broader stakeholder value: Farmers use tools for budgets and comparisons, banks and analysts assess sector-wide data, and policymakers evaluate regional impacts. Researchers leverage real-time data streams, improving publication quality and accelerating innovation.

• Transitioned from Excel-based workflows to Snowflake-powered infrastructure
• Adopted Posit Workbench for heavy simulations and Posit Connect for publishing dashboards and reports
• Trained scientists and economists in R and AI-enabled analysis through structured, spaced learning programs
• Uses GitHub for sharing and versioning code
• Introduced digital-first data collection apps to improve data quality directly from the field
• Built farmer-facing Shiny apps and dashboards, making data accessible for thousands of end users
• Established an internal R community and continuous improvement culture, reinforcing best practices

Looking ahead, DairyNZ plans to integrate LLM-powered research assistants and expand AI-driven decision tools, ensuring that New Zealand’s dairy sector continues to lead the world in profitable, sustainable, and innovative farming.