This event has ended. Visit the official site or create your own event on Sched.
Back To Schedule
Wednesday, November 9 • 12:35pm - 1:15pm
Pachyderm: Unlock the Power of Kubernetes for Big Data - Joey Zwicker, Pachyderm

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Pachyderm is an open source big data analytics platform completely deployed on Kubernetes. Pachyderm leverages K8s's jobs API to process massive data workloads and build streaming pipelines. Pachyderm's hallmark feature is version-controlled data including viewing branches, commits and diffs for petabyte-scale data sets.

In this talk we'll demonstrate how Kubernetes and Pachyderm empowers data science teams to collaborate on a shared and unified data infrastructure. Everything is run on Kubernetes including streaming data ingestion, machine learning pipelines, to automatic service deployment using Rolling Updates.

Our talk will discuss how Pachyderm couldn't exist without a large swath of advanced Kubernetes primitives and includes demo where we stream data through the system and watch Kubernetes automatically schedule analytics containers and parallelize the data processing. This demo is inspired directly by how production users are managing data in Pachyderm today.


Wednesday November 9, 2016 12:35pm - 1:15pm PST
Grand Ballroom B