CloudNativeCon + KubeCon 2016

IBM developed EGO as a powerful resource management technology, which helps to abstract all hardware into a resource pool for a variety of analytic and high performance workloads. It is similar to open source resource managers like Apache Mesos, but IBM EGO provides more enterprise features, e.g. time based scheduling, placement policies, dynamic load-balancing, and resource sharing (preemption/reclaim). To get the benefit of those enterprise features combined with Kubernetes support for cloud-native workloads, IBM provides a connector to run Kubernetes on EGO. This talk will present the details of Kubernetes on EGO integration such as how Kubernetes can leverage EGO preemption and reclaim features. This talk will also provide some key lessons learned about running Kubernetes together with other workload-specific schedulers.

With the proliferation of cloud services and the development of fine-grained virtualization techniques, the Network Service Agreement(SLA) requires to manage network resources efficiently for the large-scale, high-density computing units. Specifically, providing guaranteed Service Level Agreement for networking has been a challenge because of the high complexity between computing nodes. SLA Monitoring also plays an important role in classical SLA management model. We propose an efficient Networking Service Level Agreement system with an efficient topology-based feedback monitoring system, which provides dynamic networking SLA feedback such as Bandwidth, latency and packet drop rate, and adjustment will be based on them. The proposed solution, targeting at a general and robust design, keeps minor modification to the Cloud Management System and is verified by prototype implementation. The preliminary evaluations show that the proposed SLA scheduling is promising in terms of networking resource utilization, and can achieve higher performance compared with naive scheduling.

How do we get traffic to our Kubernetes Pods? Reaching for a Service may be our first instinct, but we're walled in by the combination of service types and provider integrations Kubernetes provides. Service resources give little in the way of user control, leaving us to bolt on separate abstractions while hoping the functionality we need is available in the next Kubernetes release. These abstractions can easily become another routing layer that deeply couples to our other resources without any benefits from native integration at the cluster manager level.

Wouldn't it be better if we could integrate our applications and traffic in a natural, Kubernetes-native way? The built-in Ingress resource solves common access problems and empowers users to build software for handling custom traffic patterns.

In this session we will talk about the mechanism of supporting multi schedulers in a Kubernetes cluster. First we will give an overview of multi-scheduler frameworks in various cluster management systems, including its evolution, the pros and cons of each different architecture. Then we will talk about how multi schedulers are currently supported in Kubernetes. We will also introduce a new multi-scheduler framework designed by Huawei PaaS team, which is inspired by the Omega paper but with lots of improvements. The framework is based on the general idea of optimistic concurrent control and global shared state. It includes lots of enhanced features like multi-level conflict criteria, various state synchronization mechanisms, priority-based conflict resolution, batch commit, policy management, etc.

Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. Every pod has its own IP address and there’s no need to explicitly create links between pods. Mapping container ports to host ports is unnecessary.
This is an elegant design where pods can be treated much like VMs or physical hosts from the perspectives of port allocation, naming, service discovery, load balancing, application configuration and migration.

A group of people passionate about this topic from several leading and respected technology companies got together recently to form a Special Interest Group (SIG) to design the specification. The same companies that collaborated on the spec also compete on the implementation as long as they follow the following fundamental requirements (barring any intentional network segmentation policies):

• All containers can communicate with all other containers without NAT;
• All nodes can communicate with all containers (and vice-versa) without NAT;
• The IP that a container sees itself as is the same IP that others see it as.

Open source is a truly fascinating new world where people working for complementary and/or competing technologies can work together and accomplish a common goal.
This is the case of the Kubernetes Networking Special Interest Group (Kubernetes-networking-sig). The group banded together to help solve the following four networking problems:

1. Highly-coupled container-to-container communications;
2. Pod-to-Pod communications;
3. Pod-to-Service communications;
4. External-to-internal communications.

Hear from a panel ranging from open source networking projects, such as MidoNet, Contiv Calico, and Flannel, who implemented their networking for Kubernetes and get their unique perspectives on how their respective solutions augment native Kubernetes networking. With the opportunity to collaborate during the SIG and compete on the execution - this is a panel you don’t want to miss!

Twitter generates tens of billions of events per hour when users interact with it. Analyzing these events to surface relevant content and to derive insights in real-time is a challenge. To address this, we developed and open sourced Heron, a new real time distributed streaming engine. In this presentation, we first describe the design goals of Heron and show how the Heron architecture achieves task isolation and resource reservation to ease debugging, troubleshooting, and seamless use of shared cluster infrastructure with other critical Twitter services. We subsequently explore how a topology self adjusts using back pressure so that the pace of the topology goes as its slowest component. Finally, we outline how Heron implements at-most-once and at-least-once semantics and we describe a few operational stories based on running Heron in production.

Kubernetes has accelerated application development time for many organizations but one of the most tedious aspects of moving from application prototypes to running pods on Kubernetes is the repetitive task writing pod configuration files. A common workflow for many teams is to write development configurations in docker-compose before running a pod in Kubernetes. After using various container clustering systems, Micah recognized the need to be able to quickly interchange between formats. In late 2014 Micah open-sourced container-transform for interchanging docker-compose and Amazon's EC2 Container Service tasks, and has since added several other application formats including Marathon, Chronos, and more recently Kubernetes! In this talk Micah will demonstrate how developers can convert an app from docker-compose to Kubernetes, or even migrate from another clustering systems to Kubernetes. https://github.com/micahhausler/container-transform

(re-submitted to include the github link)