Kubernetes is hard! Lessons learned taking our apps to Kubernetes

Kubernetes is hard! Lessons learned taking our apps to Kubernetes

Why? Environment flexibility Developer, QA, Support, Product, Solution… anyone! Per branch CI/CD Save money and resources Dogfooding @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

The end Take it easy Your app is not ready Limits are good (my mom said that) Probes Observability Community @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

🎩 @ErinMeyerINSEAD’s “Culture Map”

shownotes Slides Video Links Comments Ratings Raffle! https://jfrog.com/shownotes @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

What I’ve been promised @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

What is it in real life @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

@jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Ribbit with us: #swampUP

Your app is not ready. @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

From any “logging best practices” talk or article: @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

Your app is not ready Data Persistency You need it. Or do you? @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

Your app is not ready High availibility If you did HA before K8S, you are going to rewrite it Scaling Up and down Updates and downgrades Some pods will be newer than others (or older) @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

Now let’s look at kubernetes @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Limit everything … resources: requests: memory: “1Gi” cpu: “100m” limits: memory: “2Gi” cpu: “250m” … Always limit everything Coordinate with applications limits Java -Xms=1g -Xmx=2g RabbitMQ [rabbitmq.conf] total_memory_available_override_value = 1GB MongoDB —wiredTigerCacheSizeGB=1 @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

Take care of the health It’s always a good idea, mandatory with K8S readinessProbe When the app is ready to serve livenessProbe Is this thing alive? Probe types Exec - returns 0 Http - returns < 400 … readinessProbe: httpGet: path: /api/system/health port: 8080 … livenessProbe: exec: command: - mongo - —eval - “db.adminCommand(‘ping’)” … livenessProbe: tcpSocket: port: 5672 … Tcp - did we manage to open a port If needed, write an exec script @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

More than one container in a pod Initialization (before your container is up) Preheat your cache Run config scripts Sidecar design pattern Log collector Monitoring Network proxy (e.g. Istio) @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

How to deploy anything to k8s Copy YAML Paste YAML Fix indents Repeat @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Kubernetes resource { “kind”: “Deployment”, “apiVersion”: “extensions/v1beta1”, “metadata”: { “name”: “my-release-docker-app-chart” }, “spec”: { “containers”: [ { “name”: “docker-app-chart”, “image”: “docker.artifactory/docker-app:1.0”, @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Let’s build a new one! > docker build –t docker.artifactory/docker-app:1.1 @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

One last thing… > sed –i.bak s#docker.artifactory/docker-app:1.1#${imageTag}# deployment.yaml @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Or just use :latest “image”: “docker.artifactory/docker-app:latest” @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Enter helm @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Encapsulated packages of Kubernetes deployments All this… Becomes this xrayxray-analysis xray-event xray-indexer xray-nfs-server xray-persist xray @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Powerful templating for descriptor files { “kind”: “Deployment”, “apiVersion”: “extensions/v1beta1”, “metadata”: { “name”: “{{ template “docker-app.fullname” . }}” }, “spec”: { “containers”: [ { “name”: “{{ template “docker-app.name” . }}”, “image”: “{{ .Values.image.repository }}: {{ .Values.image.tag }}” @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Values: # Default values for docker-app. # This is a YAML-formatted file. # Declare name/value pairs to be passed into your templates. image: repository: docker.artifactory/docker-app tag: 1.1 secretName: regsecret pullPolicy: Always @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Simple! Templates Values Metadata @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Chart <-> image relationship Using templates we can reuse charts for multiple image versions Chart versions != Image versions @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Helm repositories Official repository - kubeapps.com Get a local one! Option 1: Create your own: Run an http server with index.yaml Run helm repo index to generate one the index Option 2: Use JFrog Artifactory Universal Artifact Repository which supports Docker, Helm and everything else @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

observability No more ssh and grep in logs No need for production access for everybody (unless you pretend you’re Netflix) Observability tools: Log aggregation (Sumo Logic, Splunk, Log Entries) APM (New Relic, AppDynamics) Monitoring (DataDog, SignalFX) Cloud-Native tools @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

Cloud-native observability - monitoring Prometheus Grafana @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

Cloud-native observability - logs Fluentd ElasticSearch Kibana @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

We’re big on those pipelines and promotions concepts If quality requirments are hit CI SERVER 1 If quality requirments are hit 2 Integration If quality requirments are hit 3 System Testing 4 Staging Production *

• Quality gates - @jbaruch #DevDialogue #k8S http://jfrog.com/shownotes

Integration with CI servers @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

Jfrog and k8s GoCenter.io Distribution Development, testing @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

Community Probably someone had this problem before CNCF Ambassadors Project maintainers and commiters Kudos RabbitMQ HA MongoDB (Bitnami) @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

The end Take it easy Your app is not ready Limits are good (my mom said that) Probes Observability Community @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes

Twitter ads and Q&A http://jfrog.com/shownotes @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP

Happy sailing! @jbaruch #DevDialogue #k8S Ribbit with us: #swampUP http://jfrog.com/shownotes