Hey, Helm, can you scale?
A presentation at Velocity in in London, UK by Baruch Sadogursky
Hey, Helm, can you scale? @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
About me @jbaruch
Shownotes • https://jfrog.com/shownotes • The slides • The video (by tomorrow) • All the links • Comments, ratings • Raffle! @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Poll time! Let’s calibrate the level first @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Helm what? • Heard about Helm • Played a bit with Helm • Using Helm in non-prod environments • Using Helm in production • I wrote this thing @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
What the helm is helm? Dependency manager for Kubernetes @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
How to deploy anything to k8s • Copy YAML • Paste YAML • Fix indents • Repeat @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Kubernetes resource { "kind": "Deployment", "apiVersion": "extensions/v1beta1", "metadata": { "name": "my-release-docker-app-chart" }, "spec": { "containers": [ { "name": "docker-app-chart", "image": "docker.artifactory/docker-app:1.0", @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Let’s build a new one! > docker build –t docker.artifactory/docker-app:1.1 @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
One last thing… > sed –i.bak s#docker.artifactory/docker-app:1.1#${imageTag}# deployment.yaml
@jbaruch
#VelocityConf
@HelmPack
jfrog.com/shownotes
@jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Or just use :latest "image": "docker.artifactory/docker-app:latest" @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Enter helm @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Encapsulated packages of Kubernetes deployments All this… Becomes this xrayxray-analysis xray-event xray-indexer xray-nfs-server xray-persist xray @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Powerful templating for descriptor files { "kind": "Deployment", "apiVersion": "extensions/v1beta1", "metadata": { "name": "{{ template "docker-app.fullname" . }}" }, "spec": { "containers": [ { "name": "{{ template "docker-app.name" . }}", "image": "{{ .Values.image.repository }}: {{ .Values.image.tag }}" @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Values: # Default values for docker-app. # This is a YAML-formatted file. # Declare name/value pairs to be passed into your templates. image: repository: docker.artifactory/docker-app tag: 1.1 secretName: regsecret pullPolicy: Always @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Simple! • Templates • Values • Metadata @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Chart < -> image relationship • Using templates we can reuse charts for multiple image tags • Chart versions != Image tags • Or are they…? @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Kubernetes cluster control @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Two parts Helm client • Local chart development • Managing repositories • Interacting with the Tiller server @jbaruch #VelocityConf Tiller Server • Listening for incoming requests from the Helm client • Combining a chart and configuration to build a release • Installing charts into Kubernetes, and then tracking the subsequent release • Upgrading and uninstalling charts by interacting with Kubernetes @HelmPack jfrog.com/shownotes
Helm commands > helm > helm > helm > helm > helm init search install status repo @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Helm repositories • Official repository - kubeapps.com @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
@jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Helm repositories • Official repository - kubeapps.com • Get a local one! • Option 1: Create your own: • Run an http server with index.yaml • Run helm repo index to generate one the index • Option 2: Use JFrog Artifactory (or others) • Kubernetes Registry which supports Helm, containers, and everything inside them @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Helm 3 • No tiller (who needs it anyway) • Lua instead of YAML @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Software engineering venn diagram SOFTWARE I KNOW REALLY WELL SOFTWARE I LIKE @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
What Dependency managers and printers have in common? @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
@jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
@jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
7 deadly sins of package managers 1. 2. 3. 4. 5. 6. 7. Over-architecture Not thinking of enterprise scenarios Having downloadable index Cross-site dependency resolution loopholes Author authentication done wrong Version management (or lack of thereof) Using wrong place for central registry (and hardcoding it!) @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
7 deadly sins of package managers 1. 2. 3. 4. 5. 6. 7. Over-architecture Not thinking of enterprise scenarios Having downloadable index Cross-site dependency resolution loopholes Author authentication done wrong Version management (or lack of thereof) Using wrong place for central registry (and hardcoding it!) @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Enterprise scenarios • In-house registry • User authentication and authorization • org/project/team segmentation with central management • Promotion pipelines @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
@jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Downloadable index? • Pros: • Simple server • Offline search (not sure how it’s useful?) • Offload the search off the server to spare computer power @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Downloadable index? • Pros: • Simple server • Offline search (not sure how it’s useful?) • Offload the search off the server to spare computer power • Cons: • Trends to become obsolete • Index is a bottleneck (specially if done wrong) @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Let’s run some tests on 120k of charts… @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
120k of charts?! • True, you can reuse same charts for different image tags • What will be the value in values.yaml? • LATEST probably • Every time you run you have to specify the image tag @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
120k of charts?! • True, you can reuse same charts for different image tags • What will be the value in values.yaml? • LATEST probably • Every time you run you have to specify the image tag @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
You’ll have a Cartesian product of charts from the all the tags of all the images in the charts. That’s a lot of charts. @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Index bottlenecks docker run -m=512m -it alpine-helm helm repo add helm-prod $helm-repo @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
How can we fix it? • gzip index in transit • Fixes some of the issues • Distribute the index @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
@jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Divide and conquer! • Main index: list of apps (with latest version) • artifactory:5.8.3 • App index: list of versions (and app-level metadata) • • • • description maintainers keywords sources • Version index: the details of the version • • • • appVersion created digest url @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Structured index requires structured repository • Layout! • Repo • App • ver1 • ver2 • This complicates the push! • We used to just upload the file! • See debs (or don’t) @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Let’s talk about push @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
Q&A and Twitter ads • @jbaruch • #VelocityConf • @HelmPack • https://jfrog.com/shownotes @jbaruch #VelocityConf @HelmPack jfrog.com/shownotes
ackage managers are hard. Helm learned a lot of lessons from others’ mistakes but also repeated some. For example, having a single index file per repository is not scalable. It’s the same mistake that NPM made, and it causes slower CI, high memory consumption, slower searches, and more. Another example is not having private enterprise repositories in mind, leaving out authorization and authentication features.
Baruch Sadogursky explores several solutions to those problems, covering their strengths and their weaknesses.
