Best Practices In Implementing Container Image Promotion Pipelines

Software I like Software I know really well

🎩 @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes @ErinMeyerINSEAD’s “Culture Map”

shownotes Øhttp://jfrog.com/shownotes Ø Slides Ø Video Ø Links Ø Comments, Ratings Ø Raffle @jbaruch @allthetalksconf http://jfrog.com/shownotes

The Promotion Pyramid Prod Build/Deploy time Pre-Prod Staging Integr. tests Dev Integration tests Development builds Amount of builds Amount of binaries

Pipeline: quality gates and visibility If quality requirments are hit CI SERVER 1 If quality requirments are hit 2 Integration If quality requirments are hit 3 System Testing 4 Staging Production * @jbaruch @allthetalksconf

Quality gates - http://jfrog.com/shownotes

$docker build @jbaruch @allthetalksconf http://jfrog.com/shownotes

Let’s docker build in every env! @jbaruch @allthetalksconf http://jfrog.com/shownotes

That’s why. FROM ubuntu Latest version RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www Latest version ADD app.js /var/www/app.js Latest version Latest version CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch @allthetalksconf http://jfrog.com/shownotes

That’s why. FROM ubuntu:19.04 Better now? RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www ADD app.js /var/www/app.js CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch @allthetalksconf http://jfrog.com/shownotes

That’s why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 And now? RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www ADD app.js /var/www/app.js CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch @allthetalksconf http://jfrog.com/shownotes

That’s why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www What about those? ADD app.js /var/www/app.js CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch @allthetalksconf http://jfrog.com/shownotes

That’s why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN mvn clean install What about this? CMD ”java –jar Main.class” @jbaruch @allthetalksconf http://jfrog.com/shownotes

That’s why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN download_random_sh*t_from_the_internet.sh And how about this? CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch @allthetalksconf http://jfrog.com/shownotes

That’s why you don’t trust Docker @jbaruch @allthetalksconf http://jfrog.com/shownotes

What’s up with the gates?! - QA shouldn’t test dev images - non-tested images shouldn’t be staged - non-staged, non-tested or dev images shouldn’t end up in production!!! @jbaruch @allthetalksconf http://jfrog.com/shownotes

Let’s build Rock-solid pipeline!

How do I separate dev from prod?! @jbaruch @allthetalksconf http://jfrog.com/shownotes

Option 1: metadata tags @jbaruch @allthetalksconf http://jfrog.com/shownotes

Option 2: Docker Repositories

Separate registries per environment If quality requirments are hit CI SERVER 1 If quality requirments are hit 2 Integration If quality requirments are hit 3 System Testing 4 Staging Production * @jbaruch @allthetalksconf

Quality gates - http://jfrog.com/shownotes

Trumped-up limitations @jbaruch @allthetalksconf http://jfrog.com/shownotes

The Anatomy of Docker Tag @jbaruch @allthetalksconf http://jfrog.com/shownotes

Wait a second, how can I have more than one registry per host now?! @jbaruch @allthetalksconf http://jfrog.com/shownotes

How can we support this? https://host:8081/registry/docker-dev/busybox https://host:8081/registry/docker-qa/busybox https://host:8081/registry/docker-staging/busybox https://host:8081/registry/docker-prod/busybox @jbaruch @allthetalksconf http://jfrog.com/shownotes

“ONE REGISTRY PER HOST OUGHT TO BE ENOUGH FOR ANYBODY.”

Panic! @jbaruch @allthetalksconf http://jfrog.com/shownotes

Virtual hosts/ports to the rescue docker tag host:port/busybox Registry host Tag name https://host:port/v2/busybox https://host:8081/registry/docker-dev/busybox Context name Registry name Tag name

server { listen 5001; } server_name 192.168.99.100; if ($http_x_forwarded_proto = ”) { set $http_x_forwarded_proto $scheme; } rewrite ^/(v1|v2)/(.*) /artifactory/api/docker/docker-dev/$1/$2; … } @jbaruch @allthetalksconf http://jfrog.com/shownotes

But then you realize… Wait a second, now I need to pull, retag and push for every step?! @jbaruch @allthetalksconf http://jfrog.com/shownotes

@jbaruch @allthetalksconf http://jfrog.com/shownotes

dev cluster test cluster staging cluster prod cluster @jbaruch @allthetalksconf http://jfrog.com/shownotes

Repository (docker): Top level directory in a registry Repository (the rest of the world): A registry @jbaruch @allthetalksconf http://jfrog.com/shownotes

Win-win-win

Single point of access to multiple registries when needed - Completely isolated environments - Immediate and free promotions

@jbaruch @allthetalksconf http://jfrog.com/shownotes

Win-win

Simplicity of latest - Always know what it really means - As long as you promoted immutable artifact @jbaruch @allthetalksconf http://jfrog.com/shownotes

But what about the rest of the dependencies?

@jbaruch @allthetalksconf http://jfrog.com/shownotes

Own your dependencies

Your base image - Your infra - Your application files @jbaruch @allthetalksconf http://jfrog.com/shownotes

conclusions

Build only once - Separate environments - Promote what you’ve built - Own your dependencies @jbaruch @allthetalksconf http://jfrog.com/shownotes

Q&A and Links Ø@jbaruch Ø@allthetalksconf Øhttp://jfrog.com/shownotes