A presentation at DeveloperWeek Global by Baruch Sadogursky
Best Practices In Implementing Container Image Promotion Pipelines
Software I like Software I know really well
đ© @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes @ErinMeyerINSEADâs âCulture Mapâ
shownotes Ăhttp://jfrog.com/shownotes Ă Slides Ă Video Ă Links Ă Comments, Ratings Ă Raffle @jbaruch #DeveloperWeek http://jfrog.com/shownotes
The Promotion Pyramid Prod Build/Deploy time Pre-Prod Staging Integr. tests Dev Integration tests Development builds Amount of builds Amount of binaries
Pipeline: quality gates and visibility If quality requirments are hit CI SERVER 1 If quality requirments are hit 2 Integration If quality requirments are hit 3 System Testing 4 Staging Production * @jbaruch #DeveloperWeek
$docker build @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Letâs docker build in every env! @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Thatâs why. FROM ubuntu Latest version RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www Latest version ADD app.js /var/www/app.js Latest version Latest version CMD [â/usr/bin/nodeâ, â/var/www/app.jsâ] @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Thatâs why. FROM ubuntu:19.04 Better now? RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www ADD app.js /var/www/app.js CMD [â/usr/bin/nodeâ, â/var/www/app.jsâ] @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Thatâs why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 And now? RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www ADD app.js /var/www/app.js CMD [â/usr/bin/nodeâ, â/var/www/app.jsâ] @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Thatâs why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www What about those? ADD app.js /var/www/app.js CMD [â/usr/bin/nodeâ, â/var/www/app.jsâ] @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Thatâs why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN mvn clean install What about this? CMD âjava âjar Main.classâ @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Thatâs why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN download_random_sh*t_from_the_internet.sh And how about this? CMD [â/usr/bin/nodeâ, â/var/www/app.jsâ] @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Thatâs why you donât trust Docker @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Whatâs up with the gates?! - QA shouldnât test dev images - non-tested images shouldnât be staged - non-staged, non-tested or dev images shouldnât end up in production!!! @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Letâs build Rock-solid pipeline!
How do I separate dev from prod?! @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Option 1: metadata tags @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Option 2: Docker Repositories
Separate registries per environment If quality requirments are hit CI SERVER 1 If quality requirments are hit 2 Integration If quality requirments are hit 3 System Testing 4 Staging Production * @jbaruch #DeveloperWeek
Trumped-up limitations @jbaruch #DeveloperWeek http://jfrog.com/shownotes
The Anatomy of Docker Tag @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Wait a second, how can I have more than one registry per host now?! @jbaruch #DeveloperWeek http://jfrog.com/shownotes
How can we support this? https://host:8081/registry/docker-dev/busybox https://host:8081/registry/docker-qa/busybox https://host:8081/registry/docker-staging/busybox https://host:8081/registry/docker-prod/busybox @jbaruch #DeveloperWeek http://jfrog.com/shownotes
âONE REGISTRY PER HOST OUGHT TO BE ENOUGH FOR ANYBODY.â
Panic! @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Virtual hosts/ports to the rescue docker tag host:port/busybox Registry host @jbaruch Tag name #DeveloperWeek http://jfrog.com/shownotes
Virtual hosts/ports to the rescue docker tag host:port/busybox Registry host Tag name https://host:port/v2/busybox @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Virtual hosts/ports to the rescue docker tag host:port/busybox Registry host Tag name https://host:port/v2/busybox https://host:8081/registry/docker-dev/busybox Context name Registry name Tag name
server { listen 5001; } server_name 192.168.99.100; if ($http_x_forwarded_proto = â) { set $http_x_forwarded_proto $scheme; } rewrite ^/(v1|v2)/(.*) /artifactory/api/docker/docker-dev/$1/$2; ⊠} @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Letâs abuse things! @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Letâs abuse things! @jbaruch #DeveloperWeek http://jfrog.com/shownotes
But then you realize⊠Wait a second, now I need to pull, retag and push for every step?! @jbaruch #DeveloperWeek http://jfrog.com/shownotes
@jbaruch #DeveloperWeek http://jfrog.com/shownotes
dev cluster test cluster staging cluster prod cluster @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Repository (docker): Top level directory in a registry Repository (the rest of the world): A registry @jbaruch #DeveloperWeek http://jfrog.com/shownotes
Win-win-win
@jbaruch #DeveloperWeek http://jfrog.com/shownotes
Win-win
But what about the rest of the dependencies?
@jbaruch #DeveloperWeek http://jfrog.com/shownotes
Own your dependencies
conclusions
Q&A and Links Ă@jbaruch Ă#DeveloperWeek Ăhttp://jfrog.com/shownotes
Surprisingly, implementing a secure, robust and fast promotion pipelines for container images is not as easy as it might sound. Automating dependency resolution (base images), implementing multiple registries for different maturity stages and making sure that we actually run in production containers from the images we intended can be tricky. In this talk, we will compare different approaches, compile a wish-list of features and create a pipeline that checks all the boxes using free and open-source tools.