Best Practices In Implementing Container Image Promotion Pipelines

A presentation at Docker Atlanta Meetup July 2020 in July 2020 in Atlanta, GA, USA by Baruch Sadogursky

Slide 1

Slide 1

Best Practices In Implementing Container Image Promotion Pipelines

Slide 2

Slide 2

Slide 3

Slide 3

Software I like Software I know really well

Slide 4

Slide 4

Slide 5

Slide 5

Slide 6

Slide 6

đŸŽ© @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes @ErinMeyerINSEAD’s “Culture Map”

Slide 7

Slide 7

shownotes Øhttp://jfrog.com/shownotes Ø Slides Ø Video Ø Links Ø Comments, Ratings Ø Raffle @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 8

Slide 8

Slide 9

Slide 9

Slide 10

Slide 10

The Promotion Pyramid Prod Build/Deploy time Pre-Prod Staging Integr. tests Dev Integration tests Development builds Amount of builds Amount of binaries

Slide 11

Slide 11

Pipeline: quality gates and visibility If quality requirments are hit CI SERVER 1 If quality requirments are hit 2 Integration If quality requirments are hit 3 System Testing 4 Staging Production * @jbaruch @DockerAtlanta

  • Quality gates - http://jfrog.com/shownotes

Slide 12

Slide 12

$docker build @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 13

Slide 13

Slide 14

Slide 14

Let’s docker build in every env! @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 15

Slide 15

Slide 16

Slide 16

That’s why. FROM ubuntu Latest version RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www Latest version ADD app.js /var/www/app.js Latest version Latest version CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 17

Slide 17

That’s why. FROM ubuntu:19.04 Better now? RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www ADD app.js /var/www/app.js CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 18

Slide 18

That’s why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 And now? RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www ADD app.js /var/www/app.js CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 19

Slide 19

That’s why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www What about those? ADD app.js /var/www/app.js CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 20

Slide 20

That’s why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN mvn clean install What about this? CMD ”java –jar Main.class” @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 21

Slide 21

That’s why. FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN download_random_sh*t_from_the_internet.sh And how about this? CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 22

Slide 22

That’s why you don’t trust Docker @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 23

Slide 23

Slide 24

Slide 24

Slide 25

Slide 25

What’s up with the gates?! - QA shouldn’t test dev images - non-tested images shouldn’t be staged - non-staged, non-tested or dev images shouldn’t end up in production!!! @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 26

Slide 26

Let’s build Rock-solid pipeline!

Slide 27

Slide 27

How do I separate dev from prod?! @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 28

Slide 28

Option 1: metadata tags @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 29

Slide 29

Slide 30

Slide 30

Option 2: Docker Repositories

Slide 31

Slide 31

Slide 32

Slide 32

Separate registries per environment If quality requirments are hit CI SERVER 1 If quality requirments are hit 2 Integration If quality requirments are hit 3 System Testing 4 Staging Production * @jbaruch @DockerAtlanta

  • Quality gates - http://jfrog.com/shownotes

Slide 33

Slide 33

Slide 34

Slide 34

Trumped-up limitations @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 35

Slide 35

The Anatomy of Docker Tag @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 36

Slide 36

Wait a second, how can I have more than one registry per host now?! @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 37

Slide 37

How can we support this? https://host:8081/registry/docker-dev/busybox https://host:8081/registry/docker-qa/busybox https://host:8081/registry/docker-staging/busybox https://host:8081/registry/docker-prod/busybox @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 38

Slide 38

“ONE REGISTRY PER HOST OUGHT TO BE ENOUGH FOR ANYBODY.”

Slide 39

Slide 39

Panic! @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 40

Slide 40

Virtual hosts/ports to the rescue docker tag host:port/busybox Registry host @jbaruch Tag name @DockerAtlanta http://jfrog.com/shownotes

Slide 41

Slide 41

Virtual hosts/ports to the rescue docker tag host:port/busybox Registry host Tag name https://host:port/v2/busybox @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 42

Slide 42

Virtual hosts/ports to the rescue docker tag host:port/busybox Registry host Tag name https://host:port/v2/busybox https://host:8081/registry/docker-dev/busybox Context name Registry name Tag name

Slide 43

Slide 43

server { listen 5001; } server_name 192.168.99.100; if ($http_x_forwarded_proto = ”) { set $http_x_forwarded_proto $scheme; } rewrite ^/(v1|v2)/(.*) /artifactory/api/docker/docker-dev/$1/$2; 
 } @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 44

Slide 44

Slide 45

Slide 45

Let’s abuse things! @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 46

Slide 46

Let’s abuse things! @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 47

Slide 47

But then you realize
 Wait a second, now I need to pull, retag and push for every step?! @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 48

Slide 48

@jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 49

Slide 49

dev cluster test cluster staging cluster prod cluster @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 50

Slide 50

Slide 51

Slide 51

Repository (docker): Top level directory in a registry Repository (the rest of the world): A registry @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 52

Slide 52

Win-win-win

  • Single point of access to multiple registries when needed - Completely isolated environments - Immediate and free promotions

Slide 53

Slide 53

Slide 54

Slide 54

@jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 55

Slide 55

Win-win

  • Simplicity of latest - Always know what it really means - As long as you promoted immutable artifact @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 56

Slide 56

But what about the rest of the dependencies?

Slide 57

Slide 57

@jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 58

Slide 58

Slide 59

Slide 59

Own your dependencies

  • Your base image - Your infra - Your application files @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 60

Slide 60

conclusions

  • Build only once - Separate environments - Promote what you’ve built - Own your dependencies @jbaruch @DockerAtlanta http://jfrog.com/shownotes

Slide 61

Slide 61

Q&A and Links Ø@jbaruch Ø@DockerAtlanta Øhttp://jfrog.com/shownotes