Best Practices In Implementing Container Image Promotion Pipelines

A presentation at Developers Summit 2020 in February 2020 in Tokyo, Japan by Baruch Sadogursky

Slide 1

Slide 1

Best Practices In Implementing Container Image Promotion Pipelines コンテナイメージのプロモーションパイプライン ベストプラクティス

Slide 2

Slide 2

Slide 3

Slide 3

Poll time 会場のお客様に聞く

Slide 4

Slide 4

Dockerをご存知ですか? Familiar with Docker* *Among Docker meetup attendees *Docker Meetupの参加者の結果

Slide 5

Slide 5

気に入ってるソフトウエア Software I like よく知ってるソフトウエア Software I know really well

Slide 6

Slide 6

気に入ってるけど… 信用してはいない!

Slide 7

Slide 7

バウ・サドガスキー

Slide 8

Slide 8

🎩 日本 @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes @ErinMeyerINSEAD’s “Culture Map”

Slide 9

Slide 9

shownotes Øhttp://jfrog.com/shownotes Ø Slides Ø Video Ø Links Ø Comments, Ratings Ø Raffle @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 10

Slide 10

Slide 11

Slide 11

Slide 12

Slide 12

The Promotion Pyramid プロモーションのピラミッド Prod ビルドやデプロイに かかる時間 Build/Deploy time Pre-Prod Staging Integr. tests バイナリーの量 Dev Integration tests Amount of binaries Development builds Amount of builds ビルドの量

Slide 13

Slide 13

Pipeline: quality gates and visibility 品質を管理し可視化する If quality requirments are hit CI SERVER 1 If quality requirments are hit 2 Integration If quality requirments are hit 3 System Testing 4 Staging Production * @jbaruch #DevelopersSummit

  • Quality gates - http://jfrog.com/shownotes

Slide 14

Slide 14

$docker build @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 15

Slide 15

Slide 16

Slide 16

Let’s docker build in every env! 全部の環境でdocker buildしてしまえ! 品質要件を満たすなら 次のステージに移す ステータスがOKなら VCSコミットする 品質要件を満たさなければ そのビルドは壊れてる @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 17

Slide 17

Slide 18

Slide 18

That’s why. なぜなら… FROM ubuntu Latest version RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www Latest version ADD app.js /var/www/app.js Latest version Latest version CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 19

Slide 19

That’s why. なぜなら… FROM ubuntu:19.04 Better now? RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www ADD app.js /var/www/app.js CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 20

Slide 20

That’s why. なぜなら… FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 And now? RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www ADD app.js /var/www/app.js CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 21

Slide 21

That’s why. なぜなら… FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN apt-get install -y software-properties-common python RUN apt-get install -y nodejs RUN mkdir /var/www What about those? ADD app.js /var/www/app.js CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 22

Slide 22

That’s why. なぜなら… FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN mvn clean install What about this? CMD ”java –jar Main.class” @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 23

Slide 23

That’s why. なぜなら… FROM ubuntu:4033353383af19ec179c01dda7f355a246c6adcafaf93c8f98 RUN download_random_sh*t_from_the_internet.sh And how about this? CMD [“/usr/bin/node”, “/var/www/app.js”] @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 24

Slide 24

That’s why you don’t trust Docker @jbaruch #DevelopersSummit だからDockerはあてにならない http://jfrog.com/shownotes

Slide 25

Slide 25

Slide 26

Slide 26

Slide 27

Slide 27

What’s up with the gates?! それじゃGateはどうなるの? @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 28

Slide 28

Slide 29

Slide 29

What’s up with the gates?! いやいやコイツじゃなくて

  • QA shouldn’t test dev images - non-tested images shouldn’t be staged - non-staged, non-tested or dev images shouldn’t end up in production!!! プロモートされず、テストもされてないdev imageは本 番環境に移行させない!! @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 30

Slide 30

Let’s build Rock-solid pipeline! 鉄板のPipelineを作ってみようぜ

Slide 31

Slide 31

How do I separate dev from prod?! 開発環境と本番環境を どう切り離すのか? @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 32

Slide 32

Option 1: metadata tags @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 33

Slide 33

Slide 34

Slide 34

Option 2: Docker Repositories

Slide 35

Slide 35

Slide 36

Slide 36

Separate registries per environment 環境ごとにレジストリを分ける If quality requirments are hit CI SERVER 1 If quality requirments are hit 2 Integration If quality requirments are hit 3 System Testing 4 Staging Production * @jbaruch #DevelopersSummit

  • Quality gates - http://jfrog.com/shownotes

Slide 37

Slide 37

requirements 大切なこと… -Registry per environment -Easy access to all the images -Fast Promotion -Keep working with latest and knowing what it is 常に最新のモノを十分理解して利用する @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 38

Slide 38

The Anatomy of Docker Tag Docker Tagの構造 @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 39

Slide 39

Wait a second, how can I have more than one registry per host now?! ちょっと待ってよ! ホストごとに1つ以上の レジストリって持てる? @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 40

Slide 40

How can we support this? こうゆうのをどうサポートする? https://host:8081/registry/docker-dev/busybox https://host:8081/registry/docker-qa/busybox https://host:8081/registry/docker-staging/busybox https://host:8081/registry/docker-prod/busybox @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 41

Slide 41

Panic! パニック @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 42

Slide 42

Virtual hosts/ports to the rescue docker tag host:port/busybox Registry host Tag name https://host:port/v2/busybox https://host:8081/registry/docker-dev/busybox Context name Registry name Tag name

Slide 43

Slide 43

server { listen 5001; } server_name 192.168.99.100; if ($http_x_forwarded_proto = ”) { set $http_x_forwarded_proto $scheme; } rewrite ^/(v1|v2)/(.*) /artifactory/api/docker/docker-dev/$1/$2; … } @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 44

Slide 44

But then you realize… ここでハタと気づく.. Wait a second, now I need to pull, retag and push for every step?! ちょっと待ってよ! つまりステップ毎にプルして、 タグつけ直してプッシュするっ てこと? ありえないでちゅ… @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 45

Slide 45

@jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 46

Slide 46

dev cluster test cluster staging cluster prod cluster @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 47

Slide 47

Repository (docker): Top level directory in a registry Repository (the rest of the world): A registry @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 48

Slide 48

Slide 49

Slide 49

@jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 50

Slide 50

Win-win

  • Simplicity of latest - Always know what it really means - As long as you promoted immutable artifact 不変の配布物をプロモートする限りは @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 51

Slide 51

requirements 大切なこと… -Registry per environment -Easy access to all the images - Fast Promotion - Keep working with latest and knowing what it is @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 52

Slide 52

But what about the rest of the dependencies? でもそれ以外の依存関 係はどーするの? 依存関係は?

Slide 53

Slide 53

@jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 54

Slide 54

Slide 55

Slide 55

Own your dependencies 自分でしっかり管理する

  • Your base image - Your infra - Your application files ⾃分のアプリケーション ファイル @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 56

Slide 56

conclusions まとめ…

  • Build only once - Separate environments - Promote what you’ve built - Own your dependencies 自分の依存関係は⾃分で管理する @jbaruch #DevelopersSummit http://jfrog.com/shownotes

Slide 57

Slide 57

Q&A and Links Ø@jbaruch Ø# DevelopersSummit Øhttp://jfrog.com/shownotes

Slide 58

Slide 58

Ayana Yokota / 横田紋奈(よこな) デベロッパーアドボケイト at JFrog Twitter: @ihcomega マイブームはサウナ

Slide 59

Slide 59