Refactoring to modules: Why and how – all you need to know in (less than) an hour
A presentation at GopherCon IL 2019 in in Tel Aviv-Yafo, Israel by Baruch Sadogursky
Refactoring to modules: Why and how – all you need to know in (less than) an hour @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
shownotes http://jfrog.com/shownotes Slides Video Links Comments, Ratings Raffle @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
The speakers @jbaruch @jbaruch @eyalbe4 #GopherConIL @eyalbe4 http://jfrog.com/shownotes
Let’s go back in time
Poll time! Pre 1.0 (2012) 1.2 (2013) 1.5 (2015) 1.8 (2017) 1.11 (2019) @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Why we have a problem? @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Why we have a problem? @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Simple solution! Dependencies are sources Remote import is a VCS path Dump everything together into one source tree (GOPATH) Compile Profit @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
@jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
But… how do i… Know which dependencies do I use? Know which dependencies did you use? Know which dependencies should I use? Know is it our code that I am editing right now? WTF is going on?! @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Yeah… “ To date, we’ve resorted to an email semaphore whenever someone fixes a bug a package, imploring everyone else to run go get -u. You can probably imagine how successful this is, and how much time is being spent chasing bugs that were already fixed. Dave Cheney @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Duplicate your dependencies “ Check your dependencies to your own VCS. Brad Firzpatrick @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Build your own dependency manager “ It’s not the role of the tooling provided by the language to dictate how you manage your code in the production sense. Andrew Gerrand @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
We expect you to already have a homegrown dependency manager “ If you need to build any tooling around what Go uses (Git, Mercurial, Bazaar), you already understand those tools, so it should be straightforward to integrate with whatever system you have. Andrew Gerrand @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Don’t trust what we’ve built “ go-get is nice for playing around, but if you do something serious, like deploying to production, your deploy script now involves fetching some random dude’s stuff on GitHub. Brad Firzpatrick @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
quiz time! godeps.json dependencies.tsv govendor, govend, goven, gv trash, garbage, rubbish Weapons manufacturer @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
GOPATH + VENDORING = ! @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
GOPATH, The proud son of the monorepo
Two huge problems with gopath It only allows a single version of any given package to exist at once (per GOPATH) We cannot programmatically differentiate between code the user is working on and code they merely depend on @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
vendoring “ Copy all of the files at some version from one version control repository and paste them into a different version control repository @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
What’s wrong with it (well, what’s not) History, branch, and tag information is lost Pulling updates is impossible It invites modification, divergence, and bad fork It wastes space Good luck finding which version of the code you forked @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Still wrong! You still have no idea what version are you using You have to connect each dependency as a submodule manually Switching branches and forks LOL Working on modules with other teams ROFL @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
@jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
The go dep experiment
@jbaruch #dockercon jfrog.com/shownotes
Proper dependency management? Working in project directories Local cache for dependencies Version declarations Conflict resolution @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Conflict on the conflict resolution SAT/SMT vs MVS/SIV
Enter Go modules
Enter go modules @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Backwards compatibility and migration go mod init go.mod is created The rest is the same: imports in code just work @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
That’s some serious dark magic… @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Go modules convert everything (almost?) @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
What happens to go.mod when you add import (and run go get/go build) Go checks the URL: If it’s Go Proxy (module repository), it gets the module If it’s a VCS it clones and builds the module locally If it’s a web page, looks for go-import meta tag Selects the latest compatible version tag Semantic import versioning @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
@jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Compatible?! Let’s assume SemVer works (LOL) The latest version of v1.x.x is compatible with v1.0.0 and up Premise: import path string should always be backwards compatible @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
What about version 2?! Incompatible code can’t use the same import path Add /v2/ to the module path Use /v2/ in the import path import “github.com/my/module/v2/mypkg” @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
What if it doesn’t have any semver tags?! Pseudo version v0.0.0-yyyymmddhhmmss-abcdefabcdef @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
What if (when) I want to ban a version?! You can specify “version X or later”: >= x.y.z You can use exclude or replace for better control @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
From vendoring to hierarchy of module repositories
Go modules define an hierarchy of caches Public Modules Repository GoCenter Organizational Modules Repository Project Athens JFrog Artifactory Local cache on the developer’s machine $GOPATH/pkg/mod
Local cache on the developer’s machine After the mods are resolved (or built) they are cached in $GOPATH/pkg/mod Provides immediate access Not shared Not reliable (can be wiped at any moment) @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Organizational modules repository JFrog Artifactory or Athens Provides faster (Intranet) access Provides reproducible builds as it caches the dependencies used once for build reproduction Requires team infrastructure and maintenance (SaaS offers exist) @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
public modules repositories GoCenter Google announced a vision for a federation of public repositories Provides fast access Provides reproducible builds as it caches the popular and requested dependencies from version control Highly available, requires no maintenance, free @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Demo time! @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Twitter ads and q&A jfrog.com/shownotes @eyalbe4 @jbaruch #GopherConIL @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes
Go modules are here to stay and it’s about time to start modularizing your code. We’re here to help! In this session, you’ll learn why and how to do it by examining a production code successfully refactored to modules.
In this talk, we’ll examine the refactoring of the JFrog CLI project to modules. We’ll start by introducing modules – why and how, will talk about the benefits and the downsides of using modules and the difference between modules and go-dep. Next, we’ll review the changes switching to modules require and will finish up by reviewing the real-world application, before and after.
