Refactoring to modules: Why and how – all you need to know in (less than) an hour
A presentation at GoLang Philly Meetup May 2019 in in Philadelphia, PA, USA by Baruch Sadogursky
Refactoring to modules: All you need to know in less than an hour @jbaruch #golang http://jfrog.com/shownotes
Let’s go back in time
Pre 1.0 (2012) 1.2 (2013) Poll time! 1.5 (2015) 1.8 (2017) 1.11 (2018) @jbaruch #golang http://jfrog.com/shownotes
🎩 @jbaruch #dockercon jfrog.com/shownotes @ErinMeyerINSEAD’s “Culture Map”
shownotes http://jfrog.com/shownotes Slides Video Links Comments, Ratings Raffle @jbaruch #golang http://jfrog.com/shownotes
Why we have a problem? @jbaruch #golang http://jfrog.com/shownotes
Why we have a problem? @jbaruch #golang http://jfrog.com/shownotes
Dependencies are sources Remote import is a VCS path Dump everything together into one source tree (GOPATH) Simple solution! Compile Profit @jbaruch #golang http://jfrog.com/shownotes
@jbaruch #golang http://jfrog.com/shownotes
Know which dependencies do I use? Know which dependencies did you use? Know which dependencies should I use? But… how do i… Know is it our code that I am editing right now? WTF is going on?! @jbaruch #golang http://jfrog.com/shownotes
Yeah… “ To date, we’ve resorted to an email semaphore whenever someone fixes a bug a package, imploring everyone else to run go get -u. You can probably imagine how successful this is, and how much time is being spent chasing bugs that were already fixed. Dave Cheney @jbaruch #golang http://jfrog.com/shownotes
Duplicate your dependencies “ Check your dependencies to your own VCS. Brad Firzpatrick @jbaruch #golang http://jfrog.com/shownotes
Build your own dependency manager “ It’s not the role of the tooling provided by the language to dictate how you manage your code in the production sense. Andrew Gerrand @jbaruch #golang http://jfrog.com/shownotes
We expect you to already have a homegrown dependency manager “ If you need to build any tooling around what Go uses (Git, Mercurial, Bazaar), you already understand those tools, so it should be straightforward to integrate with whatever system you have. Andrew Gerrand @jbaruch #golang http://jfrog.com/shownotes
Don’t trust what we’ve built “ go-get is nice for playing around, but if you do something serious, like deploying to production, your deploy script now involves fetching some random dude’s stuff on GitHub. Brad Firzpatrick @jbaruch #golang http://jfrog.com/shownotes
godeps.json dependencies.tsv govendor, govend, goven, gv quiz time! trash, garbage, rubbish Weapons manufacturer @jbaruch #golang http://jfrog.com/shownotes
GOPATH + VENDORING = 💖 @jbaruch #golang http://jfrog.com/shownotes
GOPATH, The proud son of the monorepo
It only allows a single version of any given package to exist at once (per GOPATH) Two huge problems with gopath @jbaruch We cannot programmatically differentiate between code the user is working on and code they merely depend on #golang http://jfrog.com/shownotes
Vendoring – the worst kind of forking “ Copy all of the files at some version from one version control repository and paste them into a different version control repository @jbaruch #golang http://jfrog.com/shownotes
History, branch, and tag information is lost Pulling updates is impossible It invites modification, divergence, and bad fork It wastes space Good luck finding which version of the code you forked What’s wrong with it (well, what’s not) @jbaruch #golang http://jfrog.com/shownotes
You still have no idea what version are you using You have to connect each dependency as a submodule manually Switching branches and forks LOL Working on modules with other teams ROFL Still wrong! @jbaruch #golang http://jfrog.com/shownotes
@jbaruch #golang http://jfrog.com/shownotes
The go dep experiment
@jbaruch #dockercon jfrog.com/shownotes
Working in project directories Proper dependency management? Local cache for dependencies Version declarations Conflict resolution @jbaruch #golang http://jfrog.com/shownotes
Conflict on the conflict resolution SAT/SMT vs MVS/SIV
Enter Go modules
Enter go modules @jbaruch #golang http://jfrog.com/shownotes
go mod init go.mod file is created Backwards compatibility and migration @jbaruch The rest is the same: imports in code just work #golang http://jfrog.com/shownotes
That’s some serious magic… @jbaruch #golang http://jfrog.com/shownotes
Go modules convert everything (almost?) @jbaruch #golang http://jfrog.com/shownotes
What happens to go.mod when you add import (and run go get/go build)
Have goimport? no yes Access import path as URL Follow goimport URL yes Invalid import path Serve the module no Is it a webpage ? no Is it a VCS? no Is it a module repo? yes Build the module locally Clone the sources yes
Latest compatible version tag
@jbaruch #golang http://jfrog.com/shownotes
Let’s assume SemVer works (LOL) The latest version of v1.x.x is compatible with v1.0.0 and up Compatible?! Premise: import path string should always be backwards compatible @jbaruch #golang http://jfrog.com/shownotes
] Incompatible code can’t use the same import path Add /v2/ to the module path What about version 2?! Use /v2/ in the import path import “github.com/my/module/v2/mypkg” @jbaruch #golang http://jfrog.com/shownotes
What if it doesn’t have any semver tags?! @jbaruch Pseudo version v0.0.0-yyyymmddhhmmss-abcdefabcdef #golang http://jfrog.com/shownotes
You can specify “version X or later”: >= x.y.z What if (when) I want to ban a version?! @jbaruch You can use exclude or replace for better control #golang http://jfrog.com/shownotes
From vendoring to hierarchy of module repositories
Go modules define an hierarchy of caches Public Modules Repository GoCenter Organizational Modules Repository The Athens Project JFrog Artifactory Local cache on the developer’s machine $GOPATH/pkg/mod
After the mods are resolved (or built) they are cached in $GOPATH/pkg/mod Local cache on the developer’s machine Provides immediate access Not shared Not reliable (can be wiped at any moment) @jbaruch #golang http://jfrog.com/shownotes
JFrog Artifactory or Project Athens Provides faster (Intranet) access Provides reproducible builds as it caches the dependencies used once for build reproduction Requires team infrastructure and maintenance (SaaS offers exist) Organizational modules repository @jbaruch #golang http://jfrog.com/shownotes
GoCenter Google announced a vision for a federation of public repositories Provides fast access Provides reproducible builds as it caches the popular and requested dependencies from version control Highly available, requires no maintenance, free public modules repositories @jbaruch #golang http://jfrog.com/shownotes
Twitter ads and q&A jfrog.com/shownotes @jbaruch #GoLang gocenter.io @jbaruch #golang http://jfrog.com/shownotes
Go modules are here to stay and it’s about time to start modularizing your code. We’re here to help! In this session, you’ll learn why and how to do it by examining a production code successfully refactored to modules.
In this talk, we’ll introduce modules – why and how, will talk about the benefits and the downsides of using modules and the difference between modules and go-dep.
