Surviving the Techpocalypse: Empowering Survivors with Developer Productivity Engineering

A presentation at UberConf 2024 in July 2024 in Westminster, CO, USA by Baruch Sadogursky

Slide 1

Slide 1

Slide 2

Slide 2

Slide 3

Slide 3

9 1 8 , 5 3 5

Slide 4

Slide 4

Slide 5

Slide 5

Slide 6

Slide 6

From cost world to throughp ut world Cost Throughput Throughput Inventory Inventory Cost

Slide 7

Slide 7

In terms of IT company Throughput Cost Sales a.k.a money Throughput Inventory Code Inventory Cost Org cost

Slide 8

Slide 8

The only initiatives that will positively impact performance are ones that increase throughput while simultaneously decreasing cost

Slide 9

Slide 9

CUTING COSTS (LAYOFFS) WITHOUT INCREASING THROUPUT BY MOVING INVENTORY (CREATING AND SELLING MORE SOFTWARE) IS USELESS

Slide 10

Slide 10

But what about silent quitting ?

Slide 11

Slide 11

But what about silent quitting ?

Slide 12

Slide 12

You’d expect selective layoffs th en?

Slide 13

Slide 13

Baruch Sadogursky - @jbaru ch × Developer Productivity Advocate × Gradle Inc × Development -> DevOps -> #DPE

Slide 14

Slide 14

shownotes × × × × speaking.jbaru.ch Slides Video All the links!

Slide 15

Slide 15

Human resources Attrition - I quit - That’s a bummer! ”The leadership takes full responsibility” 🤡 – I quit - Good riddance

  • You’re fired

Slide 16

Slide 16

16

Slide 17

Slide 17

17

Slide 18

Slide 18

Slide 19

Slide 19

Layoff Survivor Sickness

Slide 20

Slide 20

Slide 21

Slide 21

21

Slide 22

Slide 22

22

Slide 23

Slide 23

Slide 24

Slide 24

Slide 25

Slide 25

25

Slide 26

Slide 26

Slide 27

Slide 27

Slide 28

Slide 28

Slide 29

Slide 29

Don’t ruin the flow

Slide 30

Slide 30

“The build takes forever, I am distracted to do other things and the context switch is terrible”

Slide 31

Slide 31

Slide 32

Slide 32

Don’t frustrate the developers

Slide 33

Slide 33

“we have a flanky test, it is irrelevant 99.5% of the time , but it always runs and it is last in the suite”

Slide 34

Slide 34

Slide 35

Slide 35

Don’t boil the frog

Slide 36

Slide 36

“I have a feeling that everything is slower somehow…”

Slide 37

Slide 37

Slide 38

Slide 38

Developer Productivity == A/M Autonomy /P == motivation Tools and people aren’t in my way Mastery Tools and processes help me to excel Purpose I want to be productive, i.e. create the product

Slide 39

Slide 39

Developer productivity Enginee ring!

Slide 40

Slide 40

Developer Productivity Enginee Foster Faster Feedback Collaborate through Effective Tooling Eliminate Toil for Developers Dedicated Organizational Mindset ring Embrace Rigorous Observability for Proactive Improvement Prioritize Automation and Eliminate Bottlenecks Outcomes Over Output

Slide 41

Slide 41

Talk is cheap, show me the goods! 41

Slide 42

Slide 42

Small DPE improvements make a × × × × × huge difference Generate code faster: Better IDE Test better: Testcontainers Enforce better code: Sonar Test more reliably: Flaky test detection Foster Faster Feedback:

Slide 43

Slide 43

feedback efficiency × × × × IDE: Sub-seconds (I type, it marks it red) Build: Seconds CI: Minutes Production: Hours/Days

Slide 44

Slide 44

Reverse dependency on distance from developers IDE Build CI Production Feedback Time Faster Slower Distance from Developers Expected Real

Slide 45

Slide 45

Slide 46

Slide 46

It is slow!

Slide 47

Slide 47

Slide 48

Slide 48

It is slow and the developers h ave no idea why!

Slide 49

Slide 49

What is build? × × × × × Project setup Downloading the Internet Artifact generation: Compilation, packaging, etc Tests Artifact deployment

Slide 50

Slide 50

What can go wrong? × × × × × Project setup Downloading the Internet Artifact generation: Compilation, packaging, etc Tests Artifact deployment

Slide 51

Slide 51

When can it go wrong? At any time.

Slide 52

Slide 52

The Build frustrates the developers

Slide 53

Slide 53

Let’s ask Chatgpt

Slide 54

Slide 54

What the actual f*ck?! × Skipping tests defeats the purpose of the build! × How about skipping compilation? × We want faster feedback, not less feedback

Slide 55

Slide 55

What feedback do we want?

Slide 56

Slide 56

Ci/cd pipeline quality gates It compiles Basic Integration tests Basic Unit tests Basic Quality Linting Quality Code coverage Dependency scanning SAST/DAST Quality Sec Static code analysis Sec Sec Secrets scanning Resource Utilization Nonfunc Nonfunc Load Testing Nonfunc Compliance

Slide 57

Slide 57

Slide 58

Slide 58

Two types of feedback x e.g., CI/CD x we never wait for it x results are distracting x e.g., build x we’ll wait for it in the flow x we’ll be pissed off when it’s slow

Slide 59

Slide 59

Reverse dependency on distance from developers IDE Build CI Feedback Time Faster Slower Distance from Developers Production

Slide 60

Slide 60

Reverse dependency on distance from developers Faster IDE Build CI Feedback Time Commit time synchronous asynchronous Slower Distance from Developers Production

Slide 61

Slide 61

Ideal build time feedback It compiles Basic Integration tests Basic Unit tests Basic Quality Linting Quality Code coverage Dependency scanning SAST/DAST Quality Sec Static code analysis Sec Sec Secrets scanning Resource Utilization Nonfunc Nonfunc Load Testing Nonfunc Compliance

Slide 62

Slide 62

Slide 63

Slide 63

Delightful build (pick two): ☑ provides max feedback ☑ fast

Slide 64

Slide 64

Skip what can be skipped (but no more!)

Slide 65

Slide 65

Avoidance: Incremental build × Don’t build what didn’t changed × Don’t build what isn’t affected

Slide 66

Slide 66

Avoidance: Incremental build s hortcomings × Relies on produced artifacts × Relies on architectural decisions

Slide 67

Slide 67

Avoidance: Caching × × × × Makes the build faster Makes the build faster for everybody Makes the build faster always Makes all parts of the build faster

Slide 68

Slide 68

Slide 69

Slide 69

Avoidance: Predictive test selec tion × Learns code changes effects de-facto × Skips tests with high degree of confidence

Slide 70

Slide 70

How test prediction works × Code changes and test results are thrown into learning model × After a while, the model predicts which changes fail which tests

Slide 71

Slide 71

Test prediction What changed Where it changed Correlate with observed test failures Predictions which changes will fail which tests

Slide 72

Slide 72

Black magic in action × The more tests a project has, the less they break × Refactorings in Java break tests less than in JavaScript

Slide 73

Slide 73

Slide 74

Slide 74

Speed up what can’t be skipped

Slide 75

Slide 75

Test parallelization × Use max power of local machine × (Yes, your boss should buy you the bleeding edge)

Slide 76

Slide 76

Slide 77

Slide 77

Test distribution × × × × CI uses fan-out to speed-up tests Shouldn’t you enjoy it for local tests? Use the cloud to distribute test load RUN ALL THE TESTS!

Slide 78

Slide 78

Why not just using ci fan-out ? × Relying on shared CI infrastructure × CI infrastructure is not optimized for real-time feedback! × Are the agents as fast as they can be?

Slide 79

Slide 79

Don’t let it slide

Slide 80

Slide 80

Observe and improve × Measure local build times across time and environments × Detect downfacing trends × Find root causes and improve

Slide 81

Slide 81

Slide 82

Slide 82

The gains are real!

Slide 83

Slide 83

Slide 84

Slide 84

Slide 85

Slide 85

But DPE is not my job!

Slide 86

Slide 86

Slide 87

Slide 87

But Devops is not my job!

Slide 88

Slide 88

Devops vs DPE The product More productive work On everybody’s minds Slips away Organizational culture Specialized team

Slide 89

Slide 89

The best DPE organizations build focused teams

Slide 90

Slide 90

Slide 91

Slide 91

Fight downsizing survivor gu × × × × ilt with DPE It improves morale It covers productivity gaps It’s the right thing to do for both the employees and the business But it requires an organizational mindset and a dedicated effort!

Slide 92

Slide 92

Learn more and try it today! × × × × Take the Gradle/Maven Speed Challenge! Be DPE Agent of Change! Read the DPE Handbook! Watch the DPE Summit videos! x speaking.jbaru.ch

Slide 93

Slide 93