Surviving the Techpocalypse: Empowering Survivors with Developer Productivity Engineering
A presentation at DevOps Days Tel Aviv 2024 in in Tel Aviv-Yafo, Israel by Baruch Sadogursky
4 9 5 , 9 6 5
From cost world to throughp ut world Cost Throughput Throughput Inventory Inventory Cost
In terms of IT company Throughput Cost Sales a.k.a money Throughput Inventory Code Inventory Cost Org cost
The only initiatives that will positively impact performance are ones that increase throughput while simultaneously decreasing cost
CUTING COSTS (LAYOFFS) WITHOUT INCREASING THROUPUT BY MOVING INVENTORY (CREATING AND SELLING MORE SOFTWARE) IS USELESS
But what about silent quitting ?
But what about silent quitting ?
You’d expect selective layoffs th en?
Baruch Sadogursky - @jbaru ch × Developer Productivity Advocate × Gradle Inc × Development -> DevOps -> #DPE
shownotes × × × × speaking.jbaru.ch Slides Video All the links!
Human resources Attrition - I quit - That’s a bummer! ”The leadership takes full responsibility” 🤡 – I quit - Good riddance
- You’re fired
16
17
Layoff Survivor Sickness
21
22
25
Don’t ruin the flow
“The build takes forever, I am distracted to do other things and the context switch is terrible”
Don’t frustrate the developers
“we have a flanky test, it is irrelevant 99.5% of the time , but it always runs and it is last in the suite”
Don’t boil the frog
“I have a feeling that everything is slower somehow…”
Developer Productivity == A/M Autonomy /P == motivation Tools and people aren’t in my way Mastery Tools and processes help me to excel Purpose I want to be productive, i.e. create the product
Developer productivity Enginee ring!
Developer Productivity Enginee Foster Faster Feedback Collaborate through Effective Tooling Eliminate Toil for Developers Dedicated Organizational Mindset ring Embrace Rigorous Observability for Proactive Improvement Prioritize Automation and Eliminate Bottlenecks Outcomes Over Output
Talk is cheap, show me the goods! 41
Small DPE improvements make a × × × × × huge difference Generate code faster: Better IDE Test better: Testcontainers Enforce better code: Sonar Test more reliably: Flaky test detection Foster Faster Feedback:
feedback efficiency × × × × IDE: Sub-seconds (I type, it marks it red) Build: Seconds CI: Minutes Production: Hours/Days
Reverse dependency on distance from developers IDE Build CI Production Feedback Time Faster Slower Distance from Developers Expected Real
It is slow!
It is slow and the developers h ave no idea why!
What is build? × × × × × Project setup Downloading the Internet Artifact generation: Compilation, packaging, etc Tests Artifact deployment
What can go wrong? × × × × × Project setup Downloading the Internet Artifact generation: Compilation, packaging, etc Tests Artifact deployment
When can it go wrong? At any time.
The Build frustrates the developers
Let’s ask Chatgpt
What the actual f*ck?! × Skipping tests defeats the purpose of the build! × How about skipping compilation? × We want faster feedback, not less feedback
What feedback do we want?
Ci/cd pipeline quality gates It compiles Basic Integration tests Basic Unit tests Basic Quality Linting Quality Code coverage Dependency scanning SAST/DAST Quality Sec Static code analysis Sec Sec Secrets scanning Resource Utilization Nonfunc Nonfunc Load Testing Nonfunc Compliance
Two types of feedback x e.g., CI/CD x we never wait for it x results are distracting x e.g., build x we’ll wait for it in the flow x we’ll be pissed off when it’s slow
Reverse dependency on distance from developers Faster IDE Build CI Feedback Time Commit time synchronous asynchronous Slower Distance from Developers Production
Ideal build time feedback It compiles Basic Integration tests Basic Unit tests Basic Quality Linting Quality Code coverage Dependency scanning SAST/DAST Quality Sec Static code analysis Sec Sec Secrets scanning Resource Utilization Nonfunc Nonfunc Load Testing Nonfunc Compliance
Delightful build (pick two): ☑ provides max feedback ☑ fast
Skip what can be skipped (but no more!)
Avoidance: Incremental build × Don’t build what didn’t changed × Don’t build what isn’t affected
Avoidance: Incremental build s hortcomings × Relies on produced artifacts × Relies on architectural decisions
Avoidance: Caching × × × × Makes the build faster Makes the build faster for everybody Makes the build faster always Makes all parts of the build faster
Avoidance: Predictive test selec tion × Learns code changes effects de-facto × Skips tests with high degree of confidence
How test prediction works × Code changes and test results are thrown into learning model × After a while, the model predicts which changes fail which tests
Test prediction What changed Where it changed Correlate with observed test failures Predictions which changes will fail which tests
Black magic in action × The more tests a project has, the less they break × Refactorings in Java break tests less than in JavaScript
Speed up what can’t be skipped
Test parallelization × Use max power of local machine × (Yes, your boss should buy you the bleeding edge)
Test distribution × × × × CI uses fan-out to speed-up tests Shouldn’t you enjoy it for local tests? Use the cloud to distribute test load RUN ALL THE TESTS!
Why not just using ci fan-out? × Relying on shared CI infrastructure × CI infrastructure is not optimized for real-time feedback! × Are the agents as fast as they can be?
Don’t let it slide
Observe and improve × Measure local build times across time and environments × Detect downfacing trends × Find root causes and improve
The gains are real!
But DPE is not my job!
But Devops is not my job!
Devops vs DPE The product More productive work On everybody’s minds Slips away Organizational culture Specialized team
The best DPE organizations build focused teams
Fight downsizing survivor gu ilt with DPE × It improves morale × It covers productivity gaps × It’s the right thing to do for both the employees and the business × But it requires an organizational mindset and a dedicated effort!
Learn more and try it today! × × × × Take the Gradle/Maven Speed Challenge! Be DPE Agent of Change! Read the DPE Handbook! Watch the DPE Summit videos! x speaking.jbaru.ch
In these perplexing times, when jobs vanish like ice cream on a hot summer day, tech professionals must find the sweet spot between concern for their affected colleagues and the pressing need to scoop up progress.
Discover Developer Productivity Engineering (DPE), the secret sauce that helps both individuals and companies navigate the labyrinth of layoffs and limited resources with flair. Join us as we trek through a landscape teeming with dispirited engineers and floundering companies, showcasing DPE’s power to transform even the most disenchanted developer into a productivity maestro.
By adopting DPE’s clever approach, we can face economic instability with unity and a dash of wit, turning sour lemons into a refreshing lemonade of success.
Resources
The following resources were mentioned during the presentation or are useful additional information.
-
Tech layoffs tracker
-
The Phoenix Project
-
The Goal: A Process of Ongoing Improvement by Eliyahu Goldratt
-
Gallup: U.S. Engagment Hits 11-Year Low
-
Baruch’s Book Liquid Software
-
Baruch’s Book DevOps Tools for Java Developers
-
Healing the Wounds by David M. Noer
-
BizReport 2022 Layoff Aftermath Survey
-
Don’t Expect Layoff Survivors To Be Grateful by Leadership IQ
-
Drive: The Surprising Truth About What Motivites Us by Daniel Pink
-
Speed Challenge (Win Prizes!)
-
Gradle BuildScan®
-
Gradle Build Cache (Local and Remote)
-
Parallel local tests execution
-
Test Distribution (Parallel cloud test execution)
-
Trends, Reporting and Insights
-
Failure Analytics (Flaky Tests Mitigation)
-
Predictive Test Selection Feature in Develocity
-
Spring Framework Project PDT Simulations
-
Predictive Test Selection Paper from Meta
-
Developer Productivity Engineering (DPE) Handbook
-
DPE Summit
