Baruch Sadogursky - @jbaru ch × Developer Productivity Advocate × Gradle Inc × Development -> DevOps -> #DPE
Eli Aleyner - @ealeyner × × × × Co-founder AtomicJar Inc Founding team @AWS Founding team @Bing.com
shownotes × × × × speaking.jbaru.ch Slides Video All the links!
Don’t ruin the flow
“The build takes forever, I am distracted to do other things and the context switch is terrible”
Don’t frustrate the developers
“we have a flanky test, it is irrelevant 99.5% of the time , but it always runs and it is last in the suite”
Don’t boil the frog
“I have a feeling that everything is slower somehow…”
Developer Productivity == A/M Autonomy /P == motivation Tools and people aren’t in my way Mastery Tools and processes help me to excel Purpose I want to be productive, i.e. create the product
Developer productivity Enginee ring!
Developer Productivity Enginee Foster Faster Feedback Collaborate through Effective Tooling Eliminate Toil for Developers Dedicated Organizational Mindset ring Embrace Rigorous Observability for Proactive Improvement Prioritize Automation and Eliminate Bottlenecks Outcomes Over Output
Talk is cheap, show me the goods! 22
Small DPE improvements make a × × × × × huge difference Generate code faster: Better IDE Test better: Testcontainers Enforce better code: Sonar Test more reliably: Flaky test detection Foster Faster Feedback:
feedback efficiency × × × × IDE: Sub-seconds (I type, it marks it red) Build: Seconds CI: Minutes Production: Hours/Days
Reverse dependency on distance from developers IDE Build CI Feedback Time Faster Slower Distance from Developers Expected Production
Reverse dependency on distance from developers IDE Build CI Production Feedback Time Faster Slower Distance from Developers Expected Real
It is slow!
It is slow and the developers h ave no idea why!
What is build? × × × × × Project setup Downloading the Internet Artifact generation: Compilation, packaging, etc Tests Artifact deployment
What can go wrong? × × × × × Project setup Downloading the Internet Artifact generation: Compilation, packaging, etc Tests Artifact deployment
When can it go wrong? At any time.
The Build frustrates the developers
Let’s ask Chatgpt
What the actual f*ck?! × Skipping tests defeats the purpose of the build! × How about skipping compilation? × We want faster feedback, not less feedback
What feedback do we want?
Ci/cd pipeline quality gates It compiles Basic Integration tests Basic Unit tests Basic Quality Linting Quality Code coverage Dependency scanning SAST/DAST Quality Sec Static code analysis Sec Sec Secrets scanning Resource Utilization Nonfunc Nonfunc Load Testing Nonfunc Compliance
Two types of feedback x e.g., CI/CD x we never wait for it x results are distracting x e.g., build x we’ll wait for it in the flow x we’ll be pissed off when it’s slow
Reverse dependency on distance from developers Faster IDE Build CI Feedback Time Commit time synchronous asynchronous Slower Distance from Developers Production
Ideal build time feedback It compiles Basic Integration tests Basic Unit tests Basic Quality Linting Quality Code coverage Dependency scanning SAST/DAST Quality Sec Static code analysis Sec Sec Secrets scanning Resource Utilization Nonfunc Nonfunc Load Testing Nonfunc Compliance
Delightful build (pick two): ☑ provides max feedback ☑ fast
Skip what can be skipped (but no more!)
Avoidance: Incremental build × Don’t build what didn’t changed × Don’t build what isn’t affected
Avoidance: Incremental build s hortcomings × Relies on produced artifacts × Relies on architectural desicions
Avoidance: Caching × × × × Makes the build faster Makes the build faster for everybody Makes the build faster always Makes all parts of the build faster
Avoidance: Predictive test selec tion × Learns code changes effects de-facto × Skips tests with high degree of confidence
Speed up what can’t be skipped
Test parallelization × Use max power of local machine × (Yes, your boss should buy you the bleeding edge)
Test distribution × × × × CI uses fan-out to speed-up tests Shouldn’t you enjoy it for local tests? Use the cloud to distribute test load RUN ALL THE TESTS!
Why not just using ci fan-out ? × Relying on shared CI infrastructure × CI infrastructure is not optimized for real-time feedback! × Are the agents as fast as they can be?
Don’t let it slide
Observe and improve × Measure local build times across time and environments × Detect downfacing trends × Find root causes and improve
Same tradeoff for tests expensive, slow, but prod-like Or naïve, useless, but fast
CI often depends on Staging environments But fixing staging environments is just “faster horses” Value of Staging Environment tests degrades over time as they become longer to run, provide flaky outcomes, while still sitting too far from Software Development Lifecycle project maturity Cost of maintaining Staging Environment based tests increases as projects mature project maturity
Modern Infrastructure Has Made Testing Harder For Developers
The Testcontainers Experience Unit tests with real dependencies
Unit Tests with Real Dependencies Testcontainers is an open source framework for providing throwaway, lightweight instances of databases, message brokers, web browsers, or just about anything that can run in a Docker container.
Testcontainers is an open source framework for providing throwaway, lightweight instances of test dependencies. GenericContainer redis = new GenericContainer(“redis:5.0.3-alpine”) .withExposedPorts(6379)
Testcontainers is used by
Test against any database, message broker, browser… or just about anything that runs in a Docker container!
Testcontainers libraries exist for all popular languages including Java, Go, .NET, NodeJS, and more. Java Python Go .NET Node.js Rust Haskell Ruby
Simple Local Development with Real Dependencies Testcontainers Desktop is the free companion app to open source Testcontainers libraries.
Testcontainers Desktop The free companion app. Debug your Testcontainerspowered dependencies Track and analyze your test sessions Switch container runtimes and burst to the cloud The app lets you proxy any service to a fixed port to easily inspect it with your favorite debugging tools. Dashboards that provide you and your team with insights into your development and testing patterns. With Testcontainers Cloud, you can even run them in the cloud on demand, while saving your local resources.
Testcontainers Cloud: Test without limits. Ship with confidence. Developer-first Testing Effortlessly Fast CI Reliable Test Suites Test everything on your laptop without worrying about resources; no local docker daemon needed Run your ever-growing test suite without scaling your CI, and speed it up by running tests in parallel Enhance team efficiency by getting rid of flaky tests and ensuring consistency from dev to CI
How it works
The gains are real!
Learn more and try it today! × × × × Take the Gradle/Maven Speed Challenge! Be DPE Agent of Change! Read the DPE Handbook! Watch the DPE Summit videos! x speaking.jbaru.ch