Baruch Sadogursky - @jbaru ch × Developer Productivity Advocate × Gradle Inc × Development -> DevOps -> #DPE

Eli Aleyner - @ealeyner × × × × Co-founder AtomicJar Inc Founding team @AWS Founding team @Bing.com

shownotes × × × × speaking.jbaru.ch Slides Video All the links!

Don’t ruin the flow

“The build takes forever, I am distracted to do other things and the context switch is terrible”

Don’t frustrate the developers

“we have a flanky test, it is irrelevant 99.5% of the time , but it always runs and it is last in the suite”

Don’t boil the frog

“I have a feeling that everything is slower somehow…”

Developer Productivity == A/M Autonomy /P == motivation Tools and people aren’t in my way Mastery Tools and processes help me to excel Purpose I want to be productive, i.e. create the product

Developer productivity Enginee ring!

Developer Productivity Enginee Foster Faster Feedback Collaborate through Effective Tooling Eliminate Toil for Developers Dedicated Organizational Mindset ring Embrace Rigorous Observability for Proactive Improvement Prioritize Automation and Eliminate Bottlenecks Outcomes Over Output

Talk is cheap, show me the goods! 22

Small DPE improvements make a × × × × × huge difference Generate code faster: Better IDE Test better: Testcontainers Enforce better code: Sonar Test more reliably: Flaky test detection Foster Faster Feedback:

feedback efficiency × × × × IDE: Sub-seconds (I type, it marks it red) Build: Seconds CI: Minutes Production: Hours/Days

Reverse dependency on distance from developers IDE Build CI Feedback Time Faster Slower Distance from Developers Expected Production

Reverse dependency on distance from developers IDE Build CI Production Feedback Time Faster Slower Distance from Developers Expected Real

It is slow!

It is slow and the developers h ave no idea why!

What is build? × × × × × Project setup Downloading the Internet Artifact generation: Compilation, packaging, etc Tests Artifact deployment

What can go wrong? × × × × × Project setup Downloading the Internet Artifact generation: Compilation, packaging, etc Tests Artifact deployment

When can it go wrong? At any time.

The Build frustrates the developers

Let’s ask Chatgpt

What the actual f*ck?! × Skipping tests defeats the purpose of the build! × How about skipping compilation? × We want faster feedback, not less feedback

What feedback do we want?

Ci/cd pipeline quality gates It compiles Basic Integration tests Basic Unit tests Basic Quality Linting Quality Code coverage Dependency scanning SAST/DAST Quality Sec Static code analysis Sec Sec Secrets scanning Resource Utilization Nonfunc Nonfunc Load Testing Nonfunc Compliance

Two types of feedback x e.g., CI/CD x we never wait for it x results are distracting x e.g., build x we’ll wait for it in the flow x we’ll be pissed off when it’s slow

Reverse dependency on distance from developers Faster IDE Build CI Feedback Time Commit time synchronous asynchronous Slower Distance from Developers Production

Ideal build time feedback It compiles Basic Integration tests Basic Unit tests Basic Quality Linting Quality Code coverage Dependency scanning SAST/DAST Quality Sec Static code analysis Sec Sec Secrets scanning Resource Utilization Nonfunc Nonfunc Load Testing Nonfunc Compliance

Delightful build (pick two): ☑ provides max feedback ☑ fast

Skip what can be skipped (but no more!)

Avoidance: Incremental build × Don’t build what didn’t changed × Don’t build what isn’t affected

Avoidance: Incremental build s hortcomings × Relies on produced artifacts × Relies on architectural decisions

Avoidance: Caching × × × × Makes the build faster Makes the build faster for everybody Makes the build faster always Makes all parts of the build faster

Avoidance: Predictive test selec tion × Learns code changes effects de-facto × Skips tests with high degree of confidence

How test prediction works × Code changes and test results are thrown into learning model × After a while, the model predicts which changes fail which tests

Test prediction What changed Where it changed Correlate with observed test failures Predictions which changes will fail which tests

Black magic in action × The more tests a project has, the less they break × Refactorings in Java break tests less than in JavaScript

Speed up what can’t be skipped

Test parallelization × Use max power of local machine × (Yes, your boss should buy you the bleeding edge)

Test distribution × × × × CI uses fan-out to speed-up tests Shouldn’t you enjoy it for local tests? Use the cloud to distribute test load RUN ALL THE TESTS!

Why not just using ci fan-out ? × Relying on shared CI infrastructure × CI infrastructure is not optimized for real-time feedback! × Are the agents as fast as they can be?

Don’t let it slide

Observe and improve × Measure local build times across time and environments × Detect downfacing trends × Find root causes and improve

Same tradeoff for tests expensive, slow, but prod-like Or naïve, useless, but fast

AWS S3 - Bunny! × Integration Tests – as main quality gate × Creative, Relatable test status × Sad and depressed outcome over time

Why the Depressed bunny? × Flakiness due to: × Data, Deployments, Environment × Don’t care syndrome × Self-fulfilling prophecies

Delightful Test (pick two): ☑ Prod-like ☑ Local

Self contained tests are the a nswer × Real tests (not just mocks) × No dependency on centralized resources × No interdependency of tests × Treated as code! Same life-cycle × Allow for clone-and-run

Ehe.. But how? Testcontainers is the answer

Testcontainers is used by

Test against any database, message broker, browser… or just about anything that runs in a Docker container!

Very nais!

But We can Go Further! Foster Faster Feedback Collaborate through Effective Tooling Eliminate Toil for Developers Dedicated Organizational Mindset Embrace Rigorous Observability for Proactive Improvement Prioritize Automation and Eliminate Bottlenecks Outcomes Over Output

Test Observability & Debugging Eliminate Toil for Developers Embrace Rigorous Observability for Proactive Improvement Outcomes Over Output

Testcontainers Desktop: Batter × × × × ies Included Freeze container shutdown Service configuration Enables re-usable containers Bonus: Embedded Runtime

Test Consistency , Repeatability & Scale Dedicated Organizational Mindset Prioritize Automation and Eliminate Bottlenecks Outcomes Over Output

Testcontainers Cloud: Consist × × × × ent Experience for all Apple-silicon? Windows? - No problem! Docker-in-Docker in CI? – No problem! Turbo mode – for parallelization Team and enterprise-wide visibility for your Testcontainers usage

How it works

Testcontainers is DPE: Foster Faster Feedback Collaborate through Effective Tooling Eliminate Toil for Developers Dedicated Organizational Mindset Embrace Rigorous Observability for Proactive Improvement Prioritize Automation and Eliminate Bottlenecks Outcomes Over Output

The gains are real!

Learn more and try it today! × × × × Take the Gradle/Maven Speed Challenge! Be DPE Agent of Change! Read the DPE Handbook! Watch the DPE Summit videos! x speaking.jbaru.ch