#DataDrivenDevOps

#DataDrivenDevOps ”Without data you are just another person with an opinion” W. Edwards Deming @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

shownotes http://jfrog.com/shownotes Slides Video Links Comments, Ratings Raffle @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Cobbler’s children have no shoes

Poll time! How many of you are software engineers? How many of you are optimists? How many of you are self-confident in their work? @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Dunning-Kruger Effect a.k.a. ”optimism” “ People suffer from illusory superiority, mistakenly assessing their cognitive ability as greater than it is. Wikipedia @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Second-system effect a.k.a. “self-confidence” “ The tendency of small, elegant, and successful systems, to be succeeded by overengineered, bloated systems, due to inflated expectations and overconfidence. The Mythical Man-Month @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

The consequences @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

disillusion Software I like @jbaruch #DataDrivenDevOps Software I know really well #PureAccelerate http://jfrog.com/shownotes

overconfidence “ Every software is 80% ready 80% of the time Every developer, ever @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Hard to give up @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

@jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

How departments measure Support Marketing Sales HR Finance @jbaruch #DataDrivenDevOps Engineering Anecdotes from daily standup #PureAccelerate http://jfrog.com/shownotes

@jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

🎩 @jbaruch #dockercon jfrog.com/shownotes @ErinMeyerINSEAD’s “Culture Map”

Engineering metrics in pre-devops era Ops Dev • Velocity • Architecture • Compliance @jbaruch • SLA • Cost env setup • Average customer cost • Tools stability #DataDrivenDevOps #PureAccelerate • • • • • QA Incidents Defects MTTR Code coverage Test suite stability http://jfrog.com/shownotes

DEVOPS! ? @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Velocity to agile is like ? To devops… ? @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Velocity to agile is like Flow Efficiency To devops… Flow Efficiency @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Very much alike! Metric Flow Efficiency Velocity @jbaruch Easily understandable Unity actionable ✓ ✓ ✗ ✓ ✓ ✗ #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Poll time! Who knows what velocity is? Who knows what burndown chart is? Who has a burndown chart? Who looks at the burndown chart? Who trusts the burndown chart? Who knows what to do if it doesn’t look right?

What’s the problem? Velocity @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

True for both! Metric Flow Efficiency Velocity @jbaruch Easily understandable Unity actionable ✓ ✓ ✗ ✓ ✓ ✗ #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Too coarse! Flow Efficiency @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Too coarse! DevOps @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

This is devops* Ops Deep specialization Dev *Unless you’re netfliX **You’re not @jbaruch #DataDrivenDevOps Common goals, tools, culture QA #PureAccelerate http://jfrog.com/shownotes

This is devops Ops Dev @jbaruch #DataDrivenDevOps QA #PureAccelerate http://jfrog.com/shownotes

Same for metrics! Ops Specialized metrics Dev @jbaruch #DataDrivenDevOps Flow Efficiency QA #PureAccelerate http://jfrog.com/shownotes

Same for metrics! Ops Dev @jbaruch #DataDrivenDevOps QA #PureAccelerate http://jfrog.com/shownotes

How do metrics collaborate? influenced affected dev dev qa • Time to build • False reopen count • Release pipelines • QA coverage stability • Artifact replication topology ops • qa • • @jbaruch ops Avg cost of customer • Smoke tests quality Test suite stability • Time to create new Incoming defect QA lab velocity #PureAccelerate http://jfrog.com/shownotes • #DataDrivenDevOps Test execution time

Cooking with metrics @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Metrics categorization @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Why? @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

ROI – is this activity is the right thing to do?

Continuous improvement

trust

sampling @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Complete sample

@jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Representative sample

How effective are we? Samples are good enough for that As long as they are representative Need to be collected over time @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Predicting the future

Meet scrum team “avengers”

anecdotes from the daily scrum stand-ups “ We never get enough testing environments from Ops, QA is suffering. Scrum team “Avengers” @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Let’s gather some data! Environment Utilization 10 8 6 4 2 0 week 1 week 2 week 3 week 4 week 5 week 6 week 7 Utilized @jbaruch #DataDrivenDevOps week 8 week 9 week 10 week 11 week 12 week 13 week 14 Available #PureAccelerate http://jfrog.com/shownotes

That doesn’t look good! But why? Days the environment stays checked out 25 25 20 18 15 13 13 John Scott 14 15 16 10 10 6 5 1 2 7 3 0 Raj Jane Christina Helen Lee Baruch Nataly Ruppert Josh Patrick Michael Days checked out @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Environment Utilization 16 14 12 Continuous improvement 10 8 6 4 2 0 week 1 week 3 week 5 week 7 week 9 week 11 week 13 week 15 week 17 week 19 week 21 week 23 Utilized @jbaruch #DataDrivenDevOps #PureAccelerate Available http://jfrog.com/shownotes

anecdotes from the daily scrum stand-ups “ Tests are failing, because the test suite is not stable, QA makes our builds fail without reason. Scrum team “Avengers” @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Let’s gather some data!

Return on investment @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

anecdotes from the daily scrum stand-ups “ Those developers want to deploy new GC, its tons of work, without any profit. Scrum team “Avengers” @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Let’s gather some data! @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

0:00 0:40 1:20 2:00 2:40 3:20 4:00 4:40 5:20 6:00 6:40 7:20 8:00 8:40 9:20 10:00 10:40 11:20 12:00 12:40 13:20 14:00 14:40 15:20 16:00 16:40 17:20 18:00 18:40 19:20 20:00 20:40 21:20 22:00 22:40 23:20 GC time Trust and predictability GC time @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Dos and Don’ts

Measuring the right thing

LoC LOL, anyone? @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Avoid data overload

Use reliable data

”medium” In brazil ”medium” In russia Common vocabulary is important

Metrics generate incentives, be aware! @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Tweak as you go

Elevate! Integrated Dashboards Product Specific Dashboards Raw Data @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

@jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

JFROG ARTIFACTORY AT PURE STORAGE (FB) 2019 Artifactory Ent. (VM) Artifactory Ent. (VM) 2020 Artifactory Ent. (VM) Artifactory Ent. (VM) HA 130TB RepositoryShared DBRepository Artifactory Pro (VM) Artifactory Pro (VM) RepositoryShared DBRepository 66 © 2019 PURE STORAGE INC. DB Repository 2017 Artifactory Pro (VM) Artifactory Pro (VM) DB Repository RepositoryDB 2016

HYBRID CLOUD ARCHITECTURE WITH JFROG ARTIFACTORY ON PURE STORAGE - PROPOSED Scenario #1 Scenario #2 Extreme Deduplication Bucket Replication 67 Flashblade © 2019 PURE STORAGE INC. ObjectEngine Flashblade

How departments measure Support Marketing Sales HR Finance @jbaruch #DataDrivenDevOps Engineering Anecdotes from daily standup #PureAccelerate http://jfrog.com/shownotes

Step into data-driven life Removes blame game Builds accountability and trust Creates common base for discussion Enables DevOps @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes

Q&A and twitter ads @jbaruch #PureAccelerate #DataDrivenDevOps http://jfrog.com/shownotes