#DataDrivenDevOps
A presentation at Pure//Accelerate 2019 in in Austin, TX, USA by Baruch Sadogursky
#DataDrivenDevOps ”Without data you are just another person with an opinion” W. Edwards Deming @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
shownotes http://jfrog.com/shownotes Slides Video Links Comments, Ratings Raffle @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Cobbler’s children have no shoes
Poll time! How many of you are software engineers? How many of you are optimists? How many of you are self-confident in their work? @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Dunning-Kruger Effect a.k.a. ”optimism” “ People suffer from illusory superiority, mistakenly assessing their cognitive ability as greater than it is. Wikipedia @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Second-system effect a.k.a. “self-confidence” “ The tendency of small, elegant, and successful systems, to be succeeded by overengineered, bloated systems, due to inflated expectations and overconfidence. The Mythical Man-Month @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
The consequences @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
disillusion Software I like @jbaruch #DataDrivenDevOps Software I know really well #PureAccelerate http://jfrog.com/shownotes
overconfidence “ Every software is 80% ready 80% of the time Every developer, ever @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Hard to give up @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
@jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
How departments measure Support Marketing Sales HR Finance @jbaruch #DataDrivenDevOps Engineering Anecdotes from daily standup #PureAccelerate http://jfrog.com/shownotes
How departments measure Support Marketing Sales HR Finance @jbaruch #DataDrivenDevOps Engineering Anecdotes from daily standup #PureAccelerate http://jfrog.com/shownotes
@jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
@jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
🎩 @jbaruch #dockercon jfrog.com/shownotes @ErinMeyerINSEAD’s “Culture Map”
Engineering metrics in pre-devops era Ops Dev • Velocity • Architecture • Compliance @jbaruch • SLA • Cost env setup • Average customer cost • Tools stability #DataDrivenDevOps #PureAccelerate • • • • • QA Incidents Defects MTTR Code coverage Test suite stability http://jfrog.com/shownotes
DEVOPS! ? @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Velocity to agile is like ? To devops… ? @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Velocity to agile is like Flow Efficiency To devops… Flow Efficiency @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Very much alike! Metric Flow Efficiency Velocity @jbaruch Easily understandable Unity actionable ✓ ✓ ✗ ✓ ✓ ✗ #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Poll time! Who knows what velocity is? Who knows what burndown chart is? Who has a burndown chart? Who looks at the burndown chart? Who trusts the burndown chart? Who knows what to do if it doesn’t look right?
What’s the problem? Velocity @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
True for both! Metric Flow Efficiency Velocity @jbaruch Easily understandable Unity actionable ✓ ✓ ✗ ✓ ✓ ✗ #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Too coarse! Flow Efficiency @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Too coarse! DevOps @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
This is devops* Ops Deep specialization Dev *Unless you’re netfliX **You’re not @jbaruch #DataDrivenDevOps Common goals, tools, culture QA #PureAccelerate http://jfrog.com/shownotes
This is devops Ops Dev @jbaruch #DataDrivenDevOps QA #PureAccelerate http://jfrog.com/shownotes
Same for metrics! Ops Specialized metrics Dev @jbaruch #DataDrivenDevOps Flow Efficiency QA #PureAccelerate http://jfrog.com/shownotes
Same for metrics! Ops Dev @jbaruch #DataDrivenDevOps QA #PureAccelerate http://jfrog.com/shownotes
How do metrics collaborate? influenced affected dev dev qa • Time to build • False reopen count • Release pipelines • QA coverage stability • Artifact replication topology ops • qa • • @jbaruch ops Avg cost of customer • Smoke tests quality Test suite stability • Time to create new Incoming defect QA lab velocity #PureAccelerate http://jfrog.com/shownotes • #DataDrivenDevOps Test execution time
Cooking with metrics @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Metrics categorization @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Why? @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
ROI – is this activity is the right thing to do?
Continuous improvement
trust
sampling @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Complete sample
@jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Representative sample
How effective are we? Samples are good enough for that As long as they are representative Need to be collected over time @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Predicting the future
Meet scrum team “avengers”
anecdotes from the daily scrum stand-ups “ We never get enough testing environments from Ops, QA is suffering. Scrum team “Avengers” @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Let’s gather some data! Environment Utilization 10 8 6 4 2 0 week 1 week 2 week 3 week 4 week 5 week 6 week 7 Utilized @jbaruch #DataDrivenDevOps week 8 week 9 week 10 week 11 week 12 week 13 week 14 Available #PureAccelerate http://jfrog.com/shownotes
That doesn’t look good! But why? Days the environment stays checked out 25 25 20 18 15 13 13 John Scott 14 15 16 10 10 6 5 1 2 7 3 0 Raj Jane Christina Helen Lee Baruch Nataly Ruppert Josh Patrick Michael Days checked out @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Environment Utilization 16 14 12 Continuous improvement 10 8 6 4 2 0 week 1 week 3 week 5 week 7 week 9 week 11 week 13 week 15 week 17 week 19 week 21 week 23 Utilized @jbaruch #DataDrivenDevOps #PureAccelerate Available http://jfrog.com/shownotes
anecdotes from the daily scrum stand-ups “ Tests are failing, because the test suite is not stable, QA makes our builds fail without reason. Scrum team “Avengers” @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Let’s gather some data!
Return on investment @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
anecdotes from the daily scrum stand-ups “ Those developers want to deploy new GC, its tons of work, without any profit. Scrum team “Avengers” @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Let’s gather some data! @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
0:00 0:40 1:20 2:00 2:40 3:20 4:00 4:40 5:20 6:00 6:40 7:20 8:00 8:40 9:20 10:00 10:40 11:20 12:00 12:40 13:20 14:00 14:40 15:20 16:00 16:40 17:20 18:00 18:40 19:20 20:00 20:40 21:20 22:00 22:40 23:20 GC time Trust and predictability GC time @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Dos and Don’ts
Measuring the right thing
LoC LOL, anyone? @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Avoid data overload
Use reliable data
”medium” In brazil ”medium” In russia Common vocabulary is important
Metrics generate incentives, be aware! @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Tweak as you go
Elevate! Integrated Dashboards Product Specific Dashboards Raw Data @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
@jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
JFROG ARTIFACTORY AT PURE STORAGE (FB) 2019 Artifactory Ent. (VM) Artifactory Ent. (VM) 2020 Artifactory Ent. (VM) Artifactory Ent. (VM) HA 130TB RepositoryShared DBRepository Artifactory Pro (VM) Artifactory Pro (VM) RepositoryShared DBRepository 66 © 2019 PURE STORAGE INC. DB Repository 2017 Artifactory Pro (VM) Artifactory Pro (VM) DB Repository RepositoryDB 2016
HYBRID CLOUD ARCHITECTURE WITH JFROG ARTIFACTORY ON PURE STORAGE - PROPOSED Scenario #1 Scenario #2 Extreme Deduplication Bucket Replication 67 Flashblade © 2019 PURE STORAGE INC. ObjectEngine Flashblade
How departments measure Support Marketing Sales HR Finance @jbaruch #DataDrivenDevOps Engineering Anecdotes from daily standup #PureAccelerate http://jfrog.com/shownotes
Step into data-driven life Removes blame game Builds accountability and trust Creates common base for discussion Enables DevOps @jbaruch #DataDrivenDevOps #PureAccelerate http://jfrog.com/shownotes
Q&A and twitter ads @jbaruch #PureAccelerate #DataDrivenDevOps http://jfrog.com/shownotes
“Without data, you’re just another person with opinions”.
In this talk, we’ll talk about data-driven DevOps and how the cross-cutting metrics from dev, QA, and ops can be integrated to provide you and the teams you support with an insight into the status of your engineering organization. As the DevOps Evangelists of your organization, you can help your teams to adopt data-driven decision making whereas it becomes more important due to cross-pillar influence and collaborated need for success. The practical aspect will cover dos and don’ts and examples of metrics that you can implement in to help your teams today.
