Defining Nonfunctional Requirements

Reference 9 in Chapter 2: Metastable Failures in the Wild

Lexiang Huang, Matthew Magnusson, Abishek Bangalore Muralikrishna and 5 others

16th USENIX Symposium on Operating Systems Design and Implementation 2022

Reference 10 in Chapter 2: Exponential Backoff and Jitter

Marc Brooker

2015

aws.amazon.com

Reference 11 in Chapter 2: What Is Backoff For?

Marc Brooker

2022

Reference 12 in Chapter 2: Release It!

Michael T. Nygard

2018

Reference 13 in Chapter 2: Slowing Down to Speed Up—Circuit Breakers for Slack’s CI/CD

Frank Chen

2022

slack.engineering

Reference 14 in Chapter 2: Fixing Retries with Token Buckets and Circuit Breakers

Marc Brooker

2022

Reference 15 in Chapter 2: Using Load Shedding to Avoid Overload

David Yanacek

aws.amazon.com

Reference 16 in Chapter 2: Pushing Back

Matthew Sackman

2016

wellquite.org

Reference 17 in Chapter 2: Meet Bandaid, the Dropbox Service Proxy

Dmitry Kopytkov and Patrick Lee

2018

dropbox.tech

Reference 18 in Chapter 2: Fail-Slow at Scale: Evidence of Hardware Performance Faults in Large Production Systems

Haryadi S. Gunawi, Riza O. Suminto, Russell Sears and 18 others

16th USENIX Conference on File and Storage Technologies 2018

Reference 19 in Chapter 2: Is the Mean Really Useless?

Marc Brooker

2017

orangematter.solarwinds.com

Reference 20 in Chapter 2: Dynamo: Amazon's Highly Available Key-Value Store

Giuseppe DeCandia, Deniz Hastorun, Madan Jampani and 6 others

21st ACM Symposium on Operating Systems Principles 2007

allthingsdistributed.com PDF doi

Reference 21 in Chapter 2: The Need for Speed, 23 Years Later

Kathryn Whitenton

2020

nngroup.com

Reference 22 in Chapter 2: Marissa Mayer at Web 2.0

Greg Linden

2005

glinden.blogspot.com

Reference 23 in Chapter 2: Speed Matters for Google Web Search

Jake Brutlag

2009

services.google.com PDF

Reference 24 in Chapter 2: Performance Related Changes and Their User Impact

Eric Schurman and Jake Brutlag

Velocity 2009 2009

youtube.com

Reference 25 in Chapter 2: The State of Online Retail Performance

Akamai Technologies, Inc

akamai.com 2017

web.archive.org PDF

Reference 26 in Chapter 2: Understanding and Leveraging the Impact of Response Latency on User Behaviour in Web Search

Xiao Bai, Ioannis Arapakis, B. Barla Cambazoglu and Ana Freire

ACM Transactions on Information Systems 2018

hdl.handle.net doi

Reference 27 in Chapter 2: The Tail at Scale

Jeffrey Dean and Luiz André Barroso

Communications of the ACM 2013

cacm.acm.org doi

Reference 28 in Chapter 2: Implementing Service Level Objectives: A Practical Guide to SLIs, SLOs, and Error Budgets

Alex Hidalgo

2020

oreilly.com

Reference 29 in Chapter 2: Nines Are Not Enough: Meaningful Metrics for Clouds

Jeffrey C. Mogul and John Wilkes

17th Workshop on Hot Topics in Operating Systems 2019

research.google doi

Reference 30 in Chapter 2: Meaningful Availability

Tamás Hauer, Philipp Hoffmann, John Lunney and 2 others

17th USENIX Symposium on Networked Systems Design and Implementation 2020

usenix.org

Reference 31 in Chapter 2: HdrHistogram: A High Dynamic Range Histogram

Gil Tene

hdrhistogram.github.io/HdrHistogram

hdrhistogram.github.io

Reference 32 in Chapter 2: The t-digest: Efficient Estimates of Distributions

Ted Dunning

Software Impacts 2021

sciencedirect.com doi

Reference 33 in Chapter 2: How Percentile Approximation Works (and Why It's More Useful than Averages)

David Kohn

2021

timescale.com

Reference 34 in Chapter 2: Circllhist—A Log-Linear Histogram Data Structure for IT Infrastructure Monitoring

Heinrich Hartmann and Theo Schlossnagle

arXiv 2020

arxiv.org PDF

Reference 35 in Chapter 2: DDSketch: A Fast and Fully-Mergeable Quantile Sketch with Relative-Error Guarantees

Charles Masson, Jee E. Rim and Homin K. Lee

Proceedings of the VLDB Endowment 2019

vldb.org PDF doi

Reference 36 in Chapter 2: Why Percentiles Don’t Work the Way You Think

Baron Schwartz

solarwinds.com 2016

Reference 37 in Chapter 2: A Conceptual Framework for System Fault Tolerance

Walter L. Heimerdinger and Charles B. Weinstock

1992

resources.sei.cmu.edu PDF

Reference 38 in Chapter 2: Fundamentals of Fault-Tolerant Distributed Computing in Asynchronous Environments

Felix C. Gärtner

ACM Computing Surveys 1999

dl.acm.org doi

Reference 39 in Chapter 2: Basic Concepts and Taxonomy of Dependable and Secure Computing

Algirdas Avižienis, Jean-Claude Laprie, Brian Randell and Carl Landwehr

IEEE Transactions on Dependable and Secure Computing 2004

hdl.handle.net gone doi

Reference 40 in Chapter 2: Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive Systems

Ding Yuan, Yu Luo, Xin Zhuang and 5 others

11th USENIX Symposium on Operating Systems Design and Implementation 2014

Reference 41 in Chapter 2: Chaos Engineering

Casey Rosenthal and Nora Jones

2020

Reference 42 in Chapter 2: Failure Trends in a Large Disk Drive Population

Eduardo Pinheiro, Wolf-Dietrich Weber and Luiz Andre Barroso

5th USENIX Conference on File and Storage Technologies 2007

Reference 43 in Chapter 2: Disk Failures in the Real World: What Does an Mttf of 1,000,000 Hours Mean to You?

Bianca Schroeder and Garth A. Gibson

5th USENIX Conference on File and Storage Technologies 2007

Reference 44 in Chapter 2: Backblaze Drive Stats for Q2 2021

Andy Klein

2021

backblaze.com

Reference 45 in Chapter 2: SSD Failures in Datacenters: What? When? And Why?

Iyswarya Narayanan, Di Wang, Myeongjae Jeon and 7 others

9th ACM International on Systems and Storage Conference 2016

microsoft.com PDF doi

Reference 46 in Chapter 2: Storage System Design Analysis: Factors Affecting NVMe SSD Performance (1)

Alibaba Cloud Storage Team

2019

alibabacloud.com

Reference 47 in Chapter 2: Flash Reliability in Production: The Expected and the Unexpected

Bianca Schroeder, Raghav Lagisetty and Arif Merchant

14th USENIX Conference on File and Storage Technologies 2016

Reference 48 in Chapter 2: SSD Failures in the Field: Symptoms, Causes, and Prediction Models

Jacob Alter, Ji Xue, Alma Dimnaku and Evgenia Smirni

International Conference for High Performance Computing, Networking, Storage and Analysis 2019

dl.acm.org doi

Reference 49 in Chapter 2: Availability in Globally Distributed Storage Systems

Daniel Ford, François Labelle, Florentina I. Popovici and 5 others

9th USENIX Symposium on Operating Systems Design and Implementation 2010

Reference 50 in Chapter 2: Characterizing Cloud Computing Hardware Reliability

Kashi Venkatesh Vishwanath and Nachiappan Nagappan

1st ACM Symposium on Cloud Computing 2010

microsoft.com PDF doi

Reference 51 in Chapter 2: Cores That Don't Count

Peter H. Hochschild, Paul Turner, Jeffrey C. Mogul and 4 others

Workshop on Hot Topics in Operating Systems 2021

sigops.org PDF doi

Reference 52 in Chapter 2: Silent Data Corruptions at Scale

Harish Dattatraya Dixit, Sneha Pendharkar, Matt Beadon and 4 others

arXiv 2021

arxiv.org

Reference 53 in Chapter 2: Scalable Error Isolation for Distributed Systems

Diogo Behrens, Marco Serafini, Sergei Arnautov and 2 others

12th USENIX Symposium on Networked Systems Design and Implementation 2015

usenix.org

Reference 54 in Chapter 2: DRAM Errors in the Wild: A Large-Scale Field Study

Bianca Schroeder, Eduardo Pinheiro and Wolf-Dietrich Weber

11th International Joint Conference on Measurement and Modeling of Computer Systems 2009

static.googleusercontent.com PDF doi

Reference 55 in Chapter 2: Flipping Bits in Memory Without Accessing Them: An Experimental Study of DRAM Disturbance Errors

Yoongu Kim, Ross Daly, Jeremie Kim and 6 others

41st Annual International Symposium on Computer Architecture 2014

users.ece.cmu.edu PDF doi

Reference 56 in Chapter 2: Worst Case

Tim Bray

2021

tbray.org

Reference 57 in Chapter 2: Solar Superstorms: Planning for an Internet Apocalypse

Sangeetha Abdu Jyothi

ACM SIGCOMM Conference 2021

ics.uci.edu PDF doi

Reference 58 in Chapter 2: Failure Modes and Continuous Resilience

Adrian Cockcroft

2019

adrianco.medium.com

Reference 59 in Chapter 2: An In-Depth Study of Correlated Failures in Production SSD-Based Data Centers

Shujie Han, Patrick P. C. Lee, Fan Xu and 3 others

19th USENIX Conference on File and Storage Technologies 2021

usenix.org

Reference 60 in Chapter 2: Cycles, Cells and Platters: An Empirical Analysis of Hardware Failures on a Million Consumer PCs

Edmund B. Nightingale, John R. Douceur and Vince Orgovan

6th European Conference on Computer Systems 2011

eurosys2011.cs.uni-salzburg.at PDF doi

Reference 61 in Chapter 2: What Bugs Live in the Cloud? A Study of 3000+ Issues in Cloud Systems

Haryadi S. Gunawi, Mingzhe Hao, Tanakorn Leesatapornwongsa and 8 others

5th ACM Symposium on Cloud Computing 2014

ucare.cs.uchicago.edu PDF gone doi

Reference 62 in Chapter 2: Getting Real About Distributed System Reliability

Jay Kreps

2012

blog.empathybox.com

Reference 63 in Chapter 2: Leap Second Crashes Half the Internet

Nelson Minar

2012

somebits.com

Reference 64 in Chapter 2: Support Alerts—Customer Bulletin a00092491en_us

Hewlett Packard Enterprise

2019

support.hpe.com

Reference 65 in Chapter 2: Awesome Limits

Lorin Hochstein

2020

github.com

Reference 66 in Chapter 2: Clients Are Jerks: AKA How Halo 4 DoSed the Services at Launch & How We Survived

Caitie McCaffrey

2015

caitiem.com

Reference 67 in Chapter 2: Fail Through the Cracks: Cross-System Interaction Failures in Modern Cloud Systems

Lilia Tang, Chaitanya Bhandari, Yongle Zhang and 4 others

18th European Conference on Computer Systems 2023

tianyin.github.io PDF doi

Reference 68 in Chapter 2: Addressing Cascading Failures

Mike Ulrich

Site Reliability Engineering: How Google Runs Production Systems 2016

sre.google

Reference 69 in Chapter 2: Cascading Failures in Large-Scale Distributed Systems

Harri Faßbender

2022

blog.mi.hdm-stuttgart.de

Reference 70 in Chapter 2: How Complex Systems Fail

Richard I. Cook

2000

adaptivecapacitylabs.com PDF

Reference 71 in Chapter 2: STELLA: Report from the SNAFUcatchers Workshop on Coping with Complexity

David D. Woods

2017

snafucatchers.github.io

Reference 72 in Chapter 2: Why Do Internet Services Fail, and What Can Be Done About It?

David Oppenheimer, Archana Ganapathi and David A. Patterson

4th USENIX Symposium on Internet Technologies and Systems 2003

static.usenix.org PDF

Reference 73 in Chapter 2: The Field Guide to Understanding "Human Error"*, 3rd edition

Sidney Dekker

2017

Reference 74 in Chapter 2: Drift into Failure: From Hunting Broken Components to Understanding Complex Systems

Sidney Dekker

2011

taylorfrancis.com

Reference 75 in Chapter 2: Blameless PostMortems and a Just Culture

John Allspaw

2012

etsy.com

Reference 76 in Chapter 2: Uptime Guarantees—A Pragmatic Perspective

Itzy Sabo

2023

world.hey.com

Reference 77 in Chapter 2: The Human Impact of Bugs

Michael Jurewitz

2013

jury.me

Reference 78 in Chapter 2: How Software Bugs Led to ‘One of the Greatest Miscarriages of Justice’ in British History

Mark Halper

Communications of the ACM 2025

cacm.acm.org doi

Reference 79 in Chapter 2: The Legal Rule That Computers Are Presumed to be Operating Correctly—Unforeseen and Unjust Consequences

Nicholas Bohm, James Christie, Peter Bernard Ladkin and 7 others

2022

benthamsgaze.org PDF

Reference 80 in Chapter 2: Choose Boring Technology

Dan McKinley

2015

mcfunley.com

Reference 81 in Chapter 2: Building and Operating a Pretty Big Storage System Called S3

Andy Warfield

2023

allthingsdistributed.com

Reference 82 in Chapter 2: Surprising Scalability of Multitenancy

Marc Brooker

2023

Reference 83 in Chapter 2: Shared Nothing vs. Shared Disk Architectures: An Independent View

Ben Stopford

2009

benstopford.com

Reference 84 in Chapter 2: The Case for Shared Nothing

Michael Stonebraker

IEEE Database Engineering Bulletin 1986

dsf.berkeley.edu PDF

Reference 85 in Chapter 2: Socrates: The New SQL Server in the Cloud

Panagiotis Antonopoulos, Alex Budovski, Cristian Diaconu and 14 others

ACM International Conference on Management of Data 2019

microsoft.com PDF doi

Reference 86 in Chapter 2: Building Microservices*, 2nd edition

Sam Newman

2021

oreilly.com

Reference 87 in Chapter 2: When Good Software Goes Bad: The Surprising Durability of an Ephemeral Technology

Nathan Ensmenger

The Maintainers Conference 2016

themaintainers.wpengine.com PDF

Reference 88 in Chapter 2: Facts and Fallacies of Software Engineering

Robert L. Glass

2002

Reference 89 in Chapter 2: Kill It with Fire

Marianne Bellotti

2021

Reference 90 in Chapter 2: Ironies of Automation

Lisanne Bainbridge

Automatica 1983

adaptivecapacitylabs.com PDF doi

Reference 91 in Chapter 2: On Designing and Deploying Internet-Scale Services

James Hamilton

21st Large Installation System Administration Conference 2007

Reference 92 in Chapter 2: Open Source for Better Observability

Dotan Horovits

2021

horovits.medium.com

Reference 93 in Chapter 2: Big Ball of Mud

Brian Foote and Joseph Yoder

4th Conference on Pattern Languages of Programs 1997

laputan.org PDF

Reference 94 in Chapter 2: What Is a Simple System?

Marc Brooker

2022

Reference 95 in Chapter 2: No Silver Bullet—Essence and Accident in Software Engineering

Frederick P. Brooks

The Mythical Man-Month 1995

worrydream.com PDF

Reference 96 in Chapter 2: Against Essential and Accidental Complexity

Dan Luu

2020

danluu.com

Reference 97 in Chapter 2: Design Patterns: Elements of Reusable Object-Oriented Software

Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides

1994

Reference 98 in Chapter 2: Domain-Driven Design: Tackling Complexity in the Heart of Software

Eric Evans

2003