information during the development process. User Story – Description of functionality a user wants, small enough to complete in a single sprint Epic – A large user story often defined for a release that spans of user stories or epics that may span multiple releases Teams often capture requirements in user stories and epics to provide a clear operational perspective of the capability’s purpose. User stories be completed. Epics are aggregations of user stories, often used to capture strategic intent. Epics may span multiple sprints and form a release, while user stories should be implemented within a single

are being achieved. iii. Logging Levels 1. Debug – anything that happens in the program 2. Info – user driven actions or system specific 3. Warn – conditions that could become an error and will likely INTEGRATING A/B TESTING INTO OUR FEATURE TESTING i. Random subset of users shown differing versions. User groups define a cohort. Statistical analysis of cohort’s outcomes from differing versions used to DOING MORE MANUAL TESTING AND CHANGE FREEZES i. When testing failures occur, adding more testing could backfire, especially if the testing is at the end of a project and/or manual 1. Manual testing is

operations will derive NFRs. Codify these into the tests and pipeline f. BUILD REUSABLE OPERATIONS USER STORIES INTO DEVELOPMENT i. Goal – make recurring work as repeatable and deterministic as possible; CATEGORIZED AS NORMAL CHANGES i. If change approval board is required, aim for quick deployment even if manual steps are required ii. Use tooling to generate the artifacts needed for approval. Share evidence

to run in parallel ii. Enable different test categories to run in parallel iii. Generally, limit manual testing if automated tests haven’t passed iv. For resource intensive suites, performance tests, new and old code to make it well structured” j. AUTOMATE AS MANY OF OUR MANUAL TESTS AS POSSIBLE i. Reduce the reliance on manual testing ii. Elisabeth Hendrickson – “Although testing can be automated features, 20% on detailed planning, 25% on porting code amongst branches, 10% integrating, 15% manual testing. ii. Implemented Trunk-based development & CI iii. After CI: 40% effort on new features

Extra features: d. Task switching: e. Waiting: f. Motion: g. Defects: h. Nonstandard or manual work:i. Heroics: d. The Second Way: The Principles of Feedback 27 i. KEEP PUSHING QUALITY CLOSER ineffective quality controls include: a. Requiring another team to complete tedious, error-prone, and manual tasks that could be easily automated and run as needed by the team who needs the work performed

Principles to implementing a successful DevSecOps approach  Remove bottlenecks (including human ones) and manual actions.  Automate as much of the development and deployment activities as possible.  Adopt common

bottleneck, you have lost time that cannot be recovered.” Parallel: Automated Unit Testing before Manual Testing & QA Time and Batch sizes From the moment material comes into the plan to the minute

of buying off the shelf.  The risk of developing a system incrementally and altering it based on user feedback is often lower than that of buying a finished product that is hard to change.  The advantages product that does not fit our business and customize it until it does.  We offer our business users a user interface that is clunky because it was designed for users in the abstract across many possible companies get feedback, and then adjust what they have produced. As a result, the code can be developed in a user-centric way and match the enterprise’s needs precisely. Risk is low, because the team is constantly

must maintain a big picture, long-term view and focus on the next six-month release.  DO epics and user stories to concisely define the desired system functions and provide the foundation for Agile estimation estimation and planning. o They describe what the users want to accomplish with the resulting system. User stories help ensure that users, acquirers, developers, testers, and other stakeholders have a clear is established to help estimate the level of work for each requirement (i.e., as translated into user stories)

the-heck-do-you-start  The myth of the single metric  Four roles (CEO, Operations, Developer, User)  Sunburst chart how do visitors move through my siteLinux Performance Tools by Brendan Gregg (Netflix) process, entropy  Actively discourage email: it is exclusive Where there any key themes?  Real User Monitoring: more valuable than any other metric  Web Performance: 2 seconds is the magic number