Environment scripts & creation tools 3. DB scripts and reference data 4. Containers 5. Automated tests 6. Project artifacts – documentation, procedures, etc. 7. Application configuration files 8. This struggling with changes – Hard line: no changes would be accepted into GWS without accompanying automated tests. Spread Testing Grouplet team approach. Google – single, share repository of billions of files. 50% of code is changed each month. 40K code commits/day, 120K automated test suites, 75M tests cases/day e. CONTINUOUSLY BUILD, TEST, AND INTEGRATE OUR CODE AND ENVIRONMENTS i. Create automated test

every Google property d. SPREAD KNOWLEDGE BY USING AUTOMATED TESTS AS DOCUMENTATION AND COMMUNITIES OF PRACTICE i. Ensure automated tests demonstrate use and behavior of libraries and components ii Designing for fast flow, deployability, and operations will derive NFRs. Codify these into the tests and pipeline f. BUILD REUSABLE OPERATIONS USER STORIES INTO DEVELOPMENT i. Goal – make recurring issues unaddressed due to schedule and budget constraints ii. Automate as many security controls and tests as possible and incorporate into the pipeline iii. Enable fast feedback on potentially insecure

required to develop program elements. Program can leverage test infrastructure and automated tests, and testers are active throughout development. Test Environment Extensive development and operational possible. Within a sprint (e.g., one month), the development team designs, develops, integrates, and tests the software to enable the user stories selected. A guiding principle is to involve stakeholders Testers should take an active part in these discussions as well to ensure common expectations and tests of performance. In situations where primary users are not available to engage with the Agile team

the feature is ready for deployment when it passes all of its tests. But is quality a simple yes-or-no attribute? If it passes its tests, is it high quality? And how can we assess the quality of something separate the concepts of deployability and quality. A feature is simply deployable when it passes its tests; that is a yes-or-no question independent of quality.  Once the feature is in production, we can responsibility.  Focus of roles is changing: The software engineer role is increasing in importance. Tests and infrastructure are now both represented in code; with SDN, soon even the network will be. Infrastructure

David Genn (IG) @david_genn  Separate deployment from release  Automate everything  Trust your tests  Conversations are way better than project proposals  Start small, be brave quickly  Seize

built automatically using Continuous Integration, and it is tested automatically through scripted tests.  It is deployed automatically with automated deployment tools onto infrastructure that can be