KPMG Powered Testing Accelerating Business change by unlocking
Transcrição
KPMG Powered Testing Accelerating Business change by unlocking
KPMG Powered Testing Accelerating Business Change Unlocking the Full Power of Test Automation Daryl Elfield and Priya Raju KPMG Testing Services 24th Feb 2016 © 2015 KPMG LLP, a UK limited liability partnership and a member firm of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity. All rights reserved. 0 Why listen - a real world case study Client experiencing significant testing challenges with their SAP implementation • Increased project delivery risk due to high defect leakage into UAT • Compromised quality due to insufficient testing early in the project lifecycle • Delayed business benefits - longer and increased number of testing cycles impacting live date Client Testing Challenges ` And getting very little value from their IBM Rational Enterprise license Client IBM Tools Utilisation – Current State Rational Quality Manager Legend Rational Team Concert Impact analysis Project management Test Planning Requirement's Management IBM Tool Currently used IBM Tool not utilized Defects Management RIT RFT Functional Testing ST SIT UAT Messages Based Testing Integration ST Testing RPT RTVS / RIT Performance Testing Service Virtualisation ST Integration Testing © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. Features currently used Features currently not used Impacted testing phase 1 What’s in it for our clients – and us? Our Solution – KPMG Powered Testing Short Term 1. 2. 3. In the Roadmap Automate regression tests using RFT Execute Regression Testing early in the SDLC – ideally during ST and SIT Run regression testing as a managed service 1. 2. 3. 4. Implement Model Based Testing to optimise UAT test case design Use Service Virtualisation to maximise the integration testing coverage and to reduce test environment costs. Introduce Component/System Performance Testing Introduce Continuous Testing Forecast Benefits to Our Client Testing costs reduced by 80% £1m annual savings Total regression testing execution time reduced from 22 to 13 weeks – almost 50% Benefits To KPMG Benefits To IBM Potential Services in next phase 1. Six months engagement with team of four consultants 2. Managed Testing Service to support execution and automation maintenance 3. Performance testing , Service virtualization and Mobile Testing Protecting existing license spend • IBM tools under utilised due to lack of awareness and technical expertise Increasing potential additional license sell • SV and performance testing in our roadmap should increase the license utilisation and potential additional sales © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 2 Agenda • Case Study • Who are we? • What’s wrong with testing? • Introducing KPMG Powered Testing • The IBM Angle • Powered Testing and DevOps • The Future © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 3 KPMG TESTING SERVICES Who are we? Accelerating Business Change Setting a new standard in testing KPMG is setting a new standard in testing. By combining quality tools, proven practices, skilled resources and broad industry knowledge, we make testing about much more than just finding defects. HowHow we doweit…do it… we do… WhatWhat we do… We enable key business outcomes Companies use our services to: • Earlydefect removal • Overall fitness of the test organisation or test programme • Current state and benchmark it against industry testing standards • Weaknesses/potential gaps and provide pragmatic remedies Cost Reduction Optimise • Test automation • Agile testing • Lifecycle QA Reduce Time-to-Market or to Implement Regulatory Change • Company, programme or project test delivery • Return on investment (ROI) from existing testing expenditure • Test efficiency and effectiveness • Voice of customer • Business acceptance Improve Customer Satisfaction KPMG Powered Testing framework is a combination of KPMG Powered Testing : powerful tools and applied methodology which our experienced resources use to automate the most timeconsuming aspects of the testing lifecycle • Removes the obstacles to successful testing • Maximises the re-use of test artefacts • Accelerates test delivery KPMG Powered Testing is based on a core set of Manage • Test programmes using KPMG’s skilled and experienced resources • Delivery risk by employing risk-based and “shift-left” testing techniques • Customer and user acceptance to promote overall business confidence • Eliminate outages Because we use KPMG Powered Testing KPMG’s industry leading software testing framework Assess • Offshore/outsourcing • Vendor consolidation Why us? Deliver • Faster business and IT change by leveraging KPMG testing accelerators • Full lifecycle testing that maximises quality and minimises risk • Sector-specific testing, aligned to business goals and priorities principles to empower testers with the means to meet the modern testing Test Case Generation & Optimisation Automating these activities can deliver at least a 25% reduction in project delivery time and cost. 25% Data Generation & Masking 25 % Environment Virtualisation Execution Our capabilities – delivering and testing assurance services globally 1000+ RESOURCES KPMG has a successful track record delivering full lifecycle testing and test assurance to more than 300 clients in 22 countries. Our global testing practice employs more than 1,000 testers in multiple sectors and geographies. ACROSS 22 COUNTRIES We offer extensive engagement experience in all phases of technical testing (system, SIT, E2E, non-functional); in defect prevention (requirements validation and verification); as well as in pre- and post go-live assurance (user acceptance, business assurance, operational trails and beta testing). OVER 300 CLIENTS With our dedicated assurance capability, KPMG is the only “Big 4” firm accredited to deliver both TMMi and CMMI maturity assessments. The experience and quality of our people underpin our capabilities as independent testing assurers. Key contacts: Daryl Elfield Nick Urry T: +44 (0) 20 7311 6330 E: [email protected] T: +44 (0) 20 7694 2330 E: [email protected] Testing Director Partner The information contained herein is of a general nature and is not intended to address the circumstances of any particular individual or entity. Although we endeavour to provide accurate and timely information, there can be no guarantee that such information is accurate as of the date it is received or that it will continue to be accurate in the future. No one should act on such information without appropriate professional advice after a thorough examination of the particular situation. © 2015 KPMG LLP, a UK limited liability partnership and a member firm of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity. All rights reserved. The KPMG name, logo and “cutting through complexity” are registered trademarks or trademarks of KPMG International. INTERNAL USE ONLY. Designed by Create Graphics | CRT036934 | July©2015 | PrintedKPMG on recycled AG material.Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative 20XX („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. www.kpmg.co.uk 4 What’s wrong with Testing? © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 5 Some uncomfortable truths about testing More than 25% of all IT projects fail to deliver a solution at all1 Of those which do complete, over 50% of development costs are consumed fixing self-inflicted problems – costing the global economy c.$312 billion per year2 In other words – a significant amount of money is being spent on an activity which is perceived to, or actually does, deliver very little value. What’s going wrong? Software testing consumes anywhere from 25 -50% of the average application lifecycle - and often is viewed as adding no business value3 1IT Project Success Rates Survey, Ambysoft 2013 University Judge Business School 2013 3Gartner Research, 2007 2Cambridge © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 6 How our clients experience testing The standard approach to testing is expensive, inefficient and often unsuccessful Requirements Test Cases Test Data Production databases is a convenient source of test data but there may be sensitive data BA Unclear requirements results in poor coverage and inefficiency Formal testing starts too late in SDLC due to environment dependencies – defects found late Test Analyst Execution It’s now a race to execute your tests before go-live. There may have to be a compromise: a) Run reduced test pack b) Push back go-live date Prod Questions answered but has the documentation been updated too? Inconsistency in capturing requirements Environments Confidence in Test Coverage? Test case maintenance disciplines? Manually sanitizing data is prone to expensive errors and time consuming X Test packs optimised? Now it’s the TA’s job to interpret the requirements and define test scenarios X Realistic data is hard to collect & maintain. So it ages and loses relevance. Third party test environments , external systems with access fee and legacy systems limits the number of parallel test execution © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. Only now does Automation get introduced. Too little, too late? Have you benefitted from the promised ROI.?.. 7 What if you could…. Create an optimised test set? Make realistic test data without relying on production? Automate test cases before coding completes? Impossible? Eliminate test environment constraints? Do all of this automatically, on any platform? © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 8 Introducing KPMG Powered Testing © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 9 Introducing KPMG Powered Testing KPMG Powered Testing is KPMG’s industry-leading software testing framework, designed to automate the entire testing lifecycle Test Design KPMG Powered Testing is a combination of specialst tools, applied methodology and domain accelerators which our experienced resources use to automate the most time-consuming aspects of the testing lifecycle Automated Optimized Unambiguous Maintainable Test Data Secure Sanitized Manageable Accessible Test Envs KPMG Powered Testing is designed to meet the challenges of modern testing Always available Integrated early Cost effective Realistic Test Execution KPMG Powered Testing : Data-driven Automated Repeatable Traceable • Removes the obstacles to successful testing • Maximises the re-use of test artefacts • Accelerates test delivery Powered Testing delivers at least a 25% reduction in project delivery time and cost © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 10 How our clients experience Powered Testing Powered Testing removes the bottlenecks to delivering rapid, efficient testing Requirements Test Cases Environments Test Data Execution Powered Testing automates every step of the testing lifecycle. Leverage powerful production data subsetting and masking tools to automatically find and desensitise data Service virtualization removes dependency on 3rd party apps/environments Improved communication is embedded into the test design process Prod Test Analyst BA Nexus accelerates automated test script generation by Model Based Test Case Design automates the process of test case creation from business process models X • providing an easy to use interface to the underlying automation functions • Re-using test scripts and test data from earlier in the testing lifecycle to create automated test packs Dynamic test data set increases test coverage and reduces maintenance overhead. Effective impact analysis Optimised Test Coverage Model based requirements captured to remove ambiguity and improve quality Agility to respond to requirement changes ALM Tools Our Domain Accelerators mean testers are not starting from scratch Or generate synthetic data for full test coverage Increased negative testing and edge cases Sophisticated dashboards and proxies enable effective management of virtual services across test environments at enterprise level © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. Our Domain Accelerators include application specific automation packs 11 What makes Powered Testing unique? KPMG Powered Testing is a unique combination of powerful tools, applied methodology and domain accelerators Tools Methodology Domain accelerators Domain Accelerators KPMG have leveraged their significant industry expertise to create sector-specific testing packs (e.g. MFiD, FinRep) KPMG Powered Testing includes pre-built automation libraries to accelerate the testing of common IT systems and technologies such as ServiceNow and SAP Tools KPMG Powered Testing integrates both proprietary and market leading tools to automate the entire testing lifecycle from test case design to test execution. With our expertise across a wide range of tools, we can choose the right tool for our clients based on technology, operational and delivery requirements. KPMG’s partnership with leading tool vendors not only gives us access to cutting-edge tools, but also allows us to influence development roadmaps to cater to market trends or key client needs. Nexus is our proprietary test automation tool which translates the test scripts into technology-specific interaction. It integrates with open source and standard test automation tools in the market We are committed to continuously investing in innovating in-house tooling and integrate with external tools to stay ahead of the emerging trends. Methodology Powered Testing’s proprietary methodology brings the testing solutions to our clients in a systematic and structured manner. Our process and methodologies avoid delays and failures in tools implementation and enable our clients to achieve ROI faster. Our process framework can be tailored to fit the client’s specific needs and state of capability/ IT maturity to realise the maximum benefits. © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 12 The IBM Angle KPMG Powered Testing Components Test Design Test Data Test Envs Test Execution IBM Tools Legend Rational Quality Manager Rational Team Concert Tools we currently use Under consideration Tool currently not integrated Mobile Testing RFT Functional Testing RIT RPT RTVS / RIT Messages Based Testing Performance Testing Service Virtualisation Tool Features © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 13 Powered Testing and DevOps © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 14 Selling Powered Testing Key Principles • It’s not (just) a tool… • But requires tools to work • It’s a complex sell so we’ve tried to make it easier • Domain • Enterprise • Component © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 15 Powered Testing and DevOps Key Devops Principles Fail Fast, Fail Often” driven by “Test First” Feedback Testing is a continuous process Changing Requirements and & inadequate Test coverage ----------------------------------------------------------------------------------------Product ---------------------------------Backlog Requirements / user stories Performance bottle necks Daily Scrum Key Challenges Code breakdown due to frequent builds ---------------------------------------------Sprint ---------------Backlog ---------------- Sprint Planning Development Sprints Testing is cohesiveness between various teams, automation of everything possible. Late defect detection Acceptance & Delivery Speed Delayed Deliveries Test Environment Continuous integration Operate KPMG Powered Testing Model Based Test Design: Supports better collaboration between business and project teams. Embraces change and agility. Test Data Management: Removes limitations due to test data and increased the speed and efficiency of the test automation Service Virtualisation: Removes environment & system constraints and enable early testing and defect discovery. Automated Test Execution: Continuous testing and greater regression testing capability Feedback Test Design Model Based Requirements & Test Case Design ----------------------------------------------------------------------------------------Product ---------------------------------Backlog Requirements / user stories ---------------------------------------------Sprint ---------------Backlog ---------------- Sprint Planning Early performance testing Test Execution Test Data Daily Scrum Test Env Improved Regression testing Early defect detection Development Sprints © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. Acceptance & Delivery Quicker Feedback Operate 16 The Future © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 17 . Questions? © 20XX KPMG AG Wirtschaftsprüfungsgesellschaft, ein Mitglied des KPMG-Netzwerks unabhängiger Mitgliedsfirmen, die KPMG International Cooperative („KPMG International“), einer juristischen Person schweizerischen Rechts, angeschlossen sind. Alle Rechte vorbehalten. 18 The information contained herein is of a general nature and is not intended to address the circumstances of any particular individual or entity. Although we endeavour to provide accurate and timely information, there can be no guarantee that such information is accurate as of the date it is received or that it will continue to be accurate in the future. No one should act on such information without appropriate professional advice after a thorough examination of the particular situation. Note This disclaimer should only be used on external marketing materials, not client-specific documents (e.g. proposals or deliverables prepared under the terms of an engagement letter). © 2015 KPMG LLP, a UK limited liability partnership and a member firm of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity. All rights reserved. KPMG LLP is multi-disciplinary practice authorised and regulated by the Solicitors Regulation Authority. For full details of our professional regulation please refer to ‘Regulatory Information’ at www.kpmg.com/uk Note This disclaimer should only be used on Legal Services Reports. The KPMG name, logo and “cutting through complexity” are registered trademarks or trademarks of KPMG International. © 2015 KPMG LLP, a UK limited liability partnership and a member firm of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity. All rights reserved. 19