Business challenge – in product development
- The high rate of defects discovered during the last phase of a product’s release, results in elimination of features and / or compromising product quality.
- Limited unit and developer level testing increases the possibility of smoke test failure.
- Insufficient time for testing teams to iterate due to delayed availability of testable build.
- Delay in testing technical stories without a UI due to the limited skill sets of QA teams.
- Insufficient time for executing a complete regression test suite. This compromises quality assurance and increases defects in the production environment.
- Logged-in defects need tracking so that the same scenario can be tested in every product release. This ensures that there are no regression issues.
- Lack of automation techniques for repetitive functional testing increases regression efforts and adds to the cost.
- Difficulty in coordination and communication amongst distributed teams across the globe with different time zones.
Need-based and result oriented implementation of test techniques: While testing a large application a common testing technique cannot be used. This is because testing is a complex process. For example, to test the service side code where no UI is available, manual black box testing will not suffice. Upstream testing requires quick testing to prevent feature delays for further testing by downstream teams. The various innovative testing techniques implemented are:
- Functional testing before code check-in. This means compiling source code and testing code patches.
- White box testing / unit testing / DB testing, using an internally-developed customized framework:
- White box testing / unit testing is targeted at ﬁnding logical issues through errors, exception handling, unused code and code coverage.
- API testing deploys an internally-developed customized framework:
- Testing the service level methods and interfaces facilitates ﬁnding service level issues before the actual code check-in.
- Testing of public APIs used by other applications enables identifying of issues before they are used.
- Memory leak proﬁling allows identiﬁcation of memory leaks in the system using the ANTS proﬁler.
- Web service testing in SOA architecture: Empowers testing of the web services as they are developed, before exposing it to the customer or using it in the end product. An internally developed customized framework using different tools such as SoapUI and JMeter, allows web service testing. Testing individual web services facilitates identiﬁcation of the issues at service level, before use by the end consumer. This enables monitoring of response time for critical web services, which is vital for overall product performance.
- Plays the role of a catalyst to speed up turnaround time. To ensure that the turnaround time between the development and upstream testing teams is minimal, the latter conducts:
- Process deﬁnition and implementation for code testing.
- Upstream defect life cycle to expedite the bug closure.
- Incremental feature (code) delivery, by which testing is done on incremental pieces to avoid bugs at a later stage.
- Continuous Integration (CI) using build server and automated deployment to enable quick upstream testing: The latest features / code are quickly made available for testing. An upstream testing environment is created where CI builds are developed and the complete deployment process is automated. This accelerates upstream testing and saves manual deployment efforts.
- Automates repetitive and time consuming tasks. The following automated techniques are used to save manpower:
- UI automation is implemented to automate smoke test cases.
- Batch scripts and utilities are developed for carrying out routine activities. These include copying builds, switching services, installing client, upgrading database with the latest data alter scripts, stored procedures / functions and compiling local solutions.
Test approach for upstream testing
- Upstream testing is generally deployed in agile development methodology. The tester is always paired with two to three developers. Each component created by these developers is unit tested by the tester and bugs are resolved at that point.
- After the tester certiﬁes the component, it is integrated with the main branch by a continuous integration process.
- Automation scripts are executed every time on the integrated build to ensure that the overall functionality meets expectations.
- Test suites are continuously updated and the same are used for further integration and downstream testing. Following are the tools used in upstream testing:
- JUnit, TestNG, JAVA , SoapUI: These enable web services and API testing.
- Selenium, Ruby, QTP: These empower UI automation to prepare regression suites.
- In-house tool developed in JAVA: Automation frameworks are developed to test speciﬁc components like database components, JMS queues and internal services components.
Specialized role in upstream testing
- Upstream testing requires dedicated Software Development Engineers in Test [SDET], specialized in performing white box and API testing.
- Testers should be thorough in the usage of different automation tools for testing and have good knowledge of the tool / language in which the product is developed.
- Testers should be well-versed in continuous build integration process and high-level design of the software under test.
- Testers should have the skills to create and execute performance tests.
- Reduces cost of ﬁxing defects.
- Reduces defect life cycle by 8 to 24 hours by ﬁnding defects in early product life cycle.
- Improves code quality and feature stability in early phase and minimizes the regression defect density.
- Delivers high quality, cost effective products on time, through continuous and meticulous monitoring.
Mindtree’s upstream testing enables effective and early testing, constantly increasing the coverage during the development phase. It empowers developers to boost their productivity and allows the QA team to focus on integration and system testing.
- Enabled testing of Application Programming Interface (API) and web service testing, using customized automation tool developed in JAVA.
- Enforced static analysis using sonar.
- Set up continuous integration to conduct JUnit and functional tests for every check-in.
- Enforced automation runs for each sprint to avoid future regression defects and reduce the cost of defect ﬁxing.
- Tested restful web services using a customized automation tool developed in JAVA and testing.
- Planned testing for each sprint and milestone to avoid future regression defects.