Challenges Associated with Performance Testing

Posted by MindTree Blog Archives in

 Testing

Nov 5, 2009 | Total views: 196

0

MindTree Blog Archives

More articles by the author

In this section of the blog, I will cover few key challenges that we face in PT. In the next blog I will provide solutions and some tips to overcome these challenges.

Common Challenges to all types of application:

Elaborate Infrastructure Setup – PT encompasses testing with a large set of users and hence the infrastructure requirements are more stringent than those of functional testing. In many cases, it is a replica of the production environment or it’s subset. The challenge with environment setup goes beyond hardware aspects into software and its configuration. It is very critical to configure the application, web server, app server, and db server with production like settings. Setting up this environment is very critical to the entire PT exercise as it has direct impact on the outcome.

Collection and Analysis of Huge Data – PT exercise generates enormous data in the form of server logs, application logs and related metrics. This poses challenges around proper data collation and report generation which yield actionable results.

Challenges – PT of N-tier Applications:

N-tier applications present unique challenges:

    – Continually changing technologies

    – Large number of users

    – Large variety of user operating environments, e.g. Windows, Linux, Mobile

    – Architectural depth associated with an N-tier application is made up of a large number of components residing on multiple machines. This increases the scope and complexity of testing.

N-tier applications are typically multi-threaded. This introduces additional hard-to-find problems such as deadlocks, race conditions, and improper handling of shared resources.

Challenges – PT of SaaS based Applications:

The fundamental question here is “If SaaS applications are web-native, is there any difference between PT of a SaaS application and a traditional web application?”

Key differences between a SaaS application and a traditional Web based application are the number of end-users, underlying infrastructure, and the architectural complexity. It is essential to consider these factors before conducting PT on a SaaS application.

Other important questions reflecting the challenges associated with PT of SaaS based applications are:

    – What are the factors that can affect performance of the SaaS application?

    – Is it important to differentiate between the SaaS platform and the applications hosted on it while performance testing?

    – Is it necessary to carry out PT of all the nested applications (tenants) hosted on a SaaS platform or will testing only some of the tenants suffice?

    – Which tenants need to be considered for the performance tests?

    – Does customization affect the performance of the SaaS application?

It wouldn’t be wrong to say that the performance questions do not change, but the answers do. Two factors that directly affect the performance are the layers of users and customization.

In the blog, I will cover solutions for these challenges and some interesting tips. Meanwhile, do share your thoughts and questions on my points which will make the post interactive and tuned.