The problem

Performance testing of a distributed software system, such as a multi-tier Web application, is a task much more complicated than it may look on the surface. Accomplishment of this task depends on how successfully do you solve the following key problems:

• Find, deploy, and master test automation tools. Today's software market offers some selection of products in this area. But, if your system is more complicated than a simple Web site carrying "static" pages and few forms, making the right choice is not going to be quick and easy. You would have to invest some substantial time and effort evaluating applicability of available performance testing tools to the specifics of your project. Most of such tools are not only expensive but also fairly complex. Therefore it will likely require additional time and effort to deploy the chosen instruments and train the testers to use them effectively.

• Create and maintain an adequate test lab. If your product is required to support simulation of thousands or more concurrent users, this problem may become very critical: the test lab should be capable to host not only the components of your product or application, but also sufficient amount of instances of the load simulation agents.

• Organize an adequate team. In comparison with other types of software testing, performance testing imposes quite unusual requirements toward the skills and overall mindset of a tester. There is much more to the performance testing than just writing a script, launching it on multiple machines, and generating several precanned reports. Performance testing involves dealing with many factors that are hard to quantify and predict. A performance tester must be able to debug test scripts, to estimate validity and quality of test results, to understand how numerous seemingly irrelevant factors may substantially affect the experiments. In fact, a skillset of an effective performance tester should include experience and knowledge from several adjacent fields, such as math, statistics, programming, networking, administration of various kinds of servers... Even a most advanced performance testing automation tool requires an experienced tester to control it and, most importantly, correctly interpret the testing results.

Given all these problems, it quickly becomes obvious that performance testing part of your project may require substantial time and resources. Not every development team has all that time and resources in place, and this is exactly a situation when professional services offered by U.S. Computer Software may help.

The solution

Depending on the technical specifics and organizational needs of your project, we can provide adequate outsoursing of the performance testing effort: the performance tests we conduct can be superficial or comprehensive, one-time or repetitive, on-site or off-site. In general, our performance audit services include the following activities: [0-WE]

• We design abstract test cases and benchmarks for the product based on its functional and performance requirements. (Abstract test cases and benchmarks do not depend on any particular performance testing automation tool.) As a part of this process, we can document designed test cases and benchmarks according to your documentation standards.

  Depending on the project's needs and budget, we provide the optimum level of comprehensiveness and coverage. A typical test suite includes tests that target the key performance aspects (scalability tests, stress tests, endurance tests, etc.) and tests to be conducted at different points of the entire development process (acceptance tests, benchmark tests, "smoke tests", etc.).

• We select test automation tools that are most appropriate for the project.

  Note: We do not limit our customers to the choice offered by vendors of commercial off-the-shelf performance testing tools. If necessary U.S. Computer Software can build a highly optimized and customized solution based on its own distributed test automation framework, which in many cases provides vital advantages in comparison with other options.

• We implement all test cases and benchmarks that can be automated (i.e. encode them into a form executable by the selected test automation tools).

• We set up the appropriate test lab (on-site or off-site). In certain situations, it may be necessary to have a distributed lab and conduct performance tests over WAN or Internet.

• We arrange the general schedule of test executions, tying this schedule to the specifics of your project: test executions can start as early in your development cycle as you need and they can be performed regularly (with different regularities for different kinds of tests). When necessary, we can also execute certain tests on-demand or even agree on 24/7 availability of all tests.

• We execute the automated tests in the test lab in accord with the agreed schedule.

  Once a particular test is executed, we instantly conduct detailed analysis of the test results in order to estimate validity and usefullness of each test. If necessary, we re-run the tests that did not yield valuable results.

• We generate detailed reports that quantify all the aspects of the product's performance. When possible, our reports not only demonstrate detected performance problems, but also pin-point the bottlenecks in the product and include professional recommendations on how performance could be improved.

The benefits

Here are the most important benefits your team can get from using our performance auditing services:

• We can reduce the cost of testing effort. Outsourcing of performance audit can save you the cost of load testing automation software, which can easily run more than $50,000, and the cost of a load testing network, which can run over $100,000 in capital costs and more than $100,000/year in ongoing administrative costs.

• We can reduce the cost of entire product. Our performance testing services are highly adaptable to the needs and specifics of your entire project. The tests can be started on the earliest stages of the product's development and facilitate discovery of performance defects (as well as functional problems) when the cost of correction is relatively low.

• We can significantly improve the quality of the product. We always stand behind the work we do and, in case of performance testing, we strive for the quality of your final product as if it was our own. Our testers always work toward the single ultimate goal: to be the first who tells you about any performance defects in your product. This attitude, backed up by the outstanding knowledge, skills, and technologies, is our proven route to increase your product's quality and your users' satisfaction.

All the benefits described above may be especially substantial in the cases when "boilerplate" performance testing methods and tools are not suitable for the needs of the project. Armed with powerful and flexible distributed test automation framework, out specialists can build the most effective and least expensive custom testing solution for a product of any comlexity.

Interested?

For additional technical or commercial information about this service, please send email to support@uscsw.com or sales@uscsw.com.

Products|Services|Company|Contacts