Get in touch with us
We at accompio will be happy to help you.
01.02.2024
Lara has completed the ISTQB certification for performance testing and in this post, she sheds light on the fundamentals and how these activities are integrated into the ISTQB testing process.
Continuous development and certifications are an important part of our work in customer projects. I have the ISTQB Certification for Performance Testing completed and this post will cover the fundamentals and how these activities are integrated into the ISTQB testing process. In particular, the focus will be on identifying and creating usage profiles as well as elaborating load profiles. By correctly applying different types of performance tests, the system's responsiveness under varying loads will be evaluated. Finally, the analysis of test results and reporting will be examined in order to derive well-founded recommendations for system optimisation.
The performance of a system is an essential aspect in providing users with a „good feeling“. In standard ISO 25010, performance is categorised as a non-functional quality characteristic with three sub-characteristics within the product quality model:
The prioritisation of these sub-features depends on the identified risks and the needs of the various stakeholders. Further areas of risk may be identified during the analysis of test results that need to be taken into account.
Performance testing activities are aligned with the phases of the ISTQB testing process and are organised and carried out differently depending on the type of development model used. Regardless of the development model used, various types of tests can be used for performance testing. The most important are static and dynamic testing. Static testing involves manual review of work products, while dynamic testing requires the execution of the software.
Static testing activities are often more important in performance tests than in functional tests. The reason for this is that many critical performance errors are rooted in the architecture and design of the system. If performance is crucial for a system, this should also be a guiding principle in its architecture.
Dynamic test activities should be started as early as possible in the course of software development. Potential bottlenecks can be identified and resource utilisation assessed even at the component stage. In system integration testing, performance tests are carried out with the entire system, which is representative of the production environment. Finally, acceptance testing is carried out to validate that the system's performance meets the originally specified user needs and acceptance criteria.
Usage profiles define different interaction patterns with an application. They enable a repeatable, step-by-step workflow through the application for a specific use of the system. Aggregating these usage profiles results in a load profile (generally referred to as a scenario).
First, the existing data will be identified. Different types of user personas and their roles will be determined. Generic tasks to be performed by these users/roles will be assigned to them. Finally, the estimated number of users for each role/task per unit of time will be recorded. This information is particularly helpful for subsequently creating the load profiles.
The data required for this can be collected in various ways. For example, by conducting interviews or workshops with stakeholders such as Product Owners, Sales Managers, and End Users. Functional specifications and requirements can be searched for intended usage patterns, user types, and their usage profiles. Another source of information can be the evaluation of usage data and metrics from similar applications, from which captured monitoring data, logs, and usage data can be analysed.
A load profile specifies the workload that a component or system to be tested might experience in production. The load profile consists of a certain number of instances (virtual users) executing the actions of predefined usage profiles over a specific period. The following information is required to create a realistic and repeatable load profile:
The created load profile is finally elaborated into a performance test. The term Performance test Herein lies the umbrella term for all types of tests aimed at the performance (responsiveness) of the system or a component under varying load. Various types of performance tests can be defined. Each of these types can be applied depending on the test goals for a specific project:
Functional tests generally benefit from precisely defined test oracles, as the expected results are usually clearly defined and the test results can be interpreted unambiguously. Performance requirements form the basis for the analysis of the tests. The metrics defined in advance determine what needs to be measured in the test run. The collected test measurement data is then compared with the performance test objectives.
Typically, the following data are analysed:
Graphical representations make it easier to view data and identify trends. To obtain meaningful metrics, various values should be put into relation to each other. The response time metric, for example, can be more precisely defined with regard to the time of day, the number of concurrent users, the volume of data to be processed, and so on.
The results of the tests will initially be compared with the objectives of the performance tests. Once the behaviour has been understood, conclusions can be drawn, resulting in a meaningful report including recommendations. For example, recommendations for technical changes can be made, such as the reconfiguration of hardware, software, or the network infrastructure. Areas for further analysis and additional monitoring can also be identified.
An effective performance testing process is crucial to ensure that IT systems meet the requirements and expectations of users. If you are wondering how to optimally integrate performance testing into your development process, do not hesitate to get in touch with us.
Arrange an initial consultation
