How to run website performance & load testing using blended scenarios

in Technology   —  

When running performance tests, it is important not to run just a single test in isolation, but blend a number of realistic scenarios.

For example, consider a website which allows users to search ecommerce products and to refine their results. If we only load tested the search functionality, then we would simply be testing how quickly the search index can be read under heavy load.

For a realistic test, we also need to see what happens when product content is updated or published, as this will cause the search index to be rebuilt. This ensures that different elements of the underlying solution are tested together, and not just in isolation.

London Live launch advertised on the cover of the London Evening Standard

Defining blended scenarios

Imagine a website providing a job search function. It might be reasonable that the site has 20,000 registered users who are actively seeking a new role, and that around 1 person in 5 accesses the site each morning within a 2 hour window (e.g. following receipt of an overnight email showing them potential new roles).

In this case, it would be reasonable to assume that the peak load is defined as 10% of registered users accessing the site within a one hour window – so in this case, the web application needs to be able to support 2,000 visitors per hour.

Given such a case, we would suggest testing the following scenarios

  • Scenario 1: 80% of users login and view their overnight job matches (1,600 candidates per hour)
  • Scenario 2: 20% of users to login and search (400 candidates per hour)
  • Scenario 3: Several staff members login and add or update job vacancies.

What each scenario tests

Each scenario is testing a different function of the web application.

Scenario 1 tests the raw database performance. Applications that provide job shortlisting often rely on extremely large database tables to track view history, in order for users to only get notified about the positions they haven’t previously seen. These databases can store often millions of rows of data.

Scenario 2 tests how quickly a search index can be read under load, as well as any data it needs to retrieve from the database (such as job summary or description). In order for a realistic test, we also need to see what happens when new jobs are published, as this will cause the search index to be rebuilt.

Scenario 3 allows staff members to login and modify data. This could affect potentially affect both the shortlist database as new positions are added (especially when shortlists are built automatically) as well as causing the search index to be rebuilt. We would include this scenario within the load tests to make sure that this is a process which happens in the background and has with no noticeable performance impact on users who are accessing the site.

Read next

ArticleTechnology

Why brands should use the latest image technologies

To deliver a perfect experience across multiple devices it is essential to consider how images are delivered

ArticleTechnology

The pros and cons of mobile app development with Xamarin

With more than one third of the global population predicted to be smartphone users by 2017, it is essential for companies to provide their customers with a smooth and responsive mobile user experience. With more and more companies now looking to compete on the basis of customer experience alone, this is more critical than ever. On the one hand, achieving a world-class mobile experience could mean extensive time and effort being spent to develop native applications across any number of mobile platforms. Yet, there remains a strong argument to be made for using cross-platform development options, either in whole or in part, such as Xamarin, instead.

ArticleTechnology

Comparing five popular frameworks for mobile development in 2016

In part 1 of how to approach mobile development in 2016, we directly compared native vs cross-platform mobile development and highlighted that a hybrid approach can, in some instances, be a solution to provide a seamless mobile experience. In part two, we are going to evaluate some of the popular frameworks you might use when the need arises for cross-platform development, helping you find what is best suited for your project.