Load testing

Test phases

The typical phases of a load test are:

1. Definition phase:
   • Formulate scenarios by mapping user journeys.
   • Determine levels of traffic and load profiles.
   • Define primary objectives, including key performance indicators (KPIs).
2. Test script creation phase:
   • Design test scripts using the defined scenarios.
   • Run small-scale tests to ensure the quality of the scripts.
3. Load test performance phase:
   • Carry out different types of load tests based on the established scenarios.
   • Create a final report summarizing the results of the tests.
4. Cleanup phase:
   • Remove data generated by the tests once they have been completed.

Load test scenarios should focus on testing crucial workflows that mimic typical user behavior, with an emphasis on impactful and realistic scenarios for the business. For example, simulating a standard checkout flow including browsing, adding products to a cart, and completing orders.

Tests should examine the system's behavior under regular load conditions, validate its performance during high-traffic periods, and assess its performance under sustained load over an extended period.

The following table can aid in traffic design:

Types of load tests

Baseline load tests

Implement baseline load tests to simulate routine traffic scenarios and validate major architectural changes regularly. Plan for future growth based on the results from load test analyses. Keep a record of previous results, such as load driver reports featuring performance data, resource usage, and other observations for comprehensive tracking and analysis.

Aligned with industry best practices, we recommend tests run for 30 minutes, including a 15-minute period for ramping up. In this ramp-up phase, traffic gradually increases, signaling all components involved to scale up. Scaling doesn’t take place instantly, and ramping up leads to better test results. Although the test duration can be extended, it's generally enough time to collect important performance data. Running longer tests won't provide additional insights.

Peak load tests

Conduct peak load tests using anticipated peak traffic in readiness for a campaign or seasonal spikes. Increase the final load to a range of 20% to 50% over the expected peak traffic. The duration of this test should be 20 minutes, including a 15-minute ramp-up period. Alternatively, you can use the baseline test as a warm-up.

Endurance tests

You can choose to run endurance tests before the system goes live for the first time. These tests help to evaluate stability and identify possible performance issues over a long period. They often last between one to four hours and can be carried out at different times during the day. For a reliable comparison and analysis, it's important to keep the load level the same as in the baseline test.

Choosing the right environment

A typical Composable Commerce setup includes test and production Projects. To keep test and production data strictly separated—a common regulatory requirement—we recommend running performance tests against the test project. Alternatively, you can run pre-launch tests against the production project and clean up all data before launching.

Additional guidelines

• In the context of load testing, specific data such as product catalog details may be required. Do not incorporate creating this data directly into your load tests. Generate the necessary data incrementally in advance to avoid undue stress on the system. Consider keeping supporting data for consecutive tests. For instance, if you run a daily load test for order creation and use the same product data set, it's practical to keep that data set.
• Generate traffic from nearby regions to minimize network overhead.
• Start your load testing conservatively. Perform initial testing at lower loads to validate responses and confirm the functionality of your load test scripts.
• Gradually increase load levels during testing. Directly starting at maximum load is unrealistic behavior and not permitted.
• Include periods of 'think-time' in performance tests to mimic human behavior.
• Monitor resource utilization—such as CPU, memory, and network usage—on the load driver machines that simulate the traffic.
• You must clean up test data that is no longer needed after the test.

The following tests, aligned with industry standards, are considered as fair use. Reach out to your commercetools contact person to learn more about additional options.

• Baseline load tests as part of the everyday development process up to five times per week at standard load, each lasting a maximum of 30 minutes.
• One peak load test at up to 50% higher than the expected peak load once per quarter, for a total of 3 hours per test session. Each 3-hour session must be completed within 5 consecutive working days.

By following these best practices and guidelines, you can ensure the smooth operation of your system, even under increased loads.

Exclusion periods

Load tests are not allowed in the following periods of the year:

• 2 weeks before Black Friday to the Tuesday after Black Friday
• Between December 20 and December 26