From 47e13f037f1c590d6d2d3cec09b528f16f480dd6 Mon Sep 17 00:00:00 2001 From: obicke Date: Fri, 8 Sep 2017 12:10:17 -0400 Subject: [PATCH 1/2] Adding a little more context around suggestion to include testing for authenticated traffic. --- source/_docs/load-and-performance-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/_docs/load-and-performance-testing.md b/source/_docs/load-and-performance-testing.md index 7cd24fb1a2..9c8a673a73 100755 --- a/source/_docs/load-and-performance-testing.md +++ b/source/_docs/load-and-performance-testing.md @@ -37,7 +37,7 @@ The procedure for executing a load test and a performance test are similar: The Pantheon onboarding team uses Locust, an open source load testing tool. Locust makes it easy to build out test scripts, and it allows you to crawl the site instead of using predefined URLs. Crawling the site has the added benefit of loading every page that is linked to anywhere on the site. This exposes edge case performance bottlenecks that would have gone undetected under tests with predifined URLs. -Ultimately, it doesn't matter what tool you use as long as you to test your site properly. Be sure to allow for any authenticated traffic as well as anonymous.   +Ultimately, it doesn't matter what tool(s) you use as long as you test your site against the anticipated traffic patterns of the site in terms of overall volume and the proportion of anonymous versus authenticated traffic. Note that load testing for anonymous visits is considerably easier than testing authenticated workflows, which will require more investment of time and skills.   3. Determine how much load to apply. From 4a70e15e03e89fb29254e989841ac781196b1fb2 Mon Sep 17 00:00:00 2001 From: obicke Date: Fri, 8 Sep 2017 13:18:06 -0400 Subject: [PATCH 2/2] Add nuances about when to run different tests --- source/_docs/load-and-performance-testing.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/source/_docs/load-and-performance-testing.md b/source/_docs/load-and-performance-testing.md index 9c8a673a73..a388a1237c 100755 --- a/source/_docs/load-and-performance-testing.md +++ b/source/_docs/load-and-performance-testing.md @@ -9,14 +9,16 @@ Load and performance tests are critical steps in going live procedures, as they ## Load vs Performance Testing Before you start, it's important to understand the difference between load and performance testing and know when to use each. ### Performance Testing -Performance testing is the process in which you measure an application's response time to proactively expose bottlenecks. These tests should be regularly executed as part of routine maintenance. Additionally, you should run these test before any load testing. If your application is not performing well, then you can be assured that the load test will not go well. +Performance testing is the process in which you measure an application's response time to proactively expose bottlenecks. In addition to regularly referring to your New Relic reports, you should consider regularly executing performance tests as part of routine maintenance to ensure performance isn't being degraded by code or configuration changes. You should run these test before any load testing. If your application is not performing well, then you can be assured that the load test will not go well.   The scope of performance tests should be limited to the application itself on a development environment (Dev or [Multidev](/docs/multidev)) without caching. This will give you an honest look into your application and show exactly how uncached requests will perform. You can bypass cache by [setting the `no-cache` HTTP headers](/docs/cache-control) in responses. + + ### Load Testing -Load testing is the process in which you apply requests to your site that will represent the most load that your site will face once it is live. This test will ensure that the site can withstand the peak traffic spikes after launch. This test should be done on the Live environment before the site has launched, after performance testing. +Load testing is the process in which you apply requests to your site that will represent the most load that your site will face once it is live. This test will ensure that the site can withstand the peak traffic spikes after launch. This test should be done on the Live environment before the site has launched, in advance of anticipated major-traffic events, or after major overhauls, remembering to provide enough time to fix any issues identifiedafter performance testing. -If your site is already live, then you should run load tests on the Test environment. Keep in mind that the Test environment has one application container, while Live environments on sites with a service level of Business and above can have multiple application containers serving the site. So try to run a proportionate amount of traffic based on how many containers you currently have on your Live environment. +If your site is already live, then you should run load tests on the Test environment. Keep in mind that the Test environment has two application containers, while Live environments on sites with a service level of Business and above can have multiple application containers serving the site. So try to run a proportionate amount of traffic based on how many containers you currently have on your Live environment. ## Preparing for Tests The procedure for executing a load test and a performance test are similar: @@ -41,8 +43,8 @@ Ultimately, it doesn't matter what tool(s) you use as long as you test your site 3. Determine how much load to apply. - * **Performance Tests**: Smaller loads should suffice, as you should be able to see transactional bottlenecks with 10-20 concurrent users. - * **Load Tests**: Determine how many concurrent users the site is expected to serve based on historical analytics for the site. Identify the peak hourly sessions and average session duration, then do some math: `hourly_sessions / (60 / average_duration) = Concurrent Users` + * **Performance Tests**: Smaller loads should suffice, as you should be able to see transactional bottlenecks with 10-20 concurrent users. + * **Load Tests**: Determine how many concurrent users the site is expected to serve based on historical analytics for the site. Identify the peak hourly sessions and average session duration, then do some math: `hourly_sessions / (60 / average_duration) = Concurrent Users` ##Running the Tests