WHY A THRESHOLD?
A session represents a set of requests that the user generates to the application under test. In order to assess the performance of a web application we could use a threshold value, that represents the maximum acceptable value for that specific session. If the test results are below the threshold, then the test status is PASS. If it is above the threshold, the test will return FAIL. Because the results are based on statistical analysis, we also have a “gray” area around the threshold, that might require some deeper investigation, to really determine if it is acceptable.
A Pass/Fail result is very useful for test automation. Whether using the API or one of the plugins, this will be the outcome that can be associated to a pass/fail for the automation.
LOAD TESTING - HOW TO SET A THRESHOLD
The Threshold for the load testing refers to the average response time over all the requests received.
On the New Test Plan page, there is a field that says Response threshold, as shown in the figure below. It has a default value of 600ms. This value can be modified as needed.
Once the test is run, upon completion an email will be sent with one of the following subjects:
(PASSED) Divecloud Test for Plan - YourTestName is Complete
(FAILED) Divecloud Test for Plan - YourTestName is Complete
(INVESTIGATE) Divecloud Test for Plan - YourTestName is Complete
NOTE: If the test had at least on error, then the ERROR outcome will be displayed.
Also, in the result page the label PASS, FAIL, INVESTIGATE or ERROR will be displayed in the result header, like shown in the image below.
The threshold value will be displayed against the average response time. The orange line represents the threshold. In this example there are also errors, and the response time is always above the threshold.