Microsoft Azure has a Load Test tool that makes it very easy to create and execute your own load test plans at a very cost effective price. This tool can also be used to load test against non-Azure resources. It is easy to copy sample API calls you may have in Postman and execute them at scale. This document outlines how to do so.
(Note, Azure now supports scheduling load testing on a cadence, so it could be done weekly at night or similar https://learn.microsoft.com/en-us/azure/load-testing/how-to-schedule-tests?WT.mc_id=Portal-Microsoft_Azure_CloudNativeTesting)
Login to Azure, and search for the Azure Load Testing service at the top. Select Create, to create a new one. (Note, you can have multiple tests within a single resource. Typically, one resource is enough unless you plan on executing multiple load tests at the same time)
Name it as desired, then click Review and Create. It should create quickly.
Basics Tab
Navigate to your new resource. You can setup load tests in a number of ways, by HTTP requests (which can easily be tested first in Postman), by uploading a pre-written JMeter script, or even connecting to a CI/CD pipeline. This document will just outline a basic HTTP request scenario, so select Create under "Add HTTP requests."
Enter in an appropriate name for your test and description. Uncheck "Run test after creation" as you can do that later before verifying it is correct. Click Next.
Test Plan Tab
On this step, you are allowed to create your HTTP requests for the load test, up to 5. You can also have your load test input data files as shown further below.
Click Add Request to add a new HTTP request.
Specify your request name, the URL to call, and your HTTP method. (E,g, a GET to obtain a token) Add your query-specific parameters as well.
Azure Load Tests allow you to save response variables to use later, like in this case, we wish to save the generated token to use in subsequent calls. Click on the Response Variables tab and fill it out something like this:
That's it for this request. Close it, and then click Add Request to add another one. Fill in the relevant information here, and make sure you're using the right HTTP method. In the login request, we needed to use a GET, but here we are creating sample order information so we need to use a POST.
Click on the Headers section. This is where we want to pass in the token we obtained in the previous step. So the Header name is AptifyAuthorization, and the Value is SQL ${authToken} which just uses the token generated before.
Click on the Body tab. This is where you enter the body (usually JSON) that you wish to call here. Here's an example of creating a new Order with 10 subtypes. Click Add when you have everything set, unless you need to record a response variable.
{
"ID": -1,
"EntityName":"Orders",
"EmployeeID": 1,
"OrganizationID": 1,
"ShipToID": 10,
"BillToID": 10,
"OrderStatusID": 1,
"OrderSourceID": 4,
"PayTypeID": 1,
"PONumber": "ABC",
"SubTypes": [
{
"ID": -1,
"EntityName": "OrderLines",
"ProductID": 3,
"Quantity": 1
},
{ "ID": -1,
"EntityName": "OrderLines",
"ProductID": 3,
"Quantity": 1}
,
{ "ID": -1,
"EntityName": "OrderLines",
"ProductID": 3,
"Quantity": 1}
,
{ "ID": -1,
"EntityName": "OrderLines",
"ProductID": 3,
"Quantity": 1}
,
{ "ID": -1,
"EntityName": "OrderLines",
"ProductID": 3,
"Quantity": 1}
,
{ "ID": -1,
"EntityName": "OrderLines",
"ProductID": 3,
"Quantity": 1}
,
{ "ID": -1,
"EntityName": "OrderLines",
"ProductID": 3,
"Quantity": 1}
,
{ "ID": -1,
"EntityName": "OrderLines",
"ProductID": 3,
"Quantity": 1}
,
{ "ID": -1,
"EntityName": "OrderLines",
"ProductID": 3,
"Quantity": 1}
,
{ "ID": -1,
"EntityName": "OrderLines",
"ProductID": 3,
"Quantity": 1}
]
}
You should now have a HTTP-based test with two steps, one to login and obtain a token, the second to create new orders using the previous token to authenticate.
The Parameters tab has a number of options to save environmental variables, secrets and certificates if they apply to your situation.
Load Tab
Now we need to determine how much load to perform in this test. Click on the Load tab.
Here, you can set the number of Engine Instances, which is how many distinct engines are making load test calls at a given time.
Each engine is capped at 250 users maximum. If you are only looking to do a simple basic load, you may want to do a test with 5-10 instances but the max number of users. If you are trying to test reliability of your site when hit with a number of connections, you may want to choose 40-50 engine instances and fewer users per engine. If you are attempting this for the first time, we recommend choosing a small amount of engines and users- they can always be increased later.
You can also decide how the load pattern works, if it builds up linearly, in block steps, or random spikes.
This section also defines how many concurrent users per engine (250 is the maximum) as well as how long you wish to run the load test for. The ramp-up time determines how long before the load test scales up to its peak testing.
The information section under the graph helps explain what your load test will do:
The final two sections determine how the load is conducted. You can have the load distributed across different regions, for example, 50% from US East, 25% from US Central, and 25% from US West.
The Network tab determines if you are load testing a website over the public internet. If it's publicly accessible, you can leave this public.
If it's not publicly available, mark it as Private and you will need to identify the Virtual Network and subnet that your target website resides in.
Monitoring Tab
On the Monitoring tab, you can select Azure resources that will monitor and record server-side metrics during the test run. This is optional, but recommended- for example, this will record CPU usage of the target Web App and can be viewed in this app service plan selected.
Test Criteria Tab
On this tab, you can define a number of criteria both on the client-side and server-side on what should be considered a Pass or Fail for your Load Test. You can also have the Load Test automatically stop if the Error Percentage exceeds some threshold.
Click Apply. It will now update and create your Test, as well as automatically generate the Jmeter script. Should you need to modify it in the future, you can always go to the Test, click Configure, and select Test to modify the HTTP requests and other items:
Executing and Interpreting Load Tests
Once you're ready to perform a load test, go to your test in Azure Load Testing, and select Tests on the left hand side. Pick the test you created and click on it. (Remember, you can have multiple tests within the same Load Testing Resource, although you cannot run multiple within the same resource at the same time)
It will now display your test and links to previous test runs, in case you'd like to review the data. You can also select multiple runs and click "compare" or the Trends tab to see how the test results have changed over time.
You can modify the test if needed by clicking Configure and then Test.
If you're ready to run it, then go ahead and click run! Add a helpful Test Run Description so you know the scenario you are testing. You can also select debug mode, it will only run one single engine for 10 minutes, but it will collect a significant amount of information for failed requests.
It will take a few minutes to provision the engines and prepare the test, but if you refresh the page with your test runs, you should see it with a Status of Executing. You can click on the specific Test Run while it is executing if you wish to look at live information.
Once it's completed, there's a significant amount of information to help identify and resolve problems. (If there are any!) One important note- if the load test fails, one of the first things to check is that the same HTTP calls work in Postman. That is an easy way to rule out a number of problems.
First, you should check the Engine Health tab to ensure there were no resource constraints on the resources conducting the actual load test.
Next, click on the Load Tests Results tab for more details.
For instance, in this sample load test (Log in, create an order) you can see we had a 4.98% Error percentage and it was higher on the Create Order step. You can also examine Client and Server side metrics.
Two other very helpful tools are included. First, you can select multiple test results and click Compare to see how they performed against one another.
Secondly, Azure Load Testing records a significant amount of information that you can download. The Input File will provide you the jmx, json and .yaml files for the load test.
The Logs file will provide a log of the progress of the test by each engine worker.
The Results file will generate a .csv for each engine that tells you detailed information about each call, which step it was on, the HTTP response code, the URL it was connecting to, etc.
The Reports file will generate an HTML file that includes information about the load test performance, and can be shared easily with others without needing to grant them Azure access.
Comments
Please sign in to leave a comment.