API Performance Testing using JMeter- Streamlining API Testing Process
API testing involves conducting tests to determine if an API meets its functional, reliability, performance, and security requirements of the application in question. API testing is performed in the message layer as it lacks a GUI interface and is one of the challenging aspects in quality testing of a software development project. Until recently, manually testing all the APIs several times was the common method used by QA teams around the world. It is safe to say that this was a laborious process. But with automated testing tools becoming popular, API testing has become a lot more easier and faster.
Approach for API integration heavy project
From a quality assurance point of view, its best to have a “Shift Left” approach towards software and systems testing. Shift Left emphasizes to move testing towards left of the project timeline, i.e. test early and often. Identifying and rectifying a bug at the beginning of the project would save a lot of time, cost and effort as compared identifying a bug in the later stages of development from a project development standpoint.
Testing validates the underlying business logic of the software architecture. Automated testing reduces the time taken for testing by a significant margin. However, it is impossible to automate all testing needs of a project. Determining which tests should be automated and which ones to be tested manually is crucial. These are some of the reasons why API testing should be automated
- Several versions of APIs developed during the development cycle needs be tested. Manual testing of each version can take on an average 20% of total development time conservatively. For e.g. if 5 versions of APIs are created, the total testing time required would be 100% of the total development time.
- APIs tend to have stable input and predetermined outputs and therefore are ideal candidates for automated testing
- Once the initial script for automated testing of an API is written, the incremental effort required for additional scripting is minimal (less than 5% of the time required for writing initial testing script)
API Testing
Experion conducted a 2-week discovery phase to study the existing system and establish the business goals of the client. This phase helped Experion to understand the underlying drivers for a successful outcome, in addition to the requirements brief shared by the client.
To fix the API request count, Experion introduced a mechanism to queue and persist the commands on the client browser. This is then processed as a batch with the help of a service worker (a background service) and synchronized with the server at fixed intervals. This implementation reduced the request count substantially, which in turn reduced the cost of the infrastructure. We also designed a resilience mechanism to prevent any further data loss in worst-case scenarios.
Since there were a large number of requests for user authentication, the authentication mechanism was implemented as a separate service.
Additionally, an intuitive user interface was designed and developed as part of the solution re-engineering activity, that offered ease of usage to all relevant stakeholders from the client’s side.
Post-delivery, Experion continues to provide development services for the client in the ongoing engagement.
Let us look at an example which illustrates the best approach to test a project which used API testing. Experion worked with a client in the healthcare sector as their technology partner to develop a software solution which comprised two independent applications.
-
A browser-based reporting tool
-
DICOM* router interfacing different PACS*
*DICOM = Digital Imaging and Communications *PACS = Picture Archiving and Communication Systems
Radiologist will use the browser-based reporting tool to read and interpret digital images from healthcare devices. This tool also helps radiologists record their dictations using a recorder along with their workflow management.
The second product is an interface to DICOM router, that helps it to import, modify and export images among different picture archiving and communication systems (PACS). The core of the DICOM router application (implemented in Java) is to be interfaced using RESTful API which uses JSON format for data exchanges. Executing the APIs sets policy and rules in DICOM router will enable the exchange of DICOM images between PACS situated in different geographical area.
*RESTful = Representational state transfer
For such a complicated software system which uses a lot of API calls, it is quite natural to implement test automation to determine the expected outcomes. Experion deployed its test engineers at every stage of the API development process. Involvement of Experion’s Independent Verification and Validation Team members from the initial discussion enabled it to identify and analyze the customer’s business requirements and to finalize the best testing approach. The team created test strategies for functional and non-functional test such as security testing, compatibility testing, performance testing, API testing, etc.
The Solution
Experion’s Test engineers devised a two-stage test strategy to meet the challenges of the project. In stage one of testing, individual APIs were validated for mandatory parameters included in JSON request and response header and JSON response body were also validated. In stage two, APIs were called in sequence to simulate an E2E functionality of the final software solution.
For stage one of testing, the team used Postman as the testing tool. Postman is an API development tool which enables users to build, test and modify APIs. Postman, can select the appropriate API call method from a drop-down provided in the tool, set authorization, Header and Body information according to the API call.
For the second stage of testing, where the end-to-end functionality is tested, Tech QA team automated the script to check and validate APIs as part of the CI-CD (Continuous Integration/ Continuous Deployment) pipeline. Now when the developer commits a change, the CI-CD pipeline deploys the latest application on the test environment. The E2E test automation scripts starts executing to validate the application. The developer gets immediate feedback of their commits/ changes after the scripts are executed.
An automated test script was also designed to run in JMeter. The Apache JMeter™ application, a 100% pure Java application designed originally to load test functional behavior and measure performance, is an open source software. Using JMeter, to sequence API call, has an added advantage of Load Testing API with multiple virtual users.
The JMeter test plan was created with User defined variables and was initialized with values which will be consumed in the script. Simple Controllers were used to section API and under each controller, Samplers containing HTTP request along with Response assertion are used to validate the response from server. JSON Extractors was used to extract information from response body and supply to a Sampler which consumes the same. During the initial stage of development of the script, results were viewed in Result Tree for debugging and tabular summary report. In the deployment pipeline the script was run in a non-GUI mode and results viewed in JMeter text logs (.JTL) files.
Results of Testing
Even though automation requires higher initial developer time for a given feature, it increases efficiency of the overall project. Particularly in projects which involve integrating multiple APIs. Some of the advantages of using automated testing are
-
- Immediate feedback to developers
A test can be run immediately after build is committed, helping the developer to fix error and bugs, without having to wait for the testing team to validate the build. This ensures that software Integration is seamless - Better Reliability
Automated tests can be executed by the developer at his end. This increases accountability as any error can be identified and resolved immediately, resulting in an error free final product. - Higher Efficiency
Even though automation requires higher initial developer time for a given feature, time needed for testing and bug fixing is drastically reduced. Resulting in shorter overall development time. - Higher ROI
As the overall development and QA requirement are lesser for projects which use automated testing strategy. The overall cost of the project will also reduce increasing the total ROI for development and even maintenance.
- Immediate feedback to developers