RREST APIs are the mainstream mechanism to expose remote resources and functionalities, enabling diverse software systems to communicate over the Internet via a common interface. Automated test generation for REST APIs is a valuable support for quickly exploring a large number of execution scenarios (reducing manual effort) for identifying bugs, leading to more reliable and secure systems. Generating tests for REST APIs is challenging, as the source code of the system under test is typically inaccessible, due to microservices architectures, third-party libraries, or proprietary components, and APIs must be treated as black boxes.
In this context, the REST League competition aims to foster research community and industry engagement, formalizing REST API test generation as a vital and emerging sector within software engineering. In this first edition, REST League received three industrial tools and two research tools for evaluation. These tools have been validated on a dataset of real-world REST APIs by using RESTgym as evaluation platform. Evaluation criteria include fault detection, API operation coverage, source code coverage, and testing efficiency, automatically collected by RESTgym.
PDF version of the paper.