Publications

Component-based development allows one to build software from existing components and promises to improve software reuse and reduce costs. For critical applications, the user of a component must ensure that it fits the requirements of the application. To achieve this, testing is a well-suited means when the source code of the components is not available. Robustness testing is a testing methodology to detect the vulnerabilities of a component under unexpected inputs or in a stressful environment. As components may fail differently in different states, we use a state machine based approach to robustness testing. First, a set of paths is generated to cover transitions of the state machine, and it is used by the test cases to bring the component into a specific control state. Second, method calls with invalid inputs are fed to the component in different states to test the robustness. By traversing the paths, the test cases cover more states and transitions compared to stateless API testing. We apply our approach to several components, including open source software, and compare our results with existing approaches.

Component based development allows to build software upon existing components and promises to improve software reuse and reduce costs. To gain reliability of a component based system, verification technologies such as testing can be applied to check underlying components and their composition. Conformance testing checks the consistency between the behavior and component specifications. On the other hand, robustness testing detects vulnerability of software with unexpected input or stressful environment. Existing robustness testing tools aim to crash components with preset values of different data types. But they do not take into account component states, which are vital to the detecting robustness problem of a component. We propose a state machine based approach to detect robustness problems of components. Firstly, a set of paths is generated to cover transitions of the state machine. Test inputs which follow the paths achieve high coverage of the system states and examine more transitions than stateless API testing. Secondly, invalid inputs and inopportune method calls are fed to the component in different states to test the robustness. When unexpected exceptions arise in the test runs, robustness failures are reported. We do a case study on a component from an open source software and it results in positive results.

 Component-based development allows to build software from existing components and promises to improve software reuse and reduce costs. For critical applications, the user of a component must ensure that it fits the requirements of the application. To achieve this, testing is a well- suited means when the source code of the components is not available. Robustness testing is a testing methodology to detect vulnerabilities of a component under unexpected inputs or in a stressful environment. As components may fail differently in different states, we use a state machine based approach to robustness testing. Firstly, a set of paths is generated to cover transitions of the state machine, and it is used by the test cases to bring the component into a specific control state. Secondly, method calls with invalid inputs are fed to the component in different states to test the robustness. By traversing the paths, the test cases cover more states and transitions compared to stateless API testing. We apply our approach to several components, including open source software, and compare our results with existing approaches.