News

Today, embedded systems are used in a wide range of industries and devices we use, from smartphones through TV sets to cars. To assure the systems are of the highest quality, it is essential to test them and, for that to happen, create appropriate test environments. The test infrastructure team provides the essential elements for conducting tests as well as the tools for the management of the testing process itself. They are the key component both for testers, who need access to the right set of tools, as well as for the customer himself who expects reliable test results.

An important part of the test infrastructure are test environments. Testing of embedded systems is focused on the actual device, which may require various types of sensors, actuators or communication interfaces (used for interaction with the surroundings or other systems) in order to work properly. When testing particular components such as an ECU (Electronic Control Unit), for example, the test environment simulates input signals and allows the tester to check output signals produced by the device. Therefore, one of the primary tasks of the infrastructure department is finding solutions, which enable us to simulate the environment, in which the device under test will be used.

In our company, B&R X20 series, controllers, and Vector VT System devices are commonly used for this purpose, which the tester is operating using the Vector CANoe software. While the Vector VT System uses a build-in communication interface within CANoe, it was the test infrastructure team’s task to create a real-time communication solution with the other components.

The projects the infrastructure department takes on can be quite varied depending on the test requirements. After being presented with a specific test’s needs, the team members analyze the possibilities and create a proof of concept.

Then, by using tools like CANoe, TestStand, vTESTstudio or other software and hardware tools build solutions such as:

• Integration testing– that breach the gap between a testers PC and the world of physical signals of the system’s hardware,
• Software-software interfaces – that allow testers to incorporate 3^rd party software tools (like a diagnostic tool of the embedded system’s manufacturer) in test scenarios,
• Full test environments – in which a tester has complete control of the necessary inputs of the system under test. It can be comprised of multiple tools and components working together to reach test goals. This is done especially for new projects,
• Simulations of devices and/or whole systems – when some parts of the end system are still missing but are essential for the required test scope,
• Test automation – of both tiresome and error prone operations like storing test results in a database, or setting the test environment in an operational state.

In addition to creating and managing ways to interact with an embedded system the test infrastructure team provides tools for organizing and grouping of test procedures, managing test executions, smoothing administrative tasks and test management. Presenting test results in a comprehensive way requires an additional effort and can be quite challenging. For example, a common practice in testing is to repeat the same group of test cases multiple times while the system is being developed.

Due to bugs and issues being uncovered and removed, the amount and scope of tests is expected to change between the first test execution and the release of the system. To provide meaningful, unambiguous and test results for the stakeholders, it is crucial to create some sort of consolidation beside the raw test execution data. The knowledge to define the consolidation rules and implement them in a tool for ease of use is one of the many skills the members of the department have.

The infrastructure team needs to closely cooperate with test teams, developers, and other people involved in the testing process, to understand all their needs and expectations. All to ensure the test environment is complete and effective, which enables the testers to conduct trustworthy and repeatable tests and all stakeholders to interpret and understand the results. The team holds a key position assuring high-quality of products and services and its members not only create test environments but continuously manage and improve the test infrastructure.

Authors:
Aleksander Stasiak
Łukasz Rdzeń