Nowadays, software can be found in nearly every user device – from classical IT products like smartphones, tablets and computers through to domestic appliances like refrigerators and microwaves as well as bicycles or vehicles.
Most of this software is still written by human software developers – and humans can make mistakes. To correct these mistakes, different software testing methods are currently available to reduce errors to a minimum. Nevertheless, it is cost and time expensive to try to reach zero errors in a code base. In addition, this will have an impact when these “forgotten” mistakes could danger the life of other humans.
A recent example was the issue with Boeing’s 737 Max 8 flight software. With changes on the plane's software, there were no errors seen in the first place. Also, the flight software was properly tested in many test cycles and hours of test flights. However, there was this one test case, which led to the known fatal crashes.
This example shows the need for sufficient software testing when it comes to systems, which have an immediate impact on safety functions and human life. But does software testing in a vehicle have the same importance as it has in a plane’s flight software?
When looking at the codebase of some examples, modern car software has doubled the lines of code of CERN’s Large Hadron Collider and seven times as many as a modern Boeing 787. If we speak in some years about software-defined architectures and software-defined cars, this number will increase heavily.
According to Steve McConnell, author of the book “Code Complete”, there is an average of about 15 to 50 errors per 1000 Lines of Code (LoC) and with, mature testing methods, it is possible to bring it down to 0,1 to 0,5 errors per 1000 LoC in production. When multiplied with 100 Mio. LoC, there would theoretically be about 10.000 up to 50.000 errors in total even after testing measures.
With this in mind, a modern car seems not so safe and secure anymore, when compared to this number of potential errors and malfunctions.
Moreover, it is even more important to have not only one but many different mature testing methods for software in the automotive industry. This belongs not only to the Original Equipment Manufacturers (OEMs), who produce the cars, but also to those that deliver the accompanying components and software – including tier one and tier two suppliers, suppliers of standardized components (like SIM cards, processors, chips) and programmers of any third party library. Moreover, there are many different programming languages for different protocols in the car, which must also be considered.
For holistic testing, there are several crucial factors:
- Safety and security should not be treated only in its own tests, but also with an integrated approach together. A model for these tests is currently under development at the University of Ingolstadt in Germany (Program MASSiF);
- There should be automated test cycles and automated Dynamic Application Security Testing (DAST) and Static Application Security Testing (SAST) testing on every iteration of code during development to find as many possible errors in early stages;
- OnBoard scans in the vehicle for software malfunctions can help to identify software errors after production and implement fixes over software updates;
- Not only the OEM but all suppliers of any kind of software should be committed to perform quality assurance and software testing for all code before delivering it to the OEM;
- There should be a “fail mode” in every car, where the driver can take over manually and rely on mechanical failover mechanisms (like mechanically breaking the car) when the software has a malfunction.
In the end, it can be seen, that it is difficult, to build cars without any software errors. Moreover, it is important, to be sensitized, that such errors will occur, even if the software is tested for thousands of hours and how to deal with this remaining errors to make driving a (software defined) car safe and secure.