Unreliable Programming: a method for evading liability claims on software.
Members of the security and safety community often claim that software quality would improve if manufacturers would be held liable for damages caused by their products. The reasoning uses the negative incentive argument: “If we produce faulty software, we will lose money. Let's write correct software instead to increase shareholder value.”
Let's examine this claim more closely:
A user experienced damage from a malfunctioning program. How would she get compensation from the manufacturer? Surely not by simply calling and announcing that a crash caused X dollars of damage. Surely the vendor would claim that it was a user error …. So user and vendor will end up in court. The only proof of fault on the vendor side would be for the user to
recreate the state of her machine before the crash (how??)
reproduce the software error by taking actions explicitly mentioned in the software's documentation.
Now suppose that there was a magical wand for taking snapshots of computer states just before crashes. Or that the legal system would permit claims on grounds of only the second part of the proof. Then there would be a strong positive incentive to write software that fails unreproducibly: “If our software's errors cannot be demonstrated reliably in court, we will never lose money in product liability cases.”
This introduces an interesting new paradigm of programming. Methods of this school of programming could include:
- Do something random
If an exception is raised which is not caused by user input, look for a random function/method which can be called in the current context and call that.
- Procrastination
In multithreaded programs, if one thread runs into an error, simply put this thread to sleep and hope nobody notices it.
- Decoy
Produce fake virus scanner alerts, telling the user to e.g. reboot imediately, thereby erasing the traces of the error.
- Blame someone else
Inject errors in other running programs.
Example: A SEGFAULT handler looks for other programs from different vendors running on the same machine when the error occurs and forwards the signal to one of them. It then simply waits. The user might attribute the freezing of the program to the crash of the other.Of course, really unreliable code needs randomness to select the action to take. All modern operating systems now come with random number generators which could be used for that purpose.
In machines with hardwire unique ids (UIDs), e.g. from the TPM, there is the interesting (and rewarding) possibility to tie the random behaviour to the hardware. This would allow software vendors to sell horoscopes for computers!
MSHoroscope
Tuesday, Serial numbers 0x900… to 0xA00…:
- Bad day for text processing
Fri, 11 Nov 2005
[/projects]
permanent link
