Members of the security and safety community often claim
that software quality would improve if manufacturers would
be held liable for damages caused by their products.
The reasoning uses the negative incentive argument:
“If we produce faulty software, we will lose money.
Let's write correct software instead to increase shareholder
Let's examine this claim more closely:
A user experienced damage from a malfunctioning
program. How would she get compensation from the manufacturer?
Surely not by simply calling and announcing that a crash caused
X dollars of damage. Surely the vendor would claim that it
was a user error …. So user and vendor will end up in court.
The only proof of fault on the vendor side would be for the user to
- recreate the state of her machine before the crash (how??)
- reproduce the software error by taking actions explicitly mentioned in
the software's documentation.
Now suppose that there was a magical wand for taking snapshots of
computer states just before crashes. Or that the legal system
would permit claims on grounds of only the second part of the proof.
Then there would be a strong positive incentive to write software
that fails unreproducibly: “If our software's errors cannot
be demonstrated reliably in court, we will never lose money in
product liability cases.”
This introduces an interesting new paradigm of programming.
Methods of this school of programming could include:
- Do something random
- If an exception is raised which is not caused by
user input, look for a random function/method which can be called
in the current context and call that.
- In multithreaded programs, if one thread runs into an error,
simply put this thread to sleep and hope nobody notices it.
- Produce fake virus scanner alerts, telling the user to e.g.
reboot imediately, thereby erasing the traces of the error.
- Blame someone else
Inject errors in other running programs.
Example: A SEGFAULT handler looks for other programs from
different vendors running on the same machine when the error
occurs and forwards the signal to one of them. It then simply
waits. The user might attribute the freezing of the program
to the crash of the other.
Of course, really unreliable code needs randomness to select the action
to take. All modern operating systems now come with random
number generators which could be used for that purpose.
In machines with hardwire unique ids (UIDs), e.g. from the TPM,
there is the interesting (and rewarding) possibility to tie the
random behaviour to the hardware. This would allow
software vendors to sell horoscopes for computers!
- Tuesday, Serial numbers 0x900… to 0xA00…:
- Bad day for text processing