Software Security and Testing

Software security is a discipline that is closely aligned with software testing. In fact, the field of security could learn quite a bit from software testing methods and philosophy. Much of what we traditionally place in the security realm, such as vulnerabilities like SQL injection, is actually nothing more than a software bug - a failure to validate user input or the correctness of a query before execution. Much of the time software security researchers poking at code are merely doing what software testers do, only with a slightly different mindset. Software security testers look for bugs that can lead to unsafe conditions, rather than simply trying to inject faults. However, there is much overlap in the tasks of a software security evaluation and software testing. Fault injection is one clear area where these two activities overlap.

How, then, is the job of a security analyst different from the job of a software tester? The most obvious divergence is the introduction of risk analysis in the job of a security analyst. While a software tester simply looks for anomalous behaviour, a security analyst looks for opportunities to use the software maliciously. Sometimes these malicious purposes are merely an operation of correct software operation. Consider, for example, a system that allows an attacker to send spam. The system might have the clearly stated functional purpose of allowing users to send e-mail and this functionality might operate correctly in all known circumstances. This is a situation where a software tester would certify the product, but a security tester would flag the product as having a specific vulnerability.

While carrying out traditional testing when evaluating software security is enormously valuable, the software security analyst should not be lulled into thinking that simply because the software does not contain any bugs that it is safe. Security is a broad and evolving field, and much of it has to do with mitigating risk, but there is also a healthy dose of anticipation. That is, much of software security involves correctly, and proactively, guessing what the bad guys will do. Black hat hackers will often seek ways to manipulate software that fall within the specification, but which produce malicious results. The examples of this sort of behaviour are too numerous to list, but some include directory traversal, information disclosure, and other actions when are purposely built into the system but which can be used to reach beyond the capabilities designers expected to assign to users.

Keeping the differences, and the similarities between software testing and security evaluation clear is important. It is tempting to be lulled into considering the intersection of the two as security, or the compliment. One must realize that both aspects of security are necessary for the proper evaluation of software.