Security Testing of Custom Software Applications

Tools and techniques for testing applications - excerpted from Secure and Resilient Software Development by Mark Merkow and Lakshmikanth Raghavan

By Mark Mekow and Lakshmikanth Raghavan

July 28, 2010 — CSO —

Excerpted from Secure and Resilient Software Development by Mark S. Merkow and Lakshmikanth Raghavan and published by CRC Press. © 2010 Taylor and Francis Group, LLC. Reprinted with permission.

In this chapter, the authors detail the code review process, key features of automated source code scanning tools, and more.

In Chapters 6 and 7 we examined specific techniques and approaches to developing resilient software for a variety of platforms and specialized applications with a focus on preventing the most common errors and problems that lead to security incidents and data losses.

In Chapter 8 we'll begin exploring how to test the resilience of custom application code and find ways to further improve it. Topics covered in Chapter 8 include:

  • The true costs of waiting to find and eradicate software flaws
  • Manual and automated source code review techniques
  • Implementing code analysis tools
  • Penetration testing
  • Black box testing
  • Quality assurance testing

8.1 Fixing Early Versus Fixing After Release

A study by Gartner, IBM, and The National Institute of Standards and Technology (NIST) revealed that "the cost of removing an application security vulnerability during the design/development phase ranges from 30-60 times less than if removed during production."1 The key objective of integrating security processes with the software development life cycle (SDLC) is to ensure that we detect and fix security vulnerabilities early.

Also see "Source code analysis tools: How to choose and use them" on CSOonline.com


Many organizations simply do not know the costs of finding and fixing software defects, because they do not track or measure that work. If they did, they might be shocked to learn the real costs of developing software. There are direct and indirect costs to finding and fixing security bugs. If a vulnerability is found and exploited in a production application, the brand damage that results cannot be easily measured or repaired.

There are direct costs that we can certainly measure. One of the easiest to measure is the average cost to code a fix:

Average cost to code a fix = (number of developer man-days * cost per man-day) ÷ number of defects fixed

Apart from this cost, there are additional costs we need to consider:

  • System test costs
  • Implementation costs
  • System costs
  • Postproduction costs
  • Other costs, such as project management, documentation, downtime costs, etc.

These costs can skyrocket when a mission-critical or high-profile application is involved, and changes to it must not interfere or even be seen by customers using the application over the Internet—e.g. an e-banking site.

Therefore, it is far more sensible for enterprises to find and fix application software defects before they are released into the production environment. While threat modeling and design and architecture reviews can help to assure that there are no high-level defects at the design level, security testing ensures that there are no defects when implementing that secure design.

RESOURCE CENTER