VESARiA Network Security Specialists
About Vesaria Services Consulting Partners Research Customers Contact Us

Firewall Testing



How I test a firewall

Top-down design-oriented testing

The top-down design-oriented testing approach is simple: you start looking at the firewall from a very high level, see if it makes sense at that level, then look at it at increasingly lower levels until you've gotten to a point where you're pretty much trusting your components or you've pushed things as far as it's sensible to go. Determining how far is sensible is a difficult judgement, and it depends a lot on the application to which the firewall is being put. This is where I agree 100% with the Orange Book guys: if the firewall is protecting the launch console for an H-bomb, then it is entirely appropriate to not only review the high level design, but all the source code. Indeed, the source code for system library routines used, the compiler they were compiled with, the kernel itself, the processor it runs on, etc. For "normal" folks there has to be some happy medium and it's very dependent on what you have at stake. To determine the happy medium, keep things in perspective.

Keeping things in perspective

Many of the sites I have visitted have firewalls where the firewall is the only thing on their network that is remotely close to secured. They have an Internet link on the other side of the firewall, and 5 T1 lines to business partners, with little or no protection on those links, and the business partners in turn have links to their competitors, the Internet, and whatever else your worst nightmares can imagine. Often, a large, unspecified, percentage of the users have modems on their desktops and run PPP stacks to whoever they feel like. In such an environment, the firewall is the last thing that is likely to be broken into. It's still worth checking that the firewall works, is nailed down tightly, and that the firewall appears to have security properties that meet the requirements, and that are implemented correctly: but it's probably not worth worrying about a Ken Thompson trojan in the compiler the firewall was built with.

Postulating and testing for problems

First off, you need to know what you're testing, how it works, and why it works the way it does. To find this out, you need to interview the designers, or have access to very high-quality design documentation about the product. After all, if the high-level design makes no sense, you may not need to test any farther. Understanding the high-level design will permit you to guess where there may be weaknesses in the implementation, and what parts of the implementation are particularly critical to the functioning of the firewall. If the designers and design documentation do not appear to recognize these potential areas of weakness, and to have countered them, there may be a possibly fruitful area for penetration testing.

If the high level design is comprehensive and comprehensible, it should be easy enough to determine what the basic assumptions of the firewall are. From these basic assumptions, derive some simple tests. Some of the tests may be the kind that are incorporated in tools like SATAN: use such tools as directed testing engines and they're very effective. For example, if I am dealing with a firewall that blocks all traffic between two networks, I should be able to run a complete network maximum SATAN scan on a network behind the firewall, and it had better come up 100% dry.

Continue to postulate problems, based on what the designers of the firewall identify its properties as being. Another example would be a firewall that has no IP address. If the firewall has no IP address, then it should not ARP if I ARP it, it should not ping if I ping the network broadcast address - in other words it should act like it has no IP address.

Lastly, determine the protective relationships that the firewall implements: if the firewall passes some of the responsibility for security to hosts behind it, determine what those responsibilities are and how they are documented. For example, a screening router-type firewall may permit telnet through from the outside to specific hosts on the inside. If that is the case, those hosts may be vulnerable to holes in the vendor-supplied telnet daemon. The documentation should reflect this.

Top-down design-oriented firewall testing is, perforce, a rambling process. Essentially, you must play the role of a detective, re-creating the design path the engineers followed, and turning up any clues they missed.

But it's too hard, time-consuming, and expensive

The Orange Book approach was a commercial failure because of time-to-market and expense. Rigorous firewall testing suffers from the same kind of problems, if carried too far. It is too expertise-intensive as well; there are not enough real firewall experts in the industry to be brought together in one place to test firewalls, and most of them are either competitors or so insanely busy that getting a slice of their time is nearly impossible.

Vesaria, LLC

Ranum On Firewall Testing
Table of Contents

Previous Section: Let's imagine a firewall test programme

Next Section: Conclusions

Find out more about VESARiA Firewall Testing.

© 2000 - 2017 Vesaria Network Security Specialists        
   About Vesaria   |   Legal   |   Privacy   |   Contact