Integrating security into QA testing
I wrote about this some time ago.Fortify are now doing a webinar on this topic.
February 18, 2008 | Filed Under Testing | Leave a Comment
Security Testing with Watij
Watij is a tool designed for functional web testing. It’s effectively a Java API which drives an instance of Internet Explorer. You can then use your favourite unit testing framework to structure tests and make assertions of the results. Like similar functional testing tools, watij can be used to script security defects in web applications. This is inline with good security and development practice of writing abuse cases and specifically testing for failure conditions rather than only testing for success conditions.I wrote a paper (pdf) on using functional testing tools in this way, some time ago, and since then I’ve been having a look at the various tools to find one most suitable for security testing. Today, I whipped up some simple tests of the owasp.org website using Watij. It was easy to install and be up and running in Eclipse in a few minutes. Before starting on the tests, I wanted to define common functions, such as login and logout:
public static void login(String username, String password) throws Exception {
ie.navigate(“https://www.owasp.org/index.php?title=Special:Userlogin&returnto=Main_Page”);
ie.textField(name,“wpName”).set(username);
ie.textField(name,“wpPassword”).set(password);
ie.button(name, “wpLoginattempt”).click();
}
public static void logout() throws Exception {
ie.navigate(“https://www.owasp.org/index.php?title=Special:Userlogout&returnto=Special:Userlogin”);
}
//Run only once to initialise the browser
@BeforeClass
public static void init() throws Exception {
ie = new IE();
ie.start(“https://www.owasp.org/index.php/Main_Page”);
if (ie.containsText(“There is a problem with this website’s security certificate”)) {
ie.link(name,“overridelink”).click();
}
if (!ie.url().equals(“https://www.owasp.org/index.php/Main_Page”)) throw new RuntimeException(“Error getting to the front page!”);
webapp = new WebApp(ie);
}
@Test
public void testLoginLogout() throws Exception {
webapp.login(username, password);
assertTrue(“Logged in.”, ie.containsText(“You are now logged in”));
webapp.logout();
assertTrue(“Logged out.”, ie.containsText(“You are now logged out”));
}
@Test
public void testUserEnumeration() throws Exception {
webapp.login(“notauser”,“test”);
if (ie.containsText(“There is no user by the name”)) {
fail(“Usernames can be enumerated through the login error messages”);
}
}
@Test
public void testBruteForce() throws Exception {
int count=5;
for (int i=0;i<count;i++) {
webapp.login(username,“wrongpassword”);
}
webapp.login(username,password);
if (ie.containsText(“You are now logged in”)) {
fail(“After “+count+” incorrect login attempts, the user could still login. This exposes the accounts to brute force attacks.”);
webapp.logout();
}
}
Quite slick. But say I’d like to test whether a user who’s not logged in can post articles to the wiki. Well the “Edit” page is only displayed when you are logged in, so there’s no way I can click on a non-existent link. And just because the link isn’t there doesn’t necessarily mean that the server won’t accept the request. So what we need to do is perform the POST request to submit an article to the wiki directly and this is where watij stumbles. There’s no easy way to create an arbitrary POST request. The recommended approach is to use something like HTTPUnit or HtmlUnit to perform this specific test. Ok, no problem, so I can just use the login function I’ve already defined in the watij tests, then read the session cookie and pass the cookie onto an HTTPunit test, right? Wrong. You can’t read IE cookies from the watij API! So the only option is to re-code the login function in HttpUnit and use that instead. Lots of code duplication and now I’m mixing two different testing frameworks, not ideal.
To summarise the issues with using watij to test security:
- You can’t make raw POST requests without there first being a form to post. This is problematic because a lot of security tests are really aimed at testing the server side, not the client. You could augment watir with some Java code, or HttpUnit or HtmlUnit to perform the post request…
- It doesn’t appear to be possible to read IE’s cookies from watij! This is really annoying, because now I can’t reuse the login and logout function and simply pass the session cookie over to httpunit. I’d have to re-write the login function using httpunit - a whole lot of extra and messy work.
- Since it drives a real instance of IE, you have to trick it into ignoring client side validation or client side access control tests. (Of course client side security is no security at all)
- Tests can be quite slow, as watij waits for the entire page to load into the browser before continuing (turning off images and using ad blockers can help).
On the whole, I can see the attraction for using watij to positively test functionality - but because of the reasons above, I don’t think it’s the best choice for performing negative testing (i.e. test what’s not there). I hope to write a similar entry about Canoo’s WebTest, which provides similar testing functionality, but using XML tests and it’s own HTTP and Html engines.
Defining security requirements
The vast majority of security assessments I’ve worked on have used “best practice” to define the security requirements of the application under test. Clients are content to rely on security assessment firms to decide what should and shouldn’t be tested, and this is usually OK for bug hunting. But apart from the well known bugs which constitute security vulnerabilities there is an often overlooked class of security issues: lack of, or poorly implemented security features, such as:- No protection from brute force attacks against the login credentials
- a lack of cache-controls to prevent caching of sensitive content
- a failure in the access controls to prevent unauthorised users from viewing sensitive data
- etc.
January 12, 2008 | Filed Under Design | Leave a Comment