Idea: Using System Level Testing for Revealing SQL Injection-Related Error Message Information Leaks

B. Smith, L. Williams, A. Austin, "Idea: Using System Level Testing for Revealing SQL Injection-Related Error Message Information Leaks", Lecture Notes in Computer Science, vol. 5965, Engineering Secure Software and Systems (ESSoS 2010), pp. 192-200, 2010.

Abstract

Completely handling SQL injection consists of two activities: properly protecting the system from malicious input, and preventing any resultant error messages caused by SQL injection from revealing sensitive information. The goal of this research is to assess the relative effectiveness of unit and system level testing of web applications to reveal both error message information leak and SQL injection vulnerabilities. To produce 100% test coverage of 176 SQL statements in four open source web applications, we augmented the original automated unit test cases with our own system level tests that use both normal input and 132 forms of malicious input. Although we discovered no SQL injection vulnerabilities, we exposed 17 error message information leak vulnerabilities associated with SQL statements using system level testing. Our results suggest that security testers who use an iterative, test-driven development process should compose system level rather than unit level tests.

1. Introduction

In this paper, we examine two input validation vulnerabilities that are in the CWE/SANS Top 25 Most Dangerous Programming Errors¹ due to their prevalence and potential damage: SQL injection vulnerabilities and error message information leak vulnerabilities. SQL injection vulnerabilities occur when a lack of input validation could allow a user to force unintended system behavior by altering the logical structure of a SQL statement using SQL reserved words and special characters^{[1, 2]}. The CWE categorizes SQL injection vulnerabilities as a subset of input validation vulnerabilities, which occur when a system does not assert that input falls within an acceptable range, allowing the system to be exploited to perform unintended functionality^[3]. Error message information leak vulnerabilities are caused when an application does not correctly handle an exceptional condition and, as a result, sensitive information is revealed to the attacker^{[4, 5]}. We contend that in web applications, where security is paramount, input validation is comprised of both ensuring that input falls within an acceptable range (e.g. “integer”) and that the application fails gracefully when input is not within said range.

To expose and mitigate SQL injection vulnerabilities at the white box level, a development team can execute unit tests that assert that malicious input is rejected by the components that communicate with the database^[6]. In some development methodologies, components are constructed in horizontal slices that emanate from the ground up—the components that perform logic and interact with the database are composed and tested long before the user interface. However, in an iterative development methodology, teams build software on a feature-by-feature basis in vertical slices that extend from the database to the user interface. Additionally, test-driven development implies the incremental creation of tests throughout the development process^[7].

The goal of this research is to assess the relative effectiveness of system and unit level testing of web applications to reveal both SQL injection vulnerabilities and error message information leakage vulnerabilities when used with an iterative test automation practice by a feature development team. We conducted a case study on four Java-based open source web applications: iTrust², Hispacta³, LogicServices⁴, and TuduLists⁵. In our case study, we executed and compared JUnit⁶ unit tests and HtmlUnit⁷ system level tests. The purpose of this study is to determine whether system level testing⁸ could be used in an iterative or test-driven development scenario to expose both parts of input validation earlier in the lifecycle—an important component of building security in from the beginning^[8].

The rest of this paper is organized as follows. Section 2 presents the required background for understanding our study procedure. After that, Section 3 describes the case study, including the subject applications and experimental setup. Next, Section 4 presents the results of our case study. Section 5 presents limitations of the study. Finally, Section 6 describes the conclusions we reached from our study.

2. Background

In this section, we demonstrate an example of both a SQL injection vulnerability and discuss error information leakage vulnerabilities.

SQL Injection Vulnerabilities. Consider a Java method used for the deletion of a patient’s information in a medical record system. We present the relevant source code for this operation in Figure 1 (assume that patients are deleted by their names). The vulnerability in this example was introduced in the line defining the SQL statement. The example we have presented in Figure 1 performs no input validation and, as a result, the example contains a SQL injection vulnerability relative to the use of the name parameter. An attacker could cause change to the interpretation of the SQL query by entering the SQL command fragment “‘ OR TRUE --“ in the input field instead of any valid user name in the web form.

The single quotation mark (‘) indicates to the SQL parser that the character sequence for the username column is closed, the fragment OR TRUE is interpreted as always true, and the fragment of the query after the hyphens (--) is a comment. The altered WHERE clause of the SQL statement will be interpreted as always true and thus every patient is deleted from the table. Because no input validation was performed, the attacker can exploit the system by inserting the malicious input in the name field, and causing truncation of the Patients table. Thus, the bolded statement in Figure 1 is an example of a SQL injection hotspot (or just “hotspot” in this paper)—any source code location that may contain a SQL injection vulnerability^{[1, 2]}.

 ... 
 
 java.sql.Connection mySQLConnector = DriverManager.getConnection(); 
 java.sql.Statement s = mySQLConnector.createStatement("DELETE FROM Patients WHERE Name = ‘" + name + “’;”); 
 int result = s.executeUpdate(); 
  return 1 == result; 
 
 ... 

Figure 1. Patient Deletion Code in Java; hotspot is bolded

Error message information leak vulnerabilities. These vulnerabilities occur when an application does not correctly handle exceptional conditions and subsequently leaks sensitive information to a user^{[4, 5]}. This information can be obviously dangerous in the case of error messages that contain system or application passwords, or it may seem more benign, containing only version numbers or stack traces. Unfortunately, even these seemingly benign error information leaks can provide valuable information to an attacker and could expose additional attack vectors. Since a tester cannot tell what information an attacker needs to conduct future attacks, a good policy is to treat all error information leakage vulnerabilities as if they contain obviously dangerous information such as passwords.

3. Case Study

In this section, we present information about our case study. Each part of the case study was conducted using Eclipse v3.3 Europa executed using Java v1.6 running on an IBM Lenovo T61p running Windows Vista Ultimate with a 2.40Ghz Intel Core Duo processor and 2GB of RAM. We used MySQL v5.0.45-community-nt for our research database management system⁹.

To obtain our case study applications, we collected information about 12 enterprise Java web applications, which we found by searching SourceForge¹¹ with the query “Java web application,” and sampling the first 12 projects that contained the Eclipse webtools¹² project file structure. We then rejected eight subjects from our study because they did not meet one or many of the following criteria:

• Could be compiled, built, and deployed.
• Contained automated unit tests, written in JUnit, which were distributed with the source code.
• Relied upon a relational DBMS to store its data.

We were left with the four subjects presented in Table 1. In an attempt to reveal both SQL injection and error message information leak vulnerabilities in our test subjects, as stated in Section 1, we created the following systematic, system level, security testing procedure and executed it on our subjects. By design, this procedure produces an automated system level test suite that executes all reachable hotspots with normal and malicious input. We note here that by intrinsic, we mean that we did not augment or modify the existing test set in any way; values for these measures were achieved by the unit tests that were distributed with each system.

Table 1. Information about the Test Subjects (n=4)

Project	iTrust	Hispacta	LogicServices	TuduLists
Version	4.0	0.0.3	1.8	2.2
Lines of Codeδ	7707	1991	5011	6178
Production Classes*	143	42	155	132
Databases Classes	20	4	1	5
Hibernate10	No	Yes	Yes	Yes

1. Identify and Instrument Hotspots. We manually inspected the source code to discover any point where the system interacts with the database. We note here that hotspots can take many forms; we explain this issue more below. We have written the Java program SQLMarker, introduced in our earlier work^[9]. SQLMarker keeps a record of the execution state at runtime for each uniquely identified hotspot¹³. SQLMarker has a method, SQLMarker.mark(), which passes the line number and file name to a research database that stores whether the hotspot has been executed.
2. Record Hotspots. A second class we wrote, called Instrumenter, provides each manually marked hotspot with a unique identifier comprised of the filename and line number, and outputs the number of hotspots found. Once we manually marked each, we executed Instrumenter to store a record of each of these hotspots.
3. Execute Original Unit Tests. After instrumenting each subject to mark its executed SQL hotspots, we executed the intrinsic unit tests and recorded the resultant number of executed statements.
4. Create Test Cases. We used the stored file name and line number of the hotspot from Step 1 to construct an automated system level test with HtmlUnit¹⁴ that executed the SQL statement located at the stored file name and line number. We constructed an initial automated test for each hotspot by using a call hierarchy and manual testing to make web requests until the hotspot was marked as being executed and then modeled our automated test after the use case we discovered¹⁵.
5. Apply Malicious Input. We modified the test defined in Step 4 to emulate a malicious user by using 132 forms of malicious input in an attack list from NeuroFuzz¹⁶ in place of normal input. This part of the procedure is similar to “fuzzing”. The difference here is that fuzzing is a semi-random, black box activity; our approach is targeted to specifically attack the areas where user input might reach a hotspot.
6. Record Result. We then marked each test that caused incorrect SQL operations or an application error in Step 5 as a successful attack and its corresponding SQL statement as a vulnerability.

7. References

^[1] Halfond, W.G.J., Orso, A.: AMNESIA: analysis and monitoring for neutralizing SQL-injection attacks. 20th IEEE/ACM International Conference on Automated Software Engineering, Long Beach, CA, USA (2005) 174-183

^[2] Kosuga, Y., Kono, K., Hanaoka, M., Hishiyama, M., Takahama, Y.: Sania: syntactic and semantic analysis for automated testing against SQL injection. 23rd Annual Computer Security Applications Conference, Miami Beach, FL (2007) 107-117

^[3] Pietraszek, T., Berghe, C.V.: Defending against injection attacks through context-sensitive string evaluation. Recent Advances in Intrusion Detection (RAID), Vol. 3858 / 2006. Springer, Berlin, Germany (2006) 124-145

^[4] Aslam, T., Krsul, I., Spafford, E.: Use of a taxonomy of security faults. 19th National Information Systems Security Conference, Baltimore, MD (1996) 551-560

^[5] Tsipenyuk, K., Chess, B., McGraw, G.: Seven pernicious kingdoms: a taxonomy of software security errors. IEEE Security & Privacy 3 (2005) 81-84

^[6] IEEE: IEEE Standard 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology. (1990)

^[7] Beck, K.: Test-driven development: By example. Addison-Wesley, Boston, MA, USA (2003)

^[8] McGraw, G.: Software security: Building security in. Addison-Wesley, Upper Saddle River, NJ (2006)

^[9] Smith, B., Shin, Y., Williams, L.: Proposing SQL statement coverage metrics. The 4th International Workshop on Software Engineering for Secure Systems at the 30th International Conference on Software Engineering, Leipzig, Germany (2008) 49-56

^[10] Jiang, Y., Cukic, B., Menzies, T.: Fault Prediction using Early Lifecycle Data. Software Reliability, 2007. ISSRE '07. The 18th IEEE International Symposium on (2007) 237-246

^[11] Livshits, V.B., Lam, M.S.: Finding security vulnerabilities in Java applications with static analysis. USENIX Security Symposium, Baltimore, MD (2005) 18-18

^[12] Bauer, C., King, G.: Hibernate in Action. Manning Publications (2004)

^[13] Brown, M., Tapolcsanyi, E.: Mock object patterns. The 10th Conference on Pattern Languages of Programs, Monticello, USA (2003)

^[14] Thomas, S., Williams, L.: Using automated fix generation to secure SQL statements. Proceedings of the Third International Workshop on Software Engineering for Secure Systems, Minneapolis, MN (2007)

8. End Notes

1. The CWE/SANS Top 25 can be found at http://cwe.mitre.org/top25/
2. http://sourceforge.net/projects/itrust/?source=directory
3. http://sourceforge.net/projects/hispacta/?source=directory
4. http://sourceforge.net/projects/logicservice/?source=directory
5. http://sourceforge.net/projects/tudu/?source=directory
6. http://www.junit.org
7. http://sourceforge.net/projects/htmlunit/?source=directory
8. The approach we propose in this paper tests the web application in the context of its server; a system level technique. However, our approach also targets specific areas (“hotspots”) of the web application; a unit level technique. Thus, the way we use HtmlUnit in our case study is a hybrid of system level and unit level approaches, which is technically considered grey box testing ^{[8, 9]}.
9. http://www.mysql.com
10. http://www.hibernate.com
11. http://sourceforge.net
12. http://eclipse.org/webtools/
13. For larger applications, one could use a static analyzer to determine hotspots’ locations.
14. HtmlUnit is a "GUI-Less browser for Java programs". It models HTML documents and provides an API that allows you to invoke pages, fill out forms, click links, etc, just like you do in your "normal" browser. http://htmlunit.sourceforge.net.
15. However, some hotspots were not used by the JSPs in the application, perhaps because these hotspots were used for database administration only, or the development team had not finished implementing the use case that required the query. If we could not reach the SQL statement through the web interface, we augmented the white box test plan to include a malicious test that directly calls the database class.
16. http://www.neurofuzz.com/modules/wfdownloads/singlefile.php?cid=2&lid=9