Idea: Using System Level Testing for Revealing SQL Injection-Related Error Message Information Leaks

B. Smith, L. Williams, A. Austin, "Idea: Using System Level Testing for Revealing SQL Injection-Related Error Message Information Leaks", Lecture Notes in Computer Science, vol. 5965, Engineering Secure Software and Systems (ESSoS 2010), pp. 192-200, 2010.

Abstract

Completely handling SQL injection consists of two activities: properly protecting the system from malicious input, and preventing any resultant error messages caused by SQL injection from revealing sensitive information. The goal of this research is to assess the relative effectiveness of unit and system level testing of web applications to reveal both error message information leak and SQL injection vulnerabilities. To produce 100% test coverage of 176 SQL statements in four open source web applications, we augmented the original automated unit test cases with our own system level tests that use both normal input and 132 forms of malicious input. Although we discovered no SQL injection vulnerabilities, we exposed 17 error message information leak vulnerabilities associated with SQL statements using system level testing. Our results suggest that security testers who use an iterative, test-driven development process should compose system level rather than unit level tests.

1. Introduction

In this paper, we examine two input validation vulnerabilities that are in the CWE/SANS Top 25 Most Dangerous Programming Errors¹ due to their prevalence and potential damage: SQL injection vulnerabilities and error message information leak vulnerabilities. SQL injection vulnerabilities occur when a lack of input validation could allow a user to force unintended system behavior by altering the logical structure of a SQL statement using SQL reserved words and special characters^{[1, 2]}. The CWE categorizes SQL injection vulnerabilities as a subset of input validation vulnerabilities, which occur when a system does not assert that input falls within an acceptable range, allowing the system to be exploited to perform unintended functionality^[3]. Error message information leak vulnerabilities are caused when an application does not correctly handle an exceptional condition and, as a result, sensitive information is revealed to the attacker^{[4, 5]}. We contend that in web applications, where security is paramount, input validation is comprised of both ensuring that input falls within an acceptable range (e.g. “integer”) and that the application fails gracefully when input is not within said range.

To expose and mitigate SQL injection vulnerabilities at the white box level, a development team can execute unit tests that assert that malicious input is rejected by the components that communicate with the database^[6]. In some development methodologies, components are constructed in horizontal slices that emanate from the ground up—the components that perform logic and interact with the database are composed and tested long before the user interface. However, in an iterative development methodology, teams build software on a feature-by-feature basis in vertical slices that extend from the database to the user interface. Additionally, test-driven development implies the incremental creation of tests throughout the development process^[7].

The goal of this research is to assess the relative effectiveness of system and unit level testing of web applications to reveal both SQL injection vulnerabilities and error message information leakage vulnerabilities when used with an iterative test automation practice by a feature development team. We conducted a case study on four Java-based open source web applications: iTrust², Hispacta³, LogicServices⁴, and TuduLists⁵. In our case study, we executed and compared JUnit⁶ unit tests and HtmlUnit⁷ system level tests. The purpose of this study is to determine whether system level testing⁸ could be used in an iterative or test-driven development scenario to expose both parts of input validation earlier in the lifecycle—an important component of building security in from the beginning^[8].

The rest of this paper is organized as follows. Section 2 presents the required background for understanding our study procedure. After that, Section 3 describes the case study, including the subject applications and experimental setup. Next, Section 4 presents the results of our case study. Section 5 presents limitations of the study. Finally, Section 6 describes the conclusions we reached from our study.

2. Background

In this section, we demonstrate an example of both a SQL injection vulnerability and discuss error information leakage vulnerabilities.

SQL Injection Vulnerabilities. Consider a Java method used for the deletion of a patient’s information in a medical record system. We present the relevant source code for this operation in Figure 1 (assume that patients are deleted by their names). The vulnerability in this example was introduced in the line defining the SQL statement. The example we have presented in Figure 1 performs no input validation and, as a result, the example contains a SQL injection vulnerability relative to the use of the name parameter. An attacker could cause change to the interpretation of the SQL query by entering the SQL command fragment “‘ OR TRUE --“ in the input field instead of any valid user name in the web form.

The single quotation mark (‘) indicates to the SQL parser that the character sequence for the username column is closed, the fragment OR TRUE is interpreted as always true, and the fragment of the query after the hyphens (--) is a comment. The altered WHERE clause of the SQL statement will be interpreted as always true and thus every patient is deleted from the table. Because no input validation was performed, the attacker can exploit the system by inserting the malicious input in the name field, and causing truncation of the Patients table. Thus, the bolded statement in Figure 1 is an example of a SQL injection hotspot (or just “hotspot” in this paper)—any source code location that may contain a SQL injection vulnerability^{[1, 2]}.

Error message information leak vulnerabilities. These vulnerabilities occur when an application does not correctly handle exceptional conditions and subsequently leaks sensitive information to a user^{[4, 5]}. This information can be obviously dangerous in the case of error messages that contain system or application passwords, or it may seem more benign, containing only version numbers or stack traces. Unfortunately, even these seemingly benign error information leaks can provide valuable information to an attacker and could expose additional attack vectors. Since a tester cannot tell what information an attacker needs to conduct future attacks, a good policy is to treat all error information leakage vulnerabilities as if they contain obviously dangerous information such as passwords.

7. References

^[1] Halfond, W.G.J., Orso, A.: AMNESIA: analysis and monitoring for neutralizing SQL-injection attacks. 20th IEEE/ACM International Conference on Automated Software Engineering, Long Beach, CA, USA (2005) 174-183

^[2] Kosuga, Y., Kono, K., Hanaoka, M., Hishiyama, M., Takahama, Y.: Sania: syntactic and semantic analysis for automated testing against SQL injection. 23rd Annual Computer Security Applications Conference, Miami Beach, FL (2007) 107-117

^[3] Pietraszek, T., Berghe, C.V.: Defending against injection attacks through context-sensitive string evaluation. Recent Advances in Intrusion Detection (RAID), Vol. 3858 / 2006. Springer, Berlin, Germany (2006) 124-145

^[4] Aslam, T., Krsul, I., Spafford, E.: Use of a taxonomy of security faults. 19th National Information Systems Security Conference, Baltimore, MD (1996) 551-560

^[5] Tsipenyuk, K., Chess, B., McGraw, G.: Seven pernicious kingdoms: a taxonomy of software security errors. IEEE Security & Privacy 3 (2005) 81-84

^[6] IEEE: IEEE Standard 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology. (1990)

^[7] Beck, K.: Test-driven development: By example. Addison-Wesley, Boston, MA, USA (2003)

^[8] McGraw, G.: Software security: Building security in. Addison-Wesley, Upper Saddle River, NJ (2006)

^[9] Smith, B., Shin, Y., Williams, L.: Proposing SQL statement coverage metrics. The 4th International Workshop on Software Engineering for Secure Systems at the 30th International Conference on Software Engineering, Leipzig, Germany (2008) 49-56

^[10] Jiang, Y., Cukic, B., Menzies, T.: Fault Prediction using Early Lifecycle Data. Software Reliability, 2007. ISSRE '07. The 18th IEEE International Symposium on (2007) 237-246

^[11] Livshits, V.B., Lam, M.S.: Finding security vulnerabilities in Java applications with static analysis. USENIX Security Symposium, Baltimore, MD (2005) 18-18

^[12] Bauer, C., King, G.: Hibernate in Action. Manning Publications (2004)

^[13] Brown, M., Tapolcsanyi, E.: Mock object patterns. The 10th Conference on Pattern Languages of Programs, Monticello, USA (2003)

^[14] Thomas, S., Williams, L.: Using automated fix generation to secure SQL statements. Proceedings of the Third International Workshop on Software Engineering for Secure Systems, Minneapolis, MN (2007)

8. End Notes

1. The CWE/SANS Top 25 can be found at http://cwe.mitre.org/top25/
2. http://sourceforge.net/projects/itrust/?source=directory
3. http://sourceforge.net/projects/hispacta/?source=directory
4. http://sourceforge.net/projects/logicservice/?source=directory
5. http://sourceforge.net/projects/tudu/?source=directory
6. http://www.junit.org
7. http://sourceforge.net/projects/htmlunit/?source=directory
8. The approach we propose in this paper tests the web application in the context of its server; a system level technique. However, our approach also targets specific areas (“hotspots”) of the web application; a unit level technique. Thus, the way we use HtmlUnit in our case study is a hybrid of system level and unit level approaches, which is technically considered grey box testing ^{[8, 9]}.