Ben Works

Idea: Using System Level Testing for Revealing SQL Injection-Related Error Message Information Leaks: Difference between revisions

← Older edit
 
(11 intermediate revisions by the same user not shown)
Line 99: Line 99:
# '''Record Result'''. We then marked each test that caused incorrect SQL operations or an application error in Step 5 as a successful attack and its corresponding SQL statement as a vulnerability.
# '''Record Result'''. We then marked each test that caused incorrect SQL operations or an application error in Step 5 as a successful attack and its corresponding SQL statement as a vulnerability.


== 7. References ==
Identifying hotspots may seem trivial, but in fact can be difficult because hotspots may not always take the same form. One way of discovering vulnerabilities is automated static analysis tools, which can be designed to check for a particular hotspot or vulnerability type<sup>[11]</sup>. We executed static analysis tools on our subjects and the tools reported no input validation vulnerabilities in any project we examined.
 
== 4. Results ==
 
This section presents the results of our case study. We first observed, as shown in Table 2, that there were no intrinsic JUnit test cases that used malicious input. We conducted all of our 272 system level tests by using HtmlUnit to inject our attack list into a request parameter, or in the case of TuduLists, to conduct an AJAX request where the malicious input was injected into an asynchronous JavaScript call. Using our technique, we found no instances of SQL injection vulnerabilities at the system level. No application allowed us to issue commands to the database management system because prepared statements and Hibernate both perform strong type checking on the variables used in their hotspots. Hibernate allows developers to create persistent classes in the object-oriented paradigm that represent individual database records<sup>[12]</sup>. However, we found 17 error message information leak vulnerabilities among the four applications in our case study, summarized in Table 2.
 
<center>'''Table 2. Results for the Test Subjects'''
{| class="wikitable"
|-
|'''Project'''
|'''iTrust'''
|'''Hispacta'''
|'''LogicServices'''
|'''TuduLists'''
|-
|'''Hotspots'''
| 92
| 23
| 48
| 13
|-
|'''Covered by Intrinsic Tests'''
| 89
| 20
| 47
| 3
|-
|'''Statement Coverage (EclEmma)'''
| 84%
| 49%
| 53%
| 40%
|-
|'''Test Cases with Malicious Input'''
| 0
| 0
| 0
| 0
|-
|'''New System Level Test Cases (Normal and Malicious)'''
| 149
| 29
| 80
| 14
|-
|'''Confirmed Vulnerabilities'''
| 2
| 2
| 9
| 4
|}
</center>
 
We found that unit testing could have identified none of the 17 confirmed vulnerabilities; rather, these confirmed vulnerabilities are system level vulnerabilities that had to involve the application server. A missing exception handler for pages or Servlets within Apache Tomcat caused each of the vulnerabilities we discovered. The example presented in Table 3 and Table 4 helps illustrate why these vulnerabilities cannot be exposed at the unit level.
 
Table 3 presents the relevant code from one of the confirmed vulnerabilities that we found in iTrust, in the file <code>editHCPs.jsp</code>. In other pages within iTrust, there is a JSP directive declared at the top of the page’s code (along with various navigational toolbars and headers) that declares an exception handler:
 
  <%@page errorPage="/auth/exceptionHandler.jsp"%>
 
This directive does not appear in editHCPs.jsp (see Table 3). At the moment an exception is thrown, Apache Tomcat forwards the user to the page declared in this directive, if this directive is declared. Otherwise, Apache Tomcat outputs a revealing stack trace to the user’s browser window, also known as an error message information leakage.
 
Since the omission of an exception handler is something that happens in the JSP code and not the Java code, some form of interaction is required with the application server (Apache Tomcat) in order to expose the vulnerabilities. One may view each JSP as a unit, but still the exception handler is a JSP page directive that involves a separate page; the unit therefore cannot be tested in isolation. The confirmed vulnerabilities, then, are caused by a system level error: the absence of an exception handler in the JSP or Servlet code of the application. Consider a JUnit test case that is written to execute <code>undeclareHCP</code> (see Table 4). This JUnit test case would pass, but would not expose the vulnerability even if it uses the some malicious input, such as <code>‘ UNION SELECT</code>. However, an HtmlUnit test case that targets <code>editHCPs.jsp</code> (see Table 3), produced by our system level testing technique, would expose the vulnerability using the same attack. That is, the vulnerability is not that an exception is thrown, but rather that the exception is not correctly handled by the JSP.
 
<center>'''Table 3. JSP for Example Vulnerability (editHCPs.jsp)'''</center>
 
  DeclareHCPAction action = // Action class for declaring the HCP.
  String confirm = ""; // used to store the result from the DAO.
  String removeHCP = request.getParameter("removeID");
  if(removeHCP!=null && !removeHCP.equals("")){
  confirm = action.undeclareHCP(removeHCP);
  }
  List<PersonnelBean> hcps = action.getDeclaredHCPS();
 
 
 
<center>'''Table 4. Java Method undeclareHCP'''</center>
 
  //given: patientDAO, a DAO pertaining to the patients table
  //given: iTrustException, a custom-build Exception class for
  // handling alternate flow errors
  public String undeclareHCP(String input) throws iTrustException {
  try {
  long hcpID = Long.valueOf(input);
  boolean confirm = patientDAO.undeclareHCP(loggedInMID, hcpID);
  if (confirm) {
  return "HCP successfully undeclared";
  } else
  return "HCP not undeclared";
  } catch (NumberFormatException e) {
  throw new iTrustException("HCP's MID not a number");
  } catch (DBException e) {
  throw new iTrustException(e.getMessage());
  }
  }
 
== 5. Limitations ==
 
Future case studies should examine much larger web applications than the ones in this study. In addition, the selective criteria as described in Section 3 could have biased the data. For example, perhaps the fact that all of our test subjects were Tomcat Servlet applications caused or prevented some security vulnerabilities that would not have been observable in another architectural setup. In addition, if stored procedures had been used in any of our test subjects, our results may have been different. The development teams for each project may have been using other testing techniques to improve the security posture of our subjects, or security may not have been high on their list of requirements.
 
The container for the applications (in this case, Apache Tomcat) could also be emulated using a Mock Object pattern<sup>[13]</sup>, and each individual servlet or JSP could be tested in isolation from one another. However, the quality of the testing results is entirely dependent on the quality of the mock object’s ability to emulate the server<sup>[13]</sup>; additionally, mock objects may not be any less expensive than system testing. Prepared statements, which separate the user’s input from the structure of the query at the application level<sup>[14]</sup>, protected the applications in this study. However, prepared statements are only useful if developers are aware of them and choose to use them. Our own system level procedure may not have exposed all vulnerabilities latent in the four subjects. Our procedure was targeted towards SQL injection vulnerabilities, which did not exist in these sampled applications at the locations of the hotspots we identified, but other vulnerabilities of varying types may exist in our subjects.
 
== 6. Conclusions ==
 
In our investigation of the relative effectiveness of unit and system level testing techniques, we have discovered that developers sometimes miss the fact that input validation is comprised of both ensuring that input falls within an acceptable range (e.g. “integer”) and that the application fails gracefully when input is not within said range. We found that all four of our study subjects use Hibernate and/or properly constructed prepared statements, which were completely effective for asserting that input falls within a safe (non-attack) range. Using a systematic system level security testing procedure to generate an HtmlUnit test suite, we found 17 error message information leakage vulnerabilities in the four web applications of our study. We found it impossible to replicate these same 17 vulnerabilities by augmenting the intrinsic unit test suites with additional malicious tests because vulnerabilities cannot be exposed at the system level though unit testing.
 
Our results show that ensuring that error messages resulting from SQL injection attacks do not reveal sensitive information is an inherently system level activity because the web server will dictate how and when error messages are displayed. ''Thus, an iterative, a feature-based development team conducting a test-driven automation practice can use a system level test procedure like the one described in this paper to expose both SQL injection vulnerabilities and error message information leak vulnerabilities.'' From a security perspective, unit testing would not be effective toward this aim, because it cannot take into account the production environment in which the system exists.
 
== 7. Acknowledgements ==
 
We would like to thank the North Carolina State University Realsearch group for their helpful comments on the paper. In addition, we would like to thank Yonghee Shin for the foundational work she performed by providing formal definitions for our SQL hotspot metrics and for her input on the content of this paper. This work is supported by the National Science Foundation under CAREER Grant No. 0346903. Any opinions expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
 
== 8. References ==


: <sup>[1]</sup> Halfond, W.G.J., Orso, A.: AMNESIA: analysis and monitoring for neutralizing SQL-injection attacks. 20th IEEE/ACM International Conference on Automated Software Engineering, Long Beach, CA, USA (2005) 174-183
: <sup>[1]</sup> Halfond, W.G.J., Orso, A.: AMNESIA: analysis and monitoring for neutralizing SQL-injection attacks. 20th IEEE/ACM International Conference on Automated Software Engineering, Long Beach, CA, USA (2005) 174-183
Line 116: Line 227:
: <sup>[14]</sup> Thomas, S., Williams, L.: Using automated fix generation to secure SQL statements. Proceedings of the Third International Workshop on Software Engineering for Secure Systems, Minneapolis, MN (2007)
: <sup>[14]</sup> Thomas, S., Williams, L.: Using automated fix generation to secure SQL statements. Proceedings of the Third International Workshop on Software Engineering for Secure Systems, Minneapolis, MN (2007)


== 8. End Notes ==
== 9. End Notes ==


# The CWE/SANS Top 25 can be found at http://cwe.mitre.org/top25/
# The CWE/SANS Top 25 can be found at http://cwe.mitre.org/top25/
Line 127: Line 238:
# The approach we propose in this paper tests the web application in the context of its server; a system level technique. However, our approach also targets specific areas (“hotspots”) of the web application; a unit level technique.  Thus, the way we use HtmlUnit in our case study is a hybrid of system level and unit level approaches, which is technically considered grey box testing <sup>[8, 9]</sup>.
# The approach we propose in this paper tests the web application in the context of its server; a system level technique. However, our approach also targets specific areas (“hotspots”) of the web application; a unit level technique.  Thus, the way we use HtmlUnit in our case study is a hybrid of system level and unit level approaches, which is technically considered grey box testing <sup>[8, 9]</sup>.
# http://www.mysql.com
# http://www.mysql.com
# http://www.hibernate.com
# http://www.hibernate.org/
# http://sourceforge.net
# http://sourceforge.net
# http://eclipse.org/webtools/
# http://eclipse.org/webtools/
Retrieved from "https://bw.kn1.us/wiki/index.php?title=Idea:_Using_System_Level_Testing_for_Revealing_SQL_Injection-Related_Error_Message_Information_Leaks"