Idea: Using System Level Testing for Revealing SQL Injection-Related Error Message Information Leaks: Difference between revisions
Programsam (talk | contribs) |
Programsam (talk | contribs) |
||
| (3 intermediate revisions by the same user not shown) | |||
| Line 194: | Line 194: | ||
} | } | ||
== 7. References == | == 5. Limitations == | ||
Future case studies should examine much larger web applications than the ones in this study. In addition, the selective criteria as described in Section 3 could have biased the data. For example, perhaps the fact that all of our test subjects were Tomcat Servlet applications caused or prevented some security vulnerabilities that would not have been observable in another architectural setup. In addition, if stored procedures had been used in any of our test subjects, our results may have been different. The development teams for each project may have been using other testing techniques to improve the security posture of our subjects, or security may not have been high on their list of requirements. | |||
The container for the applications (in this case, Apache Tomcat) could also be emulated using a Mock Object pattern<sup>[13]</sup>, and each individual servlet or JSP could be tested in isolation from one another. However, the quality of the testing results is entirely dependent on the quality of the mock object’s ability to emulate the server<sup>[13]</sup>; additionally, mock objects may not be any less expensive than system testing. Prepared statements, which separate the user’s input from the structure of the query at the application level<sup>[14]</sup>, protected the applications in this study. However, prepared statements are only useful if developers are aware of them and choose to use them. Our own system level procedure may not have exposed all vulnerabilities latent in the four subjects. Our procedure was targeted towards SQL injection vulnerabilities, which did not exist in these sampled applications at the locations of the hotspots we identified, but other vulnerabilities of varying types may exist in our subjects. | |||
== 6. Conclusions == | |||
In our investigation of the relative effectiveness of unit and system level testing techniques, we have discovered that developers sometimes miss the fact that input validation is comprised of both ensuring that input falls within an acceptable range (e.g. “integer”) and that the application fails gracefully when input is not within said range. We found that all four of our study subjects use Hibernate and/or properly constructed prepared statements, which were completely effective for asserting that input falls within a safe (non-attack) range. Using a systematic system level security testing procedure to generate an HtmlUnit test suite, we found 17 error message information leakage vulnerabilities in the four web applications of our study. We found it impossible to replicate these same 17 vulnerabilities by augmenting the intrinsic unit test suites with additional malicious tests because vulnerabilities cannot be exposed at the system level though unit testing. | |||
Our results show that ensuring that error messages resulting from SQL injection attacks do not reveal sensitive information is an inherently system level activity because the web server will dictate how and when error messages are displayed. ''Thus, an iterative, a feature-based development team conducting a test-driven automation practice can use a system level test procedure like the one described in this paper to expose both SQL injection vulnerabilities and error message information leak vulnerabilities.'' From a security perspective, unit testing would not be effective toward this aim, because it cannot take into account the production environment in which the system exists. | |||
== 7. Acknowledgements == | |||
We would like to thank the North Carolina State University Realsearch group for their helpful comments on the paper. In addition, we would like to thank Yonghee Shin for the foundational work she performed by providing formal definitions for our SQL hotspot metrics and for her input on the content of this paper. This work is supported by the National Science Foundation under CAREER Grant No. 0346903. Any opinions expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. | |||
== 8. References == | |||
: <sup>[1]</sup> Halfond, W.G.J., Orso, A.: AMNESIA: analysis and monitoring for neutralizing SQL-injection attacks. 20th IEEE/ACM International Conference on Automated Software Engineering, Long Beach, CA, USA (2005) 174-183 | : <sup>[1]</sup> Halfond, W.G.J., Orso, A.: AMNESIA: analysis and monitoring for neutralizing SQL-injection attacks. 20th IEEE/ACM International Conference on Automated Software Engineering, Long Beach, CA, USA (2005) 174-183 | ||
| Line 211: | Line 227: | ||
: <sup>[14]</sup> Thomas, S., Williams, L.: Using automated fix generation to secure SQL statements. Proceedings of the Third International Workshop on Software Engineering for Secure Systems, Minneapolis, MN (2007) | : <sup>[14]</sup> Thomas, S., Williams, L.: Using automated fix generation to secure SQL statements. Proceedings of the Third International Workshop on Software Engineering for Secure Systems, Minneapolis, MN (2007) | ||
== | == 9. End Notes == | ||
# The CWE/SANS Top 25 can be found at http://cwe.mitre.org/top25/ | # The CWE/SANS Top 25 can be found at http://cwe.mitre.org/top25/ | ||