Proposing SQL Statement Coverage Metrics - Revision history

Programsam at 14:46, 16 May 2021

2021-05-16T14:46:31Z

← Older revision		Revision as of 14:46, 16 May 2021
Line 1:		Line 1:
	B. Smith, Y. Shin, and L. Williams, "Proposing SQL Statement Coverage Metrics", Proceedings of the Fourth International Workshop on Software Engineering for Secure Systems (SESS 2008), co-located with ICSE, pp. 49-56, 2008.		B. Smith, Y. Shin, and L. Williams, "Proposing SQL Statement Coverage Metrics", Proceedings of the Fourth International Workshop on Software Engineering for Secure Systems (SESS 2008), co-located with ICSE, pp. 49-56, 2008.
	== Abstract ==		== Abstract ==
	''An increasing number of cyber attacks are occurring at the application layer when attackers use malicious input. These input validation vulnerabilities can be exploited by (among others) SQL injection, cross site scripting, and buffer overflow attacks. Statement coverage and similar test adequacy metrics have historically been used to assess the level of functional and unit testing which has been performed on an application. However, these currently-available metrics do not highlight how well the system protects itself through validation. In this paper, we propose two SQL injection input validation testing adequacy metrics: target statement coverage and input variable coverage. A test suite which satisfies both adequacy criteria can be leveraged as a solid foundation for input validation scanning with a blacklist. To determine whether it is feasible to calculate values for our two metrics, we perform a case study on a web healthcare application and discuss some issues in implementation we have encountered. We find that the web healthcare application scored 96.7% target statement coverage and 98.5% input variable coverage''		''An increasing number of cyber attacks are occurring at the application layer when attackers use malicious input. These input validation vulnerabilities can be exploited by (among others) SQL injection, cross site scripting, and buffer overflow attacks. Statement coverage and similar test adequacy metrics have historically been used to assess the level of functional and unit testing which has been performed on an application. However, these currently-available metrics do not highlight how well the system protects itself through validation. In this paper, we propose two SQL injection input validation testing adequacy metrics: target statement coverage and input variable coverage. A test suite which satisfies both adequacy criteria can be leveraged as a solid foundation for input validation scanning with a blacklist. To determine whether it is feasible to calculate values for our two metrics, we perform a case study on a web healthcare application and discuss some issues in implementation we have encountered. We find that the web healthcare application scored 96.7% target statement coverage and 98.5% input variable coverage''

Programsam at 14:46, 16 May 2021

2021-05-16T14:46:20Z

← Older revision		Revision as of 14:46, 16 May 2021
Line 1:		Line 1:
	B. Smith, Y. Shin, and L. Williams, "Proposing SQL Statement Coverage Metrics", Proceedings of the Fourth International Workshop on Software Engineering for Secure Systems (SESS 2008), co-located with ICSE, pp. 49-56, 2008.		B. Smith, Y. Shin, and L. Williams, "Proposing SQL Statement Coverage Metrics", Proceedings of the Fourth International Workshop on Software Engineering for Secure Systems (SESS 2008), co-located with ICSE, pp. 49-56, 2008.
	== Abstract ==		== Abstract ==
	''An increasing number of cyber attacks are occurring at the application layer when attackers use malicious input. These input validation vulnerabilities can be exploited by (among others) SQL injection, cross site scripting, and buffer overflow attacks. Statement coverage and similar test adequacy metrics have historically been used to assess the level of functional and unit testing which has been performed on an application. However, these currently-available metrics do not highlight how well the system protects itself through validation. In this paper, we propose two SQL injection input validation testing adequacy metrics: target statement coverage and input variable coverage. A test suite which satisfies both adequacy criteria can be leveraged as a solid foundation for input validation scanning with a blacklist. To determine whether it is feasible to calculate values for our two metrics, we perform a case study on a web healthcare application and discuss some issues in implementation we have encountered. We find that the web healthcare application scored 96.7% target statement coverage and 98.5% input variable coverage''		''An increasing number of cyber attacks are occurring at the application layer when attackers use malicious input. These input validation vulnerabilities can be exploited by (among others) SQL injection, cross site scripting, and buffer overflow attacks. Statement coverage and similar test adequacy metrics have historically been used to assess the level of functional and unit testing which has been performed on an application. However, these currently-available metrics do not highlight how well the system protects itself through validation. In this paper, we propose two SQL injection input validation testing adequacy metrics: target statement coverage and input variable coverage. A test suite which satisfies both adequacy criteria can be leveraged as a solid foundation for input validation scanning with a blacklist. To determine whether it is feasible to calculate values for our two metrics, we perform a case study on a web healthcare application and discuss some issues in implementation we have encountered. We find that the web healthcare application scored 96.7% target statement coverage and 98.5% input variable coverage''

Programsam: /* 9. References */

2013-03-14T23:17:48Z

9. References

← Older revision		Revision as of 23:17, 14 March 2013
Line 406:		Line 406:
	This work is supported by the National Science Foundation under CAREER Grant No. 0346903. Any opinions expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.		This work is supported by the National Science Foundation under CAREER Grant No. 0346903. Any opinions expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

	== 9. References ==		== 10. References ==

	: <sup>[1]</sup> B. Beizer, Software testing techniques: Van Nostrand Reinhold Co. New York, NY, USA, 1990.		: <sup>[1]</sup> B. Beizer, Software testing techniques: Van Nostrand Reinhold Co. New York, NY, USA, 1990.
Line 427:		Line 427:
	: <sup>[18]</sup> D. Willmor and S. M. Embury, "Exploring test adequacy for database systems," in Proceedings of the 3rd UK Software Testing Research Workshop, Sheffield, UK, pp. p123-133, 2005.		: <sup>[18]</sup> D. Willmor and S. M. Embury, "Exploring test adequacy for database systems," in Proceedings of the 3rd UK Software Testing Research Workshop, Sheffield, UK, pp. p123-133, 2005.
	: <sup>[19]</sup> H. Zhu, P. A. V. Hall, and J. H. R. May, "Software Unit Test Coverage and Adequacy," ACM Computing Surveys, vol. 29, no. 4, 1997.		: <sup>[19]</sup> H. Zhu, P. A. V. Hall, and J. H. R. May, "Software Unit Test Coverage and Adequacy," ACM Computing Surveys, vol. 29, no. 4, 1997.


	== 11. End Notes ==		== 11. End Notes ==

Programsam at 23:08, 14 March 2013

2013-03-14T23:08:28Z

← Older revision		Revision as of 23:08, 14 March 2013
Line 439:		Line 439:
	# http://works.dgic.co.jp/djunit/		# http://works.dgic.co.jp/djunit/
	# For our case study, we used MySQL v5.0.45-community-nt found at http://www.mysql.com/		# For our case study, we used MySQL v5.0.45-community-nt found at http://www.mysql.com/

			[[Category:Workshop Papers]]

Programsam: /* 5.1 iTrust */

2013-03-14T22:38:02Z

5.1 iTrust

← Older revision		Revision as of 22:38, 14 March 2013
Line 338:		Line 338:
	iTrust was written to conform to a MySQL<sup>8</sup> back-end. The MySQL JDBC connector was used to implement the data storage for the web application by connecting to a remotely executing instance of MySQL v5.1.11-remote-nt. The <code>java.sql.PreparedStatement</code> class is one way of representing SQL statements in the JDBC framework. Statement objects contain a series of overloaded methods all beginning with the word execute: <code>execute(…)</code>, <code>executeQuery(…)</code>, <code>executeUpdate(…)</code>, and <code>executeBatch()</code>. These methods are the java.sql way of issuing commands to the database and each of them represents a potential change to the database. These method calls, which we have previously introduced as ''target statements'', are the focus of our coverage metrics.		iTrust was written to conform to a MySQL<sup>8</sup> back-end. The MySQL JDBC connector was used to implement the data storage for the web application by connecting to a remotely executing instance of MySQL v5.1.11-remote-nt. The <code>java.sql.PreparedStatement</code> class is one way of representing SQL statements in the JDBC framework. Statement objects contain a series of overloaded methods all beginning with the word execute: <code>execute(…)</code>, <code>executeQuery(…)</code>, <code>executeUpdate(…)</code>, and <code>executeBatch()</code>. These methods are the java.sql way of issuing commands to the database and each of them represents a potential change to the database. These method calls, which we have previously introduced as ''target statements'', are the focus of our coverage metrics.

	<center>[[File:~~Sess~~-~~figure-7~~.png]]<br />		<center>[[File:SESS-Figure7.png]]<br />
	'''Figure 7. General iTrust architecture'''</center>		'''Figure 7. General iTrust architecture'''</center>

Programsam: /* 5.1 iTrust */

2013-03-14T22:04:18Z

5.1 iTrust

← Older revision		Revision as of 22:04, 14 March 2013
Line 337:		Line 337:

	iTrust was written to conform to a MySQL<sup>8</sup> back-end. The MySQL JDBC connector was used to implement the data storage for the web application by connecting to a remotely executing instance of MySQL v5.1.11-remote-nt. The <code>java.sql.PreparedStatement</code> class is one way of representing SQL statements in the JDBC framework. Statement objects contain a series of overloaded methods all beginning with the word execute: <code>execute(…)</code>, <code>executeQuery(…)</code>, <code>executeUpdate(…)</code>, and <code>executeBatch()</code>. These methods are the java.sql way of issuing commands to the database and each of them represents a potential change to the database. These method calls, which we have previously introduced as ''target statements'', are the focus of our coverage metrics.		iTrust was written to conform to a MySQL<sup>8</sup> back-end. The MySQL JDBC connector was used to implement the data storage for the web application by connecting to a remotely executing instance of MySQL v5.1.11-remote-nt. The <code>java.sql.PreparedStatement</code> class is one way of representing SQL statements in the JDBC framework. Statement objects contain a series of overloaded methods all beginning with the word execute: <code>execute(…)</code>, <code>executeQuery(…)</code>, <code>executeUpdate(…)</code>, and <code>executeBatch()</code>. These methods are the java.sql way of issuing commands to the database and each of them represents a potential change to the database. These method calls, which we have previously introduced as ''target statements'', are the focus of our coverage metrics.

			<center>[[File:Sess-figure-7.png]]<br />
			'''Figure 7. General iTrust architecture'''</center>

	The version of iTrust we used for this study is referred to as iTrust Fall 2007, named by the year and semester it was built and redistributed to a new set of graduate students. iTrust was written to execute in Java 1.6 and thus our testing was conducted with the corresponding JRE. Code instrumentation and testing were conducted in Eclipse v3.3 Europa on an IBM Lenovo T61p running Windows Vista Ultimate with a 2.40Ghz Intel Core Duo and 2 GB of RAM.		The version of iTrust we used for this study is referred to as iTrust Fall 2007, named by the year and semester it was built and redistributed to a new set of graduate students. iTrust was written to execute in Java 1.6 and thus our testing was conducted with the corresponding JRE. Code instrumentation and testing were conducted in Eclipse v3.3 Europa on an IBM Lenovo T61p running Windows Vista Ultimate with a 2.40Ghz Intel Core Duo and 2 GB of RAM.

Programsam: /* 3.2 Input Variable Coverage */

2013-03-14T22:02:25Z

3.2 Input Variable Coverage

← Older revision		Revision as of 22:02, 14 March 2013
Line 138:		Line 138:
	'''Metric''': The input variable coverage criterion can be measured by the percentage of input variables tested at least once by the test set out of total number of input variables found in any target statement in the production code of the system.		'''Metric''': The input variable coverage criterion can be measured by the percentage of input variables tested at least once by the test set out of total number of input variables found in any target statement in the production code of the system.

	'''Input variable coverage''' = ~~'''PLACEHOLDER'''~~		'''Input variable coverage''' = [[File:Sess-eqn-2.png]]

	where Test(f) is an input variable used in at least one test.		where Test(f) is an input variable used in at least one test.
Line 148:		Line 148:
	The relationship between target statement coverage and input variable coverage is not yet known; however, we contend that input variable coverage is a useful, finer-grained measurement.		The relationship between target statement coverage and input variable coverage is not yet known; however, we contend that input variable coverage is a useful, finer-grained measurement.

	Input variable coverage has the effect of weighting a target statement which has more input variables more heavily. Since most input variables are each a separate potential vulnerability if not adequately validated, a target statement which contains more input variables is of a higher threat level.		Input variable coverage has the effect of weighting a target statement which has more input variables more heavily. Since most input variables are each a separate potential vulnerability if not adequately validated, a target statement which contains more input variables is of a higher threat level.

	== 4. Related Work ==		== 4. Related Work ==

Programsam: /* 3.1 Target Statement Coverage */

2013-03-14T22:02:12Z

3.1 Target Statement Coverage

← Older revision		Revision as of 22:02, 14 March 2013
Line 124:		Line 124:
	statements.		statements.

	'''Server-side target statement coverage''' = ~~'''PLACEHOLDER'''~~		'''Server-side target statement coverage''' = [[File:Sess-eqn-1.png]]

	where Test(''t'') is a SQL statement tested at least once.		where Test(''t'') is a SQL statement tested at least once.

Programsam: /* 1. Introduction */

2013-03-14T22:00:16Z

1. Introduction

← Older revision		Revision as of 22:00, 14 March 2013
Line 6:		Line 6:
	According to the National Vulnerability Database (NVD)<sup>1</sup>, more than half of all of the ever-increasing number of cyber vulnerabilities reported in 2002-2006 were input validation vulnerabilities. As Figure 1 shows, the number of input validation vulnerabilities is still increasing.		According to the National Vulnerability Database (NVD)<sup>1</sup>, more than half of all of the ever-increasing number of cyber vulnerabilities reported in 2002-2006 were input validation vulnerabilities. As Figure 1 shows, the number of input validation vulnerabilities is still increasing.

	<center>[[File:Sess-figure-1.png]]		<center>[[File:Sess-figure-1.png]]<br />
	'''Figure 1. NVD's reported cyber vulnerabilities<sup>2</sup>'''</center>		'''Figure 1. NVD's reported cyber vulnerabilities<sup>2</sup>'''</center>

Programsam at 22:00, 14 March 2013

2013-03-14T22:00:06Z

← Older revision		Revision as of 22:00, 14 March 2013
Line 6:		Line 6:
	According to the National Vulnerability Database (NVD)<sup>1</sup>, more than half of all of the ever-increasing number of cyber vulnerabilities reported in 2002-2006 were input validation vulnerabilities. As Figure 1 shows, the number of input validation vulnerabilities is still increasing.		According to the National Vulnerability Database (NVD)<sup>1</sup>, more than half of all of the ever-increasing number of cyber vulnerabilities reported in 2002-2006 were input validation vulnerabilities. As Figure 1 shows, the number of input validation vulnerabilities is still increasing.

	'''~~PLACEHOLDER FOR FIGURE~~ 1'''<sup>2</sup>		<center>[[File:Sess-figure-1.png]]
			'''Figure 1. NVD's reported cyber vulnerabilities<sup>2</sup>'''</center>

	Figure 1 illustrates the number of reported instances of each type of cyber vulnerability listed in the series legend for each year displayed in the x-axis. The curve with the square shaped points is the sum of all reported vulnerabilities that fall into the categories “SQL injection”, “XSS”, or “buffer overflow” when querying the National Vulnerability Database. The curve with diamond shaped points represents all cyber vulnerabilities reported for the year in the x-axis. For several years now, the number of reported input validation vulnerabilities has been half the total number of reported vulnerabilities. Additionally, the graph demonstrates that these curves are monotonically increasing; indicating that we are unlikely to see a drop in the future in ratio of reported input		Figure 1 illustrates the number of reported instances of each type of cyber vulnerability listed in the series legend for each year displayed in the x-axis. The curve with the square shaped points is the sum of all reported vulnerabilities that fall into the categories “SQL injection”, “XSS”, or “buffer overflow” when querying the National Vulnerability Database. The curve with diamond shaped points represents all cyber vulnerabilities reported for the year in the x-axis. For several years now, the number of reported input validation vulnerabilities has been half the total number of reported vulnerabilities. Additionally, the graph demonstrates that these curves are monotonically increasing; indicating that we are unlikely to see a drop in the future in ratio of reported input