Using SQL Hotspots in a Prioritization Heuristic for Detecting All Types of Web Application Vulnerabilities: Difference between revisions
Programsam (talk | contribs) |
Programsam (talk | contribs) |
||
| Line 122: | Line 122: | ||
We used the '''Mann-Whitney-Wilcoxon (MWW)''' statistic to perform any population-based comparison between two independent samples, such as between vulnerable and neutral files, or between files that contain hotspots and files that do not. The MWW test is a non-parametric determination of whether two independent samples of observations have equally large values. We used a non-parametric statistical test because we cannot assume that the outcomes in our data set are normally distributed. We also used the '''Chi-Squared Test''' to determine whether there was a statistically significance difference in the proportion of positive outcomes in two population groups. We also used the '''F Test''' to measure the difference in variance between two sample groups. | We used the '''Mann-Whitney-Wilcoxon (MWW)''' statistic to perform any population-based comparison between two independent samples, such as between vulnerable and neutral files, or between files that contain hotspots and files that do not. The MWW test is a non-parametric determination of whether two independent samples of observations have equally large values. We used a non-parametric statistical test because we cannot assume that the outcomes in our data set are normally distributed. We also used the '''Chi-Squared Test''' to determine whether there was a statistically significance difference in the proportion of positive outcomes in two population groups. We also used the '''F Test''' to measure the difference in variance between two sample groups. | ||
== 4.7. Predictive Modeling == | |||
We built logistic regression models to evaluate the number of hotspots as a predictor of whether or not a file contains any type of vulnerability in each project. We considered many alternatives for our modeling technique, and compared the precision and recall across all releases for each of the models using Weka<sup>24</sup>. Based on these scores, Weka allowed us to see that logistic regression models consistently outperformed the other choices available in the modeling toolkit provided with Weka. Our model included a term for lines of code (LOC) because intuitively the larger a file is, the more likely a code change will occur in that file. Coincidentally, the model performed better with the LOC term than without. | |||
Our logistic regression model included only hotspots and lines of code for the independent variables. Using Weka, we trained the model for each project using the information on vulnerable files from releases 1 to N, and then tested the model on release N+1. We repeated this process for each of the 15 releases of WordPress and WikkaWiki that information on vulnerabilities for use in training the model, for a total of eight comparisons in WordPress and five comparisons in WikkaWiki<sup>25</sup>. | |||
== 5. Results == | == 5. Results == | ||