Using SQL Hotspots in a Prioritization Heuristic for Detecting All Types of Web Application Vulnerabilities: Difference between revisions
Programsam (talk | contribs) |
Programsam (talk | contribs) |
||
| Line 128: | Line 128: | ||
Our logistic regression model included only hotspots and lines of code for the independent variables. Using Weka, we trained the model for each project using the information on vulnerable files from releases 1 to N, and then tested the model on release N+1. We repeated this process for each of the 15 releases of WordPress and WikkaWiki that information on vulnerabilities for use in training the model, for a total of eight comparisons in WordPress and five comparisons in WikkaWiki<sup>25</sup>. | Our logistic regression model included only hotspots and lines of code for the independent variables. Using Weka, we trained the model for each project using the information on vulnerable files from releases 1 to N, and then tested the model on release N+1. We repeated this process for each of the 15 releases of WordPress and WikkaWiki that information on vulnerabilities for use in training the model, for a total of eight comparisons in WordPress and five comparisons in WikkaWiki<sup>25</sup>. | ||
To evaluate our model, as well as the the ability of hotspots to predict whether a file will be vulnerable in the next release, we do not look at the precision and recall of the model by themselves since these measures give us no idea of how "difficult" the prediction is to make. A model that has a precision of 80% may seem imperfect, but this model would be far more useful than a model trained by the same data set with precision of 10%. The same goes for the model's recall. In light of this fact, we compared our model's precision and recall with a model that randomly assigned files as being vulnerable or neutral. To do better than random, our model must have better precision and recall than this random guess<sup>[1]</sup>. In many projects, the percentage of vulnerable files is far fewer than the percentages in our projects<sup>[17]</sup>. We designed our random guess model to assign files as being neutral or vulnerable according to the vulnerability distribution discovered in each project empirically. That is, we did not assign a "coin toss" guess of vulnerabilities, where the probability of being vulnerable is p=0.50. Instead, since the percentage of vulnerable files for WordPress is 26.1%, we created a model which gave the probability of a file being vulnerable as p=0.261. Similarly, for WikkaWiki, the percentage of vulnerable files is 29%, we assigned our model a probability of p=0.29. We ran the random guess for 10 trials. The results reported in this paper are the best precision and recall the random guess achieved in our trials. | |||
== 5. Results == | == 5. Results == | ||