Semi-Supervised Learning on Email Characteristics for Novel Worm Detection

Posts have been a little erratic lately, exhaustion is setting in. Thanks for your continued patronage.

You know me, I love novel detection methods, and mass-mailers are always interesting in some fashion. The good news is that there's never a shortage of them.

A major drawback of unsupervised learning for worm detection is the possibility of false negatives. Previous work copes with this problem by increasing the sensitivity of the unsupervised classification algorithms. This, in turn, creates many more false positives. Our focus is narrowed to worms propagating through email.

We present the following contributions. First, we examine a wide range of features calculated on email traffic to determine indicators that discriminate between infected from normal email behavior. Using these features, we next present a new method that uses semi-supervised learning for adaptive virus detection that leverages system administrator feedback to improve classification. Our approach combines the strengths of sensitive novelty detection with a parametric classifier to drastically reduce the false positives.

Source: Semi-Supervised Learning on Email Characteristics for Novel Worm Detection, Steve Martin and Anil Sewani.