Netbait: Distributed Worm Detection

When you have large scale visibility, interesting things about worms start to appear:

This paper presents Netbait, a planetary-scale service for distributed detection of Internet worms. Netbait allows users to pose queries that identify which machines on a given net work have been compromised based on the collective view of a geographically distributed set of machines. It is based on a distributed query processing architecture that evaluates queries expressed using a subset of SQL against a single logical database table. This single logical table is realized using a distributed set of relational databases, each populated by local intrusion detection systems running on Netbait server nodes. For speed, queries in Netbait are processed in parallel by distributing them over dynamically constructed query processing trees built over Tapestry, a distributed object and location routing (DOLR) layer. For efficiency, query results are compressed using application-specific aggregation and compact encodings.

We have implemented a prototype system based on a simplified version of the architecture and have deployed it on 90 nodes of the PlanetLab testbed at 42 sites spread across three continents. The system has been continuously running for over a month now and has been collecting probe information from machines compromised by both the Code Red and Nimda worms. Early results based on this data are promising. First, we observe that by having multiple machines sharing probe information from infected machines, we can identify a substantially larger set of infected hosts that would be possible otherwise. Second, we also observe that by having multiple viewpoints of the network, Netbait is able to identify compromised machines that otherwise would have been dificult to detect in cases where worms have an affinity to certain regions of the IP address space.

Source: Netbait: a Distributed Worm Detection Service, Brent N. Chun, Jason Lee, and Hakim Weatherspoon.

They are using resources built on top of PlanetLab, a research overlay network. This gives them great flexibility and power using real global Internet data. An interesting news item from March 20, 2003 (their last update on the site):

It's unclear if there's any correlation between the resurgence of Code Red II (i.e., Code Red II.F) and the war against Iraq. In any case, there appears to be a significant of amount of activity since March 11, 2003.

You can view their data using their HTML reporting interface on the research site.