Giraffe Chapter - Status Report 2009/2010

The Giraffe Chapter's continuous goal is to develop and improve honeypot technology and related tools and to conduct in-depth analysis of new attack techniques and malware specimens. This report lists our main activities and contributions from the last two and a half years.

_________________________________________________________________________________
ORGANIZATION

Much to our regret, two of the founding members of our chapter have decided to terminate their Honeynet Project membership and are thus officially moved to alumni status. We respect this step and are grateful for an adventurous journey and their numerous contributions over the years. We will continue to work closely together with our friends, and want them to know that they can rejoin the team whenever they wish to.

The Giraffe Chapter consists of the following people:

  • Felix Leder
  • Mark Schlösser
  • Tillmann Werner
  • Georg Wicherski

_________________________________________________________________________________
DEPLOYMENTS

We have a few dionaea, nepenthes, mwcollectd and honeytrap sensors deployed (all low-interaction server honeypots we develop), although our focus has shifted a bit towards developing data analysis tools, plus we lack the infrastructure for large-scale deployments, hence we do not capture very much these days.

mwcollect Alliance

Georg runs the mwcollect Alliance, a distributed network of honeypot sensors with central data aggregation. The organization has currently about 30 contributing members. Access to the collected information is granted to parties that link their own sensors into the network. People interested in joining should contact Georg directly.

_________________________________________________________________________________
RESEARCH AND DEVELOPMENT

All tools we develop for the Honeynet Project are publicly available from http://code.mwcollect.org/ and http://src.carnivore.it/. We also host some third-party projects there, please check them out. Below is a list of the major things we developed in the last two years:

dionaea

As announced in our last status report, we released a new addition to our collection of carnivorous plants, called dionaea. It was partly developed as a Honeynet Project Summer of Code (HPSoC) project. If you wonder where the name comes from, check out this Wikipedia page. Dionaea is basically a very efficient networking core that maintains connections and dispatches events to modules. Its purpose is to collect attack data and malware binaries. It embeds Python as a scripting language which makes writing new extensions simpler and faster. A libemu module detects and emulates shellcode. Collected malware samples can be processed further, e.g., by submitting them to central storage sites. Protocols and services supported by dinoaea include HTTP, FTP, TFTP, MSSQL, SIP (VoiP) and more. See also the following chronologically ordered blog posts from the HPSoC phase:

mwcollectd

mwcollectd4 is another new low-interaction server honeypot. It combines the best of nepenthes and honeytrap, our other approaches to collecting malware samples and attacks traces. The overall design is similar to dionaea's, but mwcollectd is written in C++ and relies on libnetworkd. There are plugins for shellcode emulation and protocol handling, logging to different channels, and storing data.

SMB Stack for Low-Interaction Honeypots

We have developed a Server Message Block (SMB) protocol stack for use in low-interaction honeypots. Of all services attacked the Windows SMB services are by far the most frequented ones (mostly due to the still aggressively spreading Conficker worm). A special attention to and a careful implementation of the underlying protocol can result in a much bigger number of collected samples. Our SMB implementation is used in both of our new honeypots, dionaea and mwcollectd and is part of their code branches.

RE-Google

RE-Google is our first reverse engineering helper tool. Its functionality is explained best by quoting the project web page: RE-Google is a plugin for the Interactive DisAssembler (IDA) Pro that queries Google Codesearch for information about the functions contained in a disassembled binary. The top results are then displayed as comments to the function and can be opened by just clicking on it. We also blogged about RE-Google:

libxmatch

libxmatch is a C library for matching plain patterns against XOR-encoded data. If the XOR key is no longer than half the pattern length, we can take advantage of the symmetry of the XOR operation and convert the pattern and the input into new strings in such a way that the key is eliminated. Afterwards, classical pattern matching algorithms can be applied. This approach can be used for processing shellcode without emulating it. Modules for dionaea and honeytrapexist.

multicap

Network traffic recording is a vital and fundamental task in many honeynet setups. However, standard tools like wireshark, tcpdump or daemonlogger are not necessarily best suited for this job. An ideal traffic recorder needs the capabilities of dispatching packets of certain types to separate dump files while performing nicely even on high speed and for high volumes of traffic. We have written our own packet capture daemon, called multicap, which takes advantage of some Linux kernel features to reach a good packet drop rate. It is highly configurable, can run on multiple devices at the same time, has extensive dump file rotation capabilites - in short, everything you need if you want to capture network traffic in a honeynet.

streams

streams is an interactive tool for browsing, mining and processing TCP streams in pcap files. It provides a command line prompt for filtering, selecting and dumping reassembled session data. It can further invoke external tools to pipe stream data through.

nebula

nebula, our intrusion signature generator, was extended during another HPSoC effort. A client library was developed that can be used in honeypot software to submit attack data to a nebula daemon. Further, the signature generation strategie was improved to create more accurate signatures. The following blog post contains the details:

libscizzle

<@v> oxff in 1 sentence what it do? :P
<+oxff> v: give it an arbitrary byte buffer and it's a very fast oracle if it contains shellcode or not
<+oxff> fast as in 16MiByte take ~ 300ms on one of my laptop cores
<+oxff> urandom data that is
<+oxff> with real world data, it will be a lot faster

hpfeeds

One more project currently considered as work-in-progress is hpfeeds, a simple publish-subscribe based protocol that features authentication and binary message payload support. It is still under development and active testing but promises to become an easy-to-use alternative for live data feed sharing within the Honeynet Project and also with external parties. The code of the protocol's broker and command-line client implementation in Python is available at Github. The repository also contains an experimental management webinterface. There is a deployment maintained by Mark on a test server to experiment with the project. Every HP member is invited to help testing and take a look at the already running data feeds.

_________________________________________________________________________________
FINDINGS

We mostly worked on the analysis of interesting malware specimens and blogged our findings on the official Honeynet Project blog. In early 2009 we had a lot of fun dissecting the Waledac botnet together with Greg Sinclair which resulted in a post with the title Speaking Waledac. Another blog entry provides A view on Conficker's inside which was later followed by Conficker.A going down?. One highlight was an afternoon at our 2010 annual workshop in Mexico that we spent reversing a successor of the infamous Storm Worm. We published our results here: A Breeze of Storm. Another post deals with native language spam and how it can go wrong.

As far as an overall rating of current honeypot technology is concerned, we repeat our assessment from our 2008 status report: evaluating recorded data remains the main challenge in honeypot research. For the immense amount of data, automating analysis is crucial but only few tools are available to date.

_________________________________________________________________________________
PAPERS AND PRESENTATIONS

Our most important publication is probably the paper Know Your Enemy: Containing Conficker in which we describe the inner workings of the Conficker worm and present several methods to detect and remove it.

The following presentations were given by us:

  • OpenChaos, CCC Cologne, 2009-05-28 - A Look Inside Conficker
  • DCC 2009, Redmond, 2009-10-12 - Containing Conficker - A Case Study
  • SURFnet/SURFibo, Den Haag, 2010-02-14 - Honeypots: From History To Present
  • Honeynet Project Security Workshop, Paris, 2011-03-21 Efficient Bytecode Analysis: Line-Speed Shellcode Detection
  • Honeynet Project Security Workshop, Paris, 2011-03-21 High-Performance Packet Sniffing and Traffic Mining

_________________________________________________________________________________
GOALS
Our goal for the next year is to continue active development of our tools as well as to publish analysis reports for malicious programs that stand out from the crowd for some reason.

We would also like to improve on data sharing and collaboration within the Honeynet Project by spreading the word about XMPP malware sharing and the hpfeeds project.

_________________________________________________________________________________
MISC ACTIVITIES

In 2010 we revived the legendary Honeynet Project Scan of the Month challenges. The Giraffe Chapter kicked the new series off with a pcap attack trace. We received and evaluated 91 submissions. The first 5 questions are discussed in our blog post Dissecting the SotM Attack Trace Pcap.

Georg provided The Shadowserver Foundation with a free copy of the botnet monitoring software botsnoopd and is always happy to give a hand with setup and configuration.