Rootkits and how to combat them

One of the biggest difficulties faced by virus writers is keeping malicious code hidden from the user of the infected machine. In a virus writer’s ideal world, malicious code would be undetectable by antivirus solutions as well. With writing malicious code becoming less of a private pleasure, and more of a profitable (if criminal) business, covering their tracks has become increasingly important for those hackers who work for money. So how can a program which steals bank account details, or installs a proxy server to send spam without the knowledge of the machine’s owner, be hidden?

Modern cyber criminals are taking an approach which was used by the cyber vandals of 10 or 15 years ago. One of the first known viruses for PCs was Virus.Boot.Brain.a. This boot sector virus intercepted the system functions access to disk. When the boot sector was read (for example, by an antivirus solution) the virus would replace infected data with the original, clean data. Such stealth mechanisms (intercepting system functions and replacing the data returned) have since been used in Windows viruses. (Virus.Win32.Cabanas.a).

To date, malicious programs for UNIX are not as widespread as those for the DOS and Windows environments. The term ‘rootkit’ originated in the UNIX world; however, today it’s often used to describe stealth technologies utilized by the authors of Windows Trojans. Initially, ‘rootkit’ was used to mean a collection of programs which made it possible for a hacker to evade detection. In order to do this, executable system files (such as login, ps, ls, netstat etc) or system libraries (libproc.a) are replaced, or a kernel module is installed. Both actions have the same purpose; to prevent users from receiving accurate information about what is taking place on the computer.

As Graph 1 shows, rootkits are becoming more and more widely used in order to mask the presence of malicious code on infected systems.

Graph 1. Increase in use of rootkits in malicious programs

The increased popularity of rootkits is partly due to the fact that the source code of many rootkits is now openly available on the Internet. It’s relatively easy for virus writers to make small modifications to such code. Another factor which influences the increased use of rootkits is the fact that most Windows users use the administrator’s account, rather than creating a separate user account. This makes it much easier to install a rootkit on the victim machine.

The fact that rootkits can be used to mask the presence of malicious code from both users and antivirus solutions is seen as a unique selling point (USP) by virus writers and developers of quasi-legal spyware alike.

Let’s take a more detailed look at Windows and UNIX rootkits.

Windows rootkits

Masking methods

Currently rootkits use two methods to mask their presence in the system:

1. modifying paths
2. modifying system structures

These methods are used to mask network activity, registry keys, processes e.g. all the things which could alert a user to the fact that a malicious program is active in the system.

The first method can be used both in user and kernel mode rootkits. Implementation in user mode rootkits is relatively easy. Most often, a method based on hooking API functions is used to modify the path to executables.

Pic 2. Hooking API functions.

This method exploits the fact that API functions are called by applications using special data fields (import/ export tables) or by contacting an address received by using API GetProcAddress. Program code is implemented in DLL modules, which are then integrated into the address space of existing system processes, giving the remote malicious user the ability to control all user applications. Modifying paths is a well documented and easily-implemented approach, making it simple to use in rootkits.

However, although such an approach, together with other approaches utilized by user-mode rootkits offer many benefits, there are also significant drawbacks: although the rootkit will mask activity, it will do so fairly ineffectively. This means that it is easy to detect a user-mode rootkit running in the system using dedicated utilities such as RootKit Revealer, SoftIce, etc. This is undoubtedly the reason why kernel-mode rootkits are becoming more popular, in spite of the fact that they are more difficult to develop.

Kernel-mode rootkits are unquestionably better at masking information. The vast majority of kernel-mode rootkits exploit undocumented operating system structures. For instance, such rootkits often hook KeServiceDescriptorTable services. The number of services in this table can alter according to the operating system version. This means that rootkit developers have to conduct an additional analysis of the operating system code in order to find handler indices in the table. This approach is very similar to hooking API functions.

Modifying PsActiveProcessList is one example of the second approach i.e. altering system structures. This approach is used by the FU rootkit, enabling it to mask any process from being detected or viewed using the majority of system utilities.

Pic. 3. Process list before the rootkit is launched.

Pic. 4. Process list after the rootkit is launched’

Diagram 3 shows that Notepad is visible in the list of active processes as notepad.exe (marked with a bold red line). The screenshot in Diagram 4 was taken after the FU rootkit had been launched with the command to hide a process. The screenshot shows that even though the editing program is active, the name is missing from the list of active processes.

Detecting rootkits

The first step in combating rootkits is to detect them. Rootkit authors always have the potential to be one step ahead of rootkit detection tools, as new technologies are constantly being developed, and antivirus software developers need time to analyse these technologies and develop detection tools. However, notwithstanding the seeming complexity of rootkits, version 6 of Kaspersky Lab products includes effective detection for rootkits. (Version 6.0 is currently undergoing testing). So how do our products react to the FU rootkit described above?

To reiterate: the installation of the rootkit has resulted in system processes being hidden. The anti-rootkit subsystem detects this and displays a warning to the user (picture 5.)

Pic.5 Detecting unknown, hidden processes

This subsystem makes it possible to detect not only those rootkits which have already been added to antivirus database updates, but also ones which are as yet unknown, as shown in picture 6.

There is a similar subsystem used for detecting user mode rootkits, (discussed in the first part of this paper) which inject DLLs into other processes.

In such cases, the subsystem reacts by notifying the user that a specific process is attempting to inject code into another process (diagram 6)

Pic.6. Detecting code injection into another process

Rootkits for UNIX

Masking presence in the system

Rootkits are used in much the same way in UNIX systems as in Windows. The attacker installs a rootkit on the victim machine as soon as s/he has gained root privileges. Root privileges are essential for the installation of the majority of rootkits and can be gained by exploiting known vulnerabilities, as long as the remote malicious user has standard user privileges on the victim system. In such cases, the remote user can use a local exploit or a utility to crack the password database. If s/he does not have appropriate rights, then a remote exploit can be used, or, for example, a sniffer to intercept passwords. Password interception can be used with a range of services (ftp, telnet etc) as all these services transmit passwords across networks in clear text.

Rootkits may incorporate a range of malicious programs (Trojan-DDos, backdoor etc) which will be installed on the compromised machine, and listen for commands from the remote malicious user. In addition to this, rootkits may also include patches to close vulnerabilities on the system, with the aim of preventing repeated penetration by another attacker.

Both user-mode and kernel mode rootkits exist for UNIX, just as they do for Windows.

User mode rootkits usually incorporate Trojanized versions of standard programs which mask the existence of their components in the system, and a backdoor, which provides stealthed access to the system. Some examples of user mode rootkits are lkr, trOn, ark and others. Let’s take trOn as an example of a user mode rootkit. In order to mask its presence in the system, the rootkits performs a number of actions. When installed, it stops the syslogd daemon, and then replaces the following system utilities with its own, Trojanized, version: du, find, ifconfig, login, ls, netstat, ps, top, sz. In addition to this, a Trojan version of the sshd daemon is added to the system. Finally, a sniffer is launched in background mode; launch of telnetd, rsh, and finger daemons is added to inetd.conf, inetd is relaunched and syslogd is rebooted.

TrOn is normally located in /usr/src/.puta, but this catalogue is invisible due to the ls component which is already installed.

Now let’s take a look at kernel mode rootkits. Rootkits of this type provide all the functionality of user mode rootkits, but at a lower level. User mode rootkits are designed to modify individual binary files, whereas kernel mode rootkits are intended to modify only the kernel. Such rootkits are far more effective in masking activity.

There are several methods for injecting rootkits into the system kernel:

1. Using LKM: the Linux kernel (as in many other operating systems) makes it possible to upload modules (or device drivers) on the fly, which allows the remote malicious user to modify the kernel’s system calls, and results in incorrect information being returned (e.g. a modified file list). Such attacks can be prevented by compiling a monolith kernel without LKM support, but such a solution has the drawback that all drivers have to be included in the kernel.
2. writing to /dev/kmem, which allows access to the area of memory occupied by the kernel. The entry rewrites the kernel without informing the user. Modifying the kernel can be done simply by locating the appropriate area of memory. However, there is an answer to this issue – modifications can be made which make it impossible to write directly to /dev/kmem. Writing to /dev/kmem can also be done using mmap calls.
3. infecting existing modules; this differs from the first method in that the rootkit itself does not contain a separate module, and injects itself into existing modules. This method means that the rootkit is resistant to rebooting, as it infects modules which are always uploaded (eg. file system drivers).

Detecting rootkits

Unfortunately it is impossible to give universal guidelines on how to detect rootkits. However, the following applies to the majority of rootkits:

1. investigating atypical file behaviour, use of network resources, launching of tasks on schedule and on reboot, and monitoring user accounts
2. using the following utilities which can help to detect the presence of a rootkit in the system: Saint Jude, Chrootkit, RkScan, Carbonite, Kstat, Rootkithunter, Tripware, Samhain and others.

Conclusion

All the methods for detecting active rootkits depend on the fact that they disrupt system functioning in one way or another. Kaspersky Lab products exploit this, which also makes them able to detect unknown rootkits.It will be more difficult to write rootkits for future versions of Windows, where it is impossible to modify system code and the system architecture. This step taken by the developers of the operating system should reduce, if only temporarily, the number of new rootkits for new versions of Windows.

Currently, malicious code for Windows is more common than for UNIX because Windows is the most widely used operating system. However, if UNIX starts to gain popularity, then the situation will naturally change; new rootkits for UNIX will be written, and new methods of combating them will be developed.

Finally, it should be noted that the best protection against rootkits is taking preventative action by ensuring that all systems are appropriately protected.

Bibliography

1. [VirusList] Kaspersky Lab, Viruslist.com, 2005
2. [iDEF2003] Anton Chuvakin, An Overview of Unix Rootkits, 2003
3. Linux Kernel Rootkit. Rainer Wichmann
4. Rootkit: Attacker undecover tools. Saliman Manap
5. www.phrack.org
6. www.securityfocus.org