Syscall Hijacking: Anti Fork-Bomb LKM (kernel 2.6.x)

Hi. In this post I’ll show you how to implement a simple anti fork-bomb LKM.
There is already a kernel method to prevent the fork bomb: you can search online about this stuff.
Instead I’ll show you how prevent a fork bomb attack through a simple loadable kernel module, in order to better understand how a new process is created and how we can prevent its creation.
First of all, we need to know which system calls are called when we “fork()” a process.
We can write an ad-hoc script that only runs a “fork()” (“simple_fork.c”):

#include <stdio.h>

int main() {

	fork();

	return 0;

}

We can compile it, run it and trace (man strace) the executable:

spaccio@spaccio-laptop:~$ gcc simple_fork.c -o simple_fork
spaccio@spaccio-laptop:~$ ./simple_fork
spaccio@spaccio-laptop:~$ strace ./simple_fork
execve("./simple_fork", ["./simple_fork"], [/* 47 vars */]) = 0
...
munmap(0xb78e0000, 88209)               = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb78dfb98) = 6731
exit_group(0)                           = ?
spaccio@spaccio-laptop:~$

As you can see, the “clone()” function (man clone) is called: why is’nt called the “fork()” function instead?
Looking at “man fork()”:

Since version 2.3.3, rather than invoking the kernel's fork() system call, the
glibc fork() wrapper that is provided as part of the NPTL threading
implementation invokes clone(2) with flags that provide the same effect as the
traditional system call.  The glibc wrapper invokes any fork handlers that
have been established using pthread_atfork(3).

If you want to pursue about this topic, I suggest you to read this post that explain it very well.
In the previous glibc version could be called directly the fork system call.
We want to know which system calls are associated to both the functions:

spaccio@spaccio-laptop:~$ cat /usr/include/asm-generic/unistd.h |grep -e clone -e fork
/* kernel/fork.c */
#define __NR_clone 220
__SYSCALL(__NR_clone, sys_clone)	/* .long sys_clone_wrapper */
#define __NR_vfork 1071
__SYSCALL(__NR_vfork, sys_vfork)
#define __NR_fork 1079
__SYSCALL(__NR_fork, sys_fork)
__SYSCALL(__NR_fork, sys_ni_syscall)
#define __NR_syscalls (__NR_fork+1)
spaccio@spaccio-laptop:~$ 

“Clone()” is a libc wrapper for the “sys_clone” system call: it’s defined in “arch/x86/kernel/process_32.c” (otherwise in a 64bit architecture in “process_64.c”):

asmlinkage int sys_clone(struct pt_regs regs)
{
	unsigned long clone_flags;
	unsigned long newsp;
	int __user *parent_tidptr, *child_tidptr;

	clone_flags = regs.bx;
	newsp = regs.cx;
	parent_tidptr = (int __user *)regs.dx;
	child_tidptr = (int __user *)regs.di;
	if (!newsp)
		newsp = regs.sp;
	return do_fork(clone_flags, newsp, &regs, 0, parent_tidptr, child_tidptr);
}

Istead the “fork()” is a wrapper for the “sys_fork” system call (again defined into “process_32.c”):

asmlinkage int sys_fork(struct pt_regs regs)
{
	return do_fork(SIGCHLD, regs.sp, &regs, 0, NULL, NULL);
}

Both the functions call the “do_fork()” function that handles the creation of a new process.
In this post I’ll treat only the “clone” system call.
The idea of this fork-bomb prevention is very simple: we have to hijack the “clone” system call and check for the current process number. If this number is greater then a threshold we prevent the creation of a new process.
We know how hijack a system call. Now we need to know how count the number of the running process in our system.
The kernel gives us a macro that we can use for our purpose:

#define for_each_process(p) \
        for (p = &init_task ; (p = next_task(p)) != &init_task ; )

This macro is defined in “linux/sched.h”. It simply scans all the processes in the system.
The “next_task()” function is defined as follows:

#define next_task(p)    list_entry((p)->tasks.next, struct task_struct, tasks)

The definition of “list_entry()” is the following:

/*
 * list_entry - get the struct for this entry
 * @ptr:        the &struct list_head pointer.
 * @type:       the type of the struct this is embedded in.
 * @member:     the name of the list_struct within the struct.
 */
#define list_entry(ptr, type, member) \
        container_of(ptr, type, member)

Thanks to this macro we can list all the running processes:

struct task_struct *task;

for_each_process(task) {
	printk("%s => %d\n", task->comm ,  task->pid);
}

So we have only to count the number of processes found:

int numb_proc = 0;
struct task_struct *task;

for_each_process(task) {

	numb_proc++;
}

Now we have only to hijack the “clone” system call (you can extend the module intercepting the “fork” system call) and check for the processes number.
Follows the code of the kernel module (“antifork.c”):

#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/unistd.h>
#include <asm/current.h>
#include <linux/sched.h>

#define THRESHOLD	123

unsigned long *syscall_table = (unsigned long *)0xc05d2180; 

asmlinkage int (*original_clone)(struct pt_regs regs);

asmlinkage int new_clone(struct pt_regs regs) {

	int numb_proc = 0;
	struct task_struct *task;

	for_each_process(task) {

		numb_proc++;
	}

	printk("Current process number %d\n", numb_proc);

	if ( numb_proc > THRESHOLD) {

		printk("Fork not permitted\n");
		return -EAGAIN;
	}

	return (*original_clone)(regs);

}

static int init(void) {

    printk(KERN_ALERT "\nHIJACK INIT\n");

    write_cr0 (read_cr0 () & (~ 0x10000));

    original_clone = (void *)syscall_table[__NR_clone];
    syscall_table[__NR_clone] = new_clone;  

    write_cr0 (read_cr0 () | 0x10000);

    return 0;
}

static void exit(void) {

    write_cr0 (read_cr0 () & (~ 0x10000));

    syscall_table[__NR_clone] = original_clone;  

    write_cr0 (read_cr0 () | 0x10000);

    printk(KERN_ALERT "MODULE EXIT\n");

    return;
}

module_init(init);
module_exit(exit);

Obviously, in order to work properly you need to change the “syscall_table” address according to your. If you don’t know how to do it, you can read this and this posts.
Here is the Makefile:

obj-m	:= antifork.o

KDIR    := /lib/modules/$(shell uname -r)/build
PWD    := $(shell pwd)

default:
	$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules

We can compile it:

spaccio@spaccio-laptop:~$ make
make -C /lib/modules/2.6.35-22-generic/build SUBDIRS=/home/spaccio/ modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.35-22-generic'
  CC [M]  /home/spaccio/antifork.o
/home/spaccio/antifork.c: In function ‘init’:
/home/spaccio/antifork.c:45: warning: assignment makes integer from pointer without a cast
/home/spaccio/antifork.c: In function ‘exit’:
/home/spaccio/antifork.c:56: warning: assignment makes integer from pointer without a cast
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/spaccio/antifork.mod.o
  LD [M]  /home/spaccio/antifork.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.35-22-generic'
spaccio@spaccio-laptop:~$ 

Now we can load the module and check for the processes number:

spaccio@spaccio-laptop:~$ sudo insmod antifork.ko 
spaccio@spaccio-laptop:~$ tail /var/log/messages 
...
Feb  7 21:33:39 spaccio kernel: [  699.921277] Current process number 123
spaccio@spaccio-laptop:~$

Voluntarily I’ve decreased the threshold in the source to 123 so I can test the module correctness.
Now, if I run the “simple_fork” script, the module hijacks the clone system call, checks for the processes number and will prevent to create a new process:

spaccio@spaccio-laptop:~$ ./simple_fork
fork: Resource temporarily unavailable
spaccio@spaccio-laptop:~$

As you can see we can’t create a new process. We can also verify this from the “messages” log:

spaccio@spaccio-laptop:~$ tail /var/log/messages 
...
Feb  7 21:37:17 spaccio kernel: [  917.821033] Current process number 123
Feb  7 21:37:17 spaccio kernel: [  917.833976] Current process number 124
Feb  7 21:37:17 spaccio kernel: [  917.834047] Fork not permitted

The current process number was greater than the threshold so the kernel module blocks the fork.
The post is finished and I hope that it can help you to better understand the kernel programming and how the kernel handles the fork bomb prevention.
As usual, for any questions you can use comments.
Bye.