# Linux Syscalls for Exploit Development > How Linux syscalls work at the instruction level, int 0x80 vs syscall, register setup, and the key syscalls used in shellcode and ROP chains. - Published: 2025-08-14 - Difficulty: intermediate - Series: Linux Exploitation Fundamentals - Tags: Linux, Assembly, x86, x64, Internals, Exploit Development, Shellcode - Source: https://stevenfoerster.com/tutorials/linux-syscalls-for-exploit-development/ ## Prerequisites - Understanding of x86/x64 registers and calling conventions - Basic assembly reading ability - Familiarity with GDB and GDB-PEDA Every useful thing shellcode or a ROP chain does, spawning a shell, reading a file, changing memory permissions, ultimately happens through a syscall. Syscalls are the interface between user-space code and the kernel. When you execute `execve("/bin/sh", NULL, NULL)`, you're asking the kernel to replace the current process with a shell. The kernel doesn't care whether that request came from a normal program or from shellcode injected through a buffer overflow. This tutorial covers how syscalls work at the instruction level on both x86 and x64 Linux, then walks through the specific syscalls you'll use most in exploit development. ## The User/Kernel Boundary User-space code cannot directly access hardware, manage processes, or interact with the filesystem. These operations require kernel privileges. A syscall is the mechanism for crossing that boundary: 1. User-space code places the syscall number and arguments in registers 2. A special instruction transfers control to the kernel 3. The kernel validates the request, performs the operation, and returns a result 4. Execution resumes in user space with the return value in a register The two instructions that trigger this transition are `int 0x80` (x86) and `syscall` (x64). The following diagram shows this transition. User-space code sets up registers and executes the trap instruction. The CPU switches to ring 0, the kernel's syscall handler runs, and control returns to user space with the result in EAX/RAX: ```text User Space (Ring 3) Kernel Space (Ring 0) ┌─────────────────────┐ ┌─────────────────────┐ │ │ │ │ │ Set up registers: │ │ syscall handler: │ │ EAX/RAX = number │ │ validate args │ │ args in regs │ │ perform operation │ │ │ │ set return value │ │ int 0x80 / syscall ─┼──→│ (EAX/RAX) │ │ instruction │ │ │ │ │←──┼─ return to user │ │ Check return value │ │ │ │ in EAX/RAX │ │ │ │ │ │ │ └─────────────────────┘ └─────────────────────┘ CPU Ring 3 CPU Ring 0 (unprivileged) (privileged) ``` The `int 0x80` instruction triggers a software interrupt that the kernel's Interrupt Descriptor Table (IDT) routes to the syscall handler. The `syscall` instruction uses a faster mechanism -- Model-Specific Registers (MSRs) -- to jump directly to the handler without the IDT lookup. In both cases, the CPU automatically switches privilege levels. ## x86 Syscalls: int 0x80 On 32-bit Linux, syscalls are invoked by triggering software interrupt 0x80. ### Register Setup | Register | Purpose | | -------- | -------------- | | EAX | Syscall number | | EBX | Argument 1 | | ECX | Argument 2 | | EDX | Argument 3 | | ESI | Argument 4 | | EDI | Argument 5 | | EBP | Argument 6 | The return value is placed in EAX. A negative return value indicates an error (the negated errno). ### Example: write(1, "hello\n", 6) ```asm section .data msg db "hello", 0x0a ; "hello\n" section .text global _start _start: mov eax, 4 ; syscall 4 = write mov ebx, 1 ; fd = 1 (stdout) mov ecx, msg ; buffer address mov edx, 6 ; length int 0x80 ; invoke syscall ``` ### Assembling and Running ```bash $ nasm -f elf32 hello.asm -o hello.o $ ld -m elf_i386 hello.o -o hello $ ./hello hello ``` ## x64 Syscalls: syscall Instruction On 64-bit Linux, the `syscall` instruction replaced `int 0x80`. It's faster; it uses a dedicated CPU mechanism (MSRs) instead of the interrupt descriptor table. ### Register Setup | Register | Purpose | | -------- | -------------- | | RAX | Syscall number | | RDI | Argument 1 | | RSI | Argument 2 | | RDX | Argument 3 | | R10 | Argument 4 | | R8 | Argument 5 | | R9 | Argument 6 | The return value goes in RAX. > **Note**: The 4th argument uses R10, not RCX. The `syscall` instruction overwrites RCX with the return address and R11 with the saved RFLAGS. These registers are clobbered on every syscall, never rely on their values afterward. ### Example: write(1, "hello\n", 6) ```asm section .data msg db "hello", 0x0a section .text global _start _start: mov rax, 1 ; syscall 1 = write (different number than x86!) mov rdi, 1 ; fd = 1 (stdout) mov rsi, msg ; buffer address mov rdx, 6 ; length syscall ``` ### Assembling and Running ```bash $ nasm -f elf64 hello.asm -o hello.o $ ld hello.o -o hello $ ./hello hello ``` ### Syscall Numbers Differ Between Architectures The same operation has different syscall numbers on x86 and x64: | Syscall | x86 (int 0x80) | x64 (syscall) | | -------- | -------------- | ------------- | | read | 3 | 0 | | write | 4 | 1 | | open | 5 | 2 | | close | 6 | 3 | | execve | 11 | 59 | | dup2 | 63 | 33 | | mprotect | 125 | 10 | | exit | 1 | 60 | Getting the syscall number wrong is a common mistake when translating shellcode between architectures. ## Syscalls for Exploit Development You don't need to know all 300+ syscalls. For exploitation, a handful cover the vast majority of use cases. > [!tip] > To convert between hex and ASCII: `python3 -c "print(chr(0x41))"` gives `A`, and `python3 -c "print(hex(ord('A')))"` gives `0x41`. The string `/bin/sh` in hex is `\x2f\x62\x69\x6e\x2f\x73\x68`. ### execve. Spawn a Shell `execve` replaces the current process with a new program. This is the syscall behind every `system("/bin/sh")` call and the goal of most shellcode. **Signature**: `execve(const char *pathname, char *const argv[], char *const envp[])` **x86, execve("/bin/sh", NULL, NULL):** ```asm xor eax, eax ; Zero EAX push eax ; Push null terminator push 0x68732f2f ; "//sh" push 0x6e69622f ; "/bin" mov ebx, esp ; EBX = pointer to "/bin//sh\0" xor ecx, ecx ; ECX = NULL (argv) xor edx, edx ; EDX = NULL (envp) mov al, 0xb ; Syscall 11 = execve int 0x80 ``` The `//sh` trick pads the string to 4 bytes. The shell treats `//` the same as `/`. **x64, execve("/bin/sh", NULL, NULL):** ```asm xor rsi, rsi ; RSI = NULL (argv) push rsi ; Push null terminator onto stack mov rdi, 0x68732f2f6e69622f ; "/bin//sh" as a 64-bit immediate push rdi mov rdi, rsp ; RDI = pointer to "/bin//sh\0" xor rdx, rdx ; RDX = NULL (envp) xor eax, eax ; Zero RAX first to avoid stale upper bits mov al, 59 ; Syscall 59 = execve syscall ``` > **Note**: Passing NULL for both argv and envp works for spawning a basic shell. Some programs expect a valid argv array, in those cases you'd need to construct `argv[0]` on the stack and point to it. ### read. Read Input `read` pulls bytes from a file descriptor into a buffer. In exploitation, it's used for staged payloads: a small first-stage shellcode calls `read` to pull a larger second-stage into memory. **Signature**: `read(int fd, void *buf, size_t count)` **x64 example, read 256 bytes from stdin into the stack:** ```asm xor rax, rax ; Syscall 0 = read xor rdi, rdi ; fd = 0 (stdin) mov rsi, rsp ; buf = current stack pointer mov rdx, 256 ; count = 256 bytes syscall jmp rsp ; Execute the bytes we just read ``` After the `read` syscall writes shellcode onto the stack (starting at RSP), the `jmp rsp` gadget redirects execution to the freshly written shellcode. This works because `read` writes data at the address in RSI, and we point RSI at RSP. This pattern is the foundation of a `read` + `jmp rsp` staged payload. ### write. Write Output `write` sends bytes from a buffer to a file descriptor. Useful for leaking memory contents, write the Global Offset Table (GOT), where the runtime linker stores resolved library function addresses, stack addresses, or canary values back to an attacker-controlled socket. **Signature**: `write(int fd, const void *buf, size_t count)` **x64 example, leak 8 bytes from an address:** ```asm mov rax, 1 ; Syscall 1 = write mov rdi, 1 ; fd = 1 (stdout), or socket fd mov rsi, 0x601020 ; Address to leak (e.g., GOT entry) mov rdx, 8 ; Leak 8 bytes syscall ``` ### mprotect. Change Memory Permissions `mprotect` changes the permissions on a memory region. In exploitation, it's used to make the stack (or another writable region) executable, bypassing NX. **Signature**: `mprotect(void *addr, size_t len, int prot)` Protection flags: - `PROT_READ` = 1 - `PROT_WRITE` = 2 - `PROT_EXEC` = 4 To make memory readable, writable, and executable: `1 | 2 | 4 = 7` **x64 example, make a stack page executable:** ```asm mov rax, 10 ; Syscall 10 = mprotect mov rdi, 0x7ffffffde000 ; Page-aligned address mov rsi, 0x21000 ; Length (must cover the target region) mov rdx, 7 ; PROT_READ | PROT_WRITE | PROT_EXEC syscall ``` > **Note**: The address passed to `mprotect` must be page-aligned (a multiple of 0x1000 on most systems). If it's not, the syscall fails with EINVAL. In a ROP chain, you typically get the page-aligned stack address from `vmmap` output. ### dup2, Redirect File Descriptors `dup2` duplicates a file descriptor. In remote exploits, once you have code execution, you redirect stdin/stdout/stderr to the network socket so the spawned shell communicates over the network instead of the local terminal. **Signature**: `dup2(int oldfd, int newfd)` **x64 example, redirect stdin, stdout, and stderr to socket fd 4:** ```asm ; Redirect stdin mov rax, 33 ; Syscall 33 = dup2 mov rdi, 4 ; oldfd = socket xor rsi, rsi ; newfd = 0 (stdin) syscall ; Redirect stdout mov rax, 33 mov rdi, 4 mov rsi, 1 ; newfd = 1 (stdout) syscall ; Redirect stderr mov rax, 33 mov rdi, 4 mov rsi, 2 ; newfd = 2 (stderr) syscall ``` After these three calls, the socket (fd 4) is connected to the shell's standard I/O. In typical exploits, the socket file descriptor is often 4 or 5 (after stdin=0, stdout=1, stderr=2, and the listening socket=3). You can find the exact value by examining the return value of `accept()` in GDB or by trying common values. ### exit. Clean Termination `exit` terminates the process. Calling it after your shellcode finishes prevents the program from crashing into undefined memory, which can attract attention or trigger alerts. **Signature**: `exit(int status)` **x64:** ```asm mov rax, 60 ; Syscall 60 = exit xor rdi, rdi ; status = 0 syscall ``` **x86:** ```asm mov eax, 1 ; Syscall 1 = exit xor ebx, ebx ; status = 0 int 0x80 ``` ## Syscalls via ROP Chains When you can't inject shellcode (NX is enabled), you can invoke syscalls through ROP gadgets. The pattern is the same, load registers, then hit a `syscall` or `int 0x80` gadget, but every value comes from the stack. ### x64 execve via ROP To call `execve("/bin/sh", NULL, NULL)` through a ROP chain, you need: 1. RAX = 59 2. RDI = address of "/bin/sh" string 3. RSI = 0 4. RDX = 0 5. A `syscall; ret` gadget The chain structure: ```text [ pop rax; ret ] [ 59 ] [ pop rdi; ret ] [ "/bin/sh" addr ] [ pop rsi; ret ] [ 0 ] [ pop rdx; ret ] [ 0 ] [ syscall ] ``` Finding these gadgets is covered in the ROP gadget hunting workflow. For a comprehensive guide to finding gadgets, see the [ROP Gadget Hunting Toolkit](https://stevenfoerster.com/tutorials/rop-gadget-hunting-toolkit/) tutorial. The key insight is that syscall-based ROP chains are architecture-dependent: the registers and syscall numbers must match the target. ### Checking for Syscall Gadgets ```bash $ ROPgadget --binary ./vuln --only "syscall|ret" $ ropper -f ./vuln --search "syscall" ``` In GDB-PEDA: ```text gdb-peda$ ropsearch "syscall" ``` If the binary lacks a `syscall` gadget, check libc; it always contains one. ## Looking Up Syscall Numbers ### The Header Files The canonical source for syscall numbers: ```bash # x64 ABI $ grep -R "^#define __NR_execve " /usr/include/x86_64-linux-gnu/asm/unistd_64.h # Output: #define __NR_execve 59 # x86 (32-bit) ABI $ grep -R "^#define __NR_execve " /usr/include/x86_64-linux-gnu/asm/unistd_32.h # Output: #define __NR_execve 11 ``` > **Note**: Header paths vary by distro and toolchain. If these exact paths don't exist, search for `unistd_32.h` and `unistd_64.h` under `/usr/include`. ### Using Python ```python >>> import ctypes >>> libc = ctypes.CDLL("libc.so.6") >>> libc.syscall(1, 1, b"hello\n", 6) # write(1, "hello\n", 6) ``` > [!note] > The Python ctypes example above shows how to _invoke_ a syscall, but it doesn't help you _look up_ syscall numbers. The definitive source for numbers is the kernel header `/usr/include/asm/unistd_64.h` (or `unistd_32.h` for x86). You can also dump all numbers at once with `ausyscall --dump` (from the `audit` package). For a quick one-liner: > > ```bash > $ grep __NR_execve /usr/include/asm/unistd_64.h > #define __NR_execve 59 > ``` ### Quick Reference Table | Syscall | x86 | x64 | Args (in order) | | -------- | --- | --- | -------------------- | | read | 3 | 0 | fd, buf, count | | write | 4 | 1 | fd, buf, count | | execve | 11 | 59 | pathname, argv, envp | | dup2 | 63 | 33 | oldfd, newfd | | mprotect | 125 | 10 | addr, len, prot | | exit | 1 | 60 | status | ## Tracing Syscalls ### strace `strace` intercepts and logs every syscall a program makes: ```bash $ strace ./binary execve("./binary", ["./binary"], ...) = 0 write(1, "hello\n", 6) = 6 exit(0) = ? ``` This is useful for understanding what a binary does before exploiting it, and for verifying that your shellcode makes the expected syscalls. ### Filtering Specific Syscalls ```bash $ strace -e trace=write,execve ./binary ``` ### In GDB Set a catchpoint to break on a specific syscall: ```text gdb-peda$ catch syscall execve gdb-peda$ r Catchpoint 1 (call to syscall execve), 0x00007ffff7ac5e07 in execve () ``` This lets you verify that your shellcode or ROP chain reaches the syscall with the right register values.