# Tuxscope Lab 4: Network Monitoring > Monitor TCP connections in real time by probing tcp_v4_connect and inet_csk_accept, capture source and destination IPs, ports, and connection direction. - Published: 2026-03-23 - Updated: 2026-07-14 - Difficulty: intermediate - Series: Tuxscope: Linux Kernel Observability with eBPF - Tags: Linux, eBPF, Rust, Observability, Networking, Kernel - Source: https://stevenfoerster.com/tutorials/tuxscope-network-monitoring/ ## Prerequisites - Completed Labs 1-3 (Hello eBPF, Syscall Tracing, File I/O Observation) - Basic TCP/IP knowledge helpful In the previous labs you observed syscalls and file I/O. You could see that a process called `write` or `read`, but you could not see where the data was going over the network. In this lab you probe two functions deep in the TCP stack, `tcp_v4_connect` for outbound connections and `inet_csk_accept` for inbound connections, to capture IP addresses, ports, and connection direction in real time. This lab introduces two new techniques: kretprobes (probing function return instead of entry) and `bpf_probe_read_kernel` (safely reading kernel memory from BPF programs). The complete source code is at [gitlab.com/sfoerster/tuxscope](https://gitlab.com/sfoerster/tuxscope). > [!note] Prerequisites > You need a Linux system running kernel 5.8 or later with root access and the tuxscope binary built from source. You should have completed [Labs 1-3](https://stevenfoerster.com/tutorials/tuxscope-hello-ebpf/). Familiarity with TCP/IP basics (what ports and IP addresses are, the difference between client and server) is helpful but not strictly required. ## TCP connections in the Linux kernel When a process makes a TCP connection (e.g., `curl https://example.com`), the path through the kernel looks like this: ```text User Space Kernel Space ┌──────────┐ ┌─────────────────────────────────┐ │ │ │ │ │ connect() ┼─────────────────→│ sys_connect() │ │ │ │ │ │ │ │ │ v │ │ │ │ tcp_v4_connect() ← outbound │ │ │ │ │ │ │ │ │ v │ │ │ │ Send SYN packet │ │ │ │ │ │ accept() ┼─────────────────→│ sys_accept() │ │ │ │ │ │ │ │ │ v │ │ │ │ inet_csk_accept() ← inbound │ │ │ │ │ │ │ │ │ v │ │ │ │ Return new socket │ │ │ │ │ └──────────┘ └─────────────────────────────────┘ ``` **`tcp_v4_connect`** is called when a process initiates an outbound IPv4 TCP connection. Its signature is: ```c int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len); ``` The second argument, `uaddr`, points to a `sockaddr_in` structure containing the destination IP and port. By reading this structure, you know where the process is connecting to. **`inet_csk_accept`** is called when a listening socket accepts an inbound connection. Its signature is: ```c struct sock *inet_csk_accept(struct sock *sk, int flags, int *err, bool kern); ``` The return value is a pointer to the new socket structure representing the accepted connection. The socket contains both the remote (source) and local (destination) addresses. Since the interesting data is in the return value, you need a kretprobe here, not a regular kprobe. The signature above is for kernels before 6.1. On 6.1+ the trailing `int flags, int *err, bool kern` arguments were collapsed into a single `struct proto_accept_arg *arg`, so the function is now `struct sock *inet_csk_accept(struct sock *sk, struct proto_accept_arg *arg)`. The return value (the new socket) is unchanged, so the kretprobe approach below works either way; only adjust the entry-argument handling if you probe the arguments on a 6.1+ kernel. ## Kretprobes In [Lab 3](https://stevenfoerster.com/tutorials/tuxscope-file-io-observation/) you used kprobes, which fire when a function is entered. A kretprobe fires when the function returns. This gives you access to the return value. ```text kprobe kretprobe │ │ v v ┌───────────────────────────────────────────┐ │ function_entry: │ │ ... function body ... │ │ return value; ──────────────────────│──→ captured here └───────────────────────────────────────────┘ ``` In Aya, a kretprobe is declared with `#[kretprobe]` and receives a `RetProbeContext`. The return value is accessed with `ctx.ret()`. This is why `inet_csk_accept` needs a kretprobe: the information you want (the new socket with remote address information) is the return value. At function entry, the new socket does not exist yet. ## Reading kernel memory safely: bpf_probe_read_kernel In a BPF program, you cannot simply dereference a kernel pointer. The BPF verifier will reject any program that does this. Instead, you must use `bpf_probe_read_kernel` to copy data from a kernel address into your BPF stack: ```rust use aya_ebpf::helpers::bpf_probe_read_kernel; // WRONG: the verifier will reject this // let value = *some_kernel_ptr; // CORRECT: safely copy the value without dereferencing it directly let value_ptr = some_kernel_ptr as *const u32; let value: u32 = unsafe { bpf_probe_read_kernel(value_ptr)? }; ``` The pointer might be invalid: the memory could have been freed, the address might be in an unmapped region, or the data might be in a different address space. `bpf_probe_read_kernel` handles these cases safely: if the read fails, it returns an error instead of crashing the kernel. In this lab, you need to follow pointers to reach the IP address and port. The socket layer copies the user-supplied address into kernel memory before the protocol implementation calls `tcp_v4_connect`, so its `uaddr` argument is read with `bpf_probe_read_kernel`, not the user-memory helper. The chain is: ```text uaddr (struct sockaddr *) → sockaddr_in.sin_port → sockaddr_in.sin_addr.s_addr ``` For `inet_csk_accept`, the return value is a `struct sock *`, and you need: ```text sock → __sk_common.skc_dport (remote port) → __sk_common.skc_rcv_saddr (local IP) → __sk_common.skc_daddr (remote IP) → __sk_common.skc_num (local port) ``` Each of these requires a `bpf_probe_read_kernel` call. ## The event struct ```rust // tuxscope-common/src/lib.rs #[repr(C)] #[derive(Clone, Copy)] pub struct NetEvent { pub pid: u32, pub sport: u16, // source port (remote peer for accept events) pub dport: u16, // destination port (local listener for accept events) pub saddr: u32, // source IPv4 address (network byte order) pub daddr: u32, // destination IPv4 address (network byte order) pub timestamp_ns: u64, pub comm: [u8; 16], pub event_type: u8, // 0 = connect (outbound), 1 = accept (inbound) } ``` IPv4 addresses are stored as 32-bit integers in network byte order (big-endian). Userspace converts them to dotted-quad notation for display. The `event_type` field distinguishes outbound connections (`connect`) from inbound connections (`accept`), similar to how the `op` field in Lab 3 distinguished reads from writes. ## The eBPF programs ### Outbound connections: tcp_v4_connect ```rust // tuxscope-ebpf/src/net.rs use aya_ebpf::{ macros::{kprobe, kretprobe, map}, maps::RingBuf, programs::{ProbeContext, RetProbeContext}, helpers::{ bpf_get_current_pid_tgid, bpf_ktime_get_ns, bpf_get_current_comm, bpf_probe_read_kernel, }, }; use tuxscope_common::NetEvent; #[map] static EVENTS: RingBuf = RingBuf::with_byte_size(256 * 1024, 0); #[kprobe] pub fn trace_tcp_connect(ctx: ProbeContext) -> u32 { match try_tcp_connect(&ctx) { Ok(()) => 0, Err(_) => 1, } } fn try_tcp_connect(ctx: &ProbeContext) -> Result<(), i64> { let pid = (bpf_get_current_pid_tgid() >> 32) as u32; let timestamp_ns = unsafe { bpf_ktime_get_ns() }; let comm = bpf_get_current_comm().map_err(|e| e as i64)?; // Argument 1 (index 1) is struct sockaddr *uaddr let sockaddr_ptr: *const u8 = ctx.arg(1).ok_or(1i64)?; // sockaddr_in layout: // offset 0: sa_family (u16) // offset 2: sin_port (u16, network byte order) // offset 4: sin_addr (u32, network byte order) let dport: u16 = unsafe { bpf_probe_read_kernel((sockaddr_ptr.add(2)) as *const u16)? }; let daddr: u32 = unsafe { bpf_probe_read_kernel((sockaddr_ptr.add(4)) as *const u32)? }; let event = NetEvent { pid, sport: 0, // source port not yet assigned at connect time dport: u16::from_be(dport), saddr: 0, // source address not yet assigned daddr, timestamp_ns, comm, event_type: 0, // connect }; if let Some(mut entry) = EVENTS.reserve::(0) { entry.write(event); entry.submit(0); } Ok(()) } ``` The `sockaddr_in` structure has a fixed layout defined by POSIX. The port is at byte offset 2 and the address at byte offset 4. Both are in network byte order (big-endian), so the port is converted with `u16::from_be()` for display. The IP address stays in network byte order and is converted in userspace. > [!note] Source port at connect time > When `tcp_v4_connect` is called, the kernel has not yet assigned a source port; that happens later in the connection setup. The source port and address fields are zero in the connect event. If you need the source port, you would probe `tcp_v4_connect` with a kretprobe and read the socket after the connection is established. ### Inbound connections: inet_csk_accept ```rust // tuxscope-ebpf/src/net.rs (continued) #[kretprobe] pub fn trace_tcp_accept(ctx: RetProbeContext) -> u32 { match try_tcp_accept(&ctx) { Ok(()) => 0, Err(_) => 1, } } fn try_tcp_accept(ctx: &RetProbeContext) -> Result<(), i64> { // The return value is a struct sock * for the new connection let sock_ptr: *const u8 = ctx.ret().ok_or(1i64)?; // A null return means accept failed if sock_ptr.is_null() { return Ok(()); } let pid = (bpf_get_current_pid_tgid() >> 32) as u32; let timestamp_ns = unsafe { bpf_ktime_get_ns() }; let comm = bpf_get_current_comm().map_err(|e| e as i64)?; // Read fields from struct sock → __sk_common // These offsets come from the kernel's sock_common structure. // They can vary between kernel versions — use BTF for production tools. // struct sock begins with __sk_common at offset 0. // skc_daddr: remote IPv4 address (the client for an accept event) let remote_addr: u32 = unsafe { bpf_probe_read_kernel(sock_ptr as *const u32)? }; // skc_rcv_saddr: local IPv4 address (the listening server) let local_addr: u32 = unsafe { bpf_probe_read_kernel(sock_ptr.add(4) as *const u32)? }; // skc_dport: remote port (network byte order) let remote_port_be: u16 = unsafe { bpf_probe_read_kernel(sock_ptr.add(12) as *const u16)? }; // skc_num: local port (host byte order) let local_port: u16 = unsafe { bpf_probe_read_kernel(sock_ptr.add(14) as *const u16)? }; let event = NetEvent { pid, sport: u16::from_be(remote_port_be), dport: local_port, saddr: remote_addr, daddr: local_addr, timestamp_ns, comm, event_type: 1, // accept }; if let Some(mut entry) = EVENTS.reserve::(0) { entry.write(event); entry.submit(0); } Ok(()) } ``` > [!warning] Struct offsets and kernel versions > The offsets used here are `0`, `4`, `12`, and `14`. `struct sock` begins with `struct sock_common __sk_common`, and these are the offsets reported by `pahole` for the reference kernel. Tuxscope hardcodes them so the moving parts remain visible; this lab does **not** use CO-RE. Verify them against the BTF on the kernel where you run the lab instead of assuming that a copied layout is portable. > > Confirm against your kernel before running for real: > > ```bash > pahole -C sock_common /sys/kernel/btf/vmlinux | head -25 > # Look for the offset numbers next to skc_daddr, skc_rcv_saddr, skc_dport, skc_num. > ``` > > Production eBPF tools (libbpf-tools, bpftrace, Cilium) use BTF + CO-RE relocations so the verifier patches the correct offsets in at load time. We cover that pattern in the BTF lab later in the series; until then, treat the addresses you read here as "trust but verify against `ss -tnp` output". Unlike the connect probe, this one fires on function return via `#[kretprobe]` (the `RetProbeContext` provides access to the return value), reads the return value with `ctx.ret()`, here a pointer to the new `struct sock`, checks that pointer for null since a failed accept returns null, and has the full connection tuple because the TCP handshake is already finished. ## The userspace formatter Userspace converts the raw 32-bit addresses to human-readable dotted-quad notation: ```rust fn format_ipv4(addr: u32) -> String { // The helper copied four network-order bytes into a native u32. Recreate // those bytes in native order before handing them to the formatter. let bytes = addr.to_ne_bytes(); format!("{}.{}.{}.{}", bytes[0], bytes[1], bytes[2], bytes[3]) } ``` `bpf_probe_read_kernel` copies the four address bytes without converting them. On a little-endian host, interpreting those bytes as a `u32` changes the numeric value but not the underlying byte sequence. `to_ne_bytes()` reconstructs the bytes as they appeared in kernel memory; `to_be_bytes()` would reverse them on the x86-64 hosts this lab targets. ## Running it ```bash sudo tuxscope net ``` In another terminal, make some connections: ```bash curl -s https://example.com > /dev/null curl -s https://httpbin.org/get > /dev/null ``` The output has this shape (addresses are anonymized with documentation ranges because DNS answers and peers vary): ```text PID COMM TYPE SRC DST 6201 curl connect 0.0.0.0:0 192.0.2.20:443 6205 curl connect 0.0.0.0:0 198.51.100.20:443 ``` The source is `0.0.0.0:0` because the kernel has not assigned the source address and port at the time `tcp_v4_connect` runs. The destination address and port are populated from the `sockaddr_in` passed by the process. If you are running a web server, you will also see inbound connections: ```text PID COMM TYPE SRC DST 8401 nginx accept 203.0.113.42:51234 192.0.2.10:8080 8401 nginx accept 203.0.113.42:51236 192.0.2.10:8080 8401 nginx accept 198.51.100.7:49821 192.0.2.10:8080 ``` For accept events, `SRC` is the remote client and `DST` is the local server. The source port is the client's ephemeral port, and the destination port is the listening port (8080). Keeping that convention consistent makes `SRC` and `DST` mean the same thing for both connect and accept events. Filter by PID: ```bash sudo tuxscope net --pid 6201 ``` JSON output: ```bash sudo tuxscope net --format json ``` ```json {"pid":6201,"event_type":"connect","sport":0,"dport":443,"saddr":"0.0.0.0","daddr":"192.0.2.20","comm":"curl","timestamp_ns":9825200293847} {"pid":8401,"event_type":"accept","sport":51234,"dport":8080,"saddr":"203.0.113.42","daddr":"192.0.2.10","comm":"nginx","timestamp_ns":9825200312054} ``` ## Interpreting network events Common patterns to look for: **DNS-over-TLS and TCP fallback** appear as connect events to port 853 or, less commonly, TCP port 53. Ordinary DNS is usually UDP, so it will not appear in this lab's output. If you want to trace conventional DNS lookups, add a UDP probe as in Exercise 3. **HTTPS traffic** shows up as connect events to port 443. You can see which processes are making outbound HTTPS connections and to which IP addresses. You cannot see the hostname from the IP alone (many sites share IPs), but combining this with DNS traces gives a complete picture. **Service mesh / sidecar** patterns appear as many connections to `127.0.0.1`, the application talks to a local proxy (Envoy, Linkerd, etc.) which makes the actual outbound connection. **Port scanning** looks like rapid connect events to many different ports on the same destination address. If you see this from an unexpected process, investigate. > [!note] IPv6 connections > This lab only traces IPv4 connections. IPv6 uses `tcp_v6_connect` and a different socket structure. Adding IPv6 support is a natural extension: the event struct would need 128-bit address fields, and you would add kprobes on the `v6` variants of the same functions. ## Verification notes The `sock_common` layout in this revision was rechecked with `pahole` against `/sys/kernel/btf/vmlinux` on x86-64 Linux 7.0.11: `skc_daddr` at byte 0, `skc_rcv_saddr` at 4, `skc_dport` at 12, and `skc_num` at 14. `pahole -C sock` also confirmed that `__sk_common` begins at byte 0 of `struct sock`. These observations validate this reference build; they do not turn hardcoded offsets into a portable ABI. Run the commands in the warning against your kernel before trusting the events. Primary references: - [Linux `struct sock_common` and `struct sock`](https://github.com/torvalds/linux/blob/master/include/net/sock.h) - [Linux socket layer, which copies user addresses above the protocol layer](https://github.com/torvalds/linux/blob/master/net/socket.c) - [Aya `bpf_probe_read_kernel` helper documentation](https://docs.rs/aya-ebpf/latest/aya_ebpf/helpers/fn.bpf_probe_read_kernel.html) ## Exercises 1. **Add IPv6 support.** Probe `tcp_v6_connect` and extend the `NetEvent` struct with 128-bit address fields. Use a union or a separate event type. Display IPv6 addresses in standard notation. 2. **Track connection duration.** Use a BPF HashMap to store the timestamp when a connection is established (from the connect or accept event). Probe `tcp_close` to capture when the connection ends. Compute the duration in userspace. This is the foundation of connection-level latency monitoring. 3. **Correlate with DNS.** Probe the `udp_sendmsg` function to capture outbound DNS queries (port 53). Parse the DNS query to extract the hostname. When a connect event arrives, look up the destination IP in the DNS cache to show the hostname alongside the IP address. 4. **Build a connection summary.** After running for a period, print a summary showing the top destination IPs and ports by connection count. Group connections by process name. This gives a high-level view of which processes are talking to which endpoints, useful for auditing and security monitoring. ## What's next In [Lab 5: Tracing Process Lifecycle](https://stevenfoerster.com/tutorials/tuxscope-process-lifecycle/), you will attach to three tracepoints simultaneously to trace fork, exec, and exit events in real time and understand how Linux creates, transforms, and destroys processes.