Steven Foerster

Writing

Private AI in Your SOC: How to Run LLMs Locally

Operating LLMs locally can help analysts summarize and triage sensitive telemetry without shipping data to third-party services.

Security Operations AI Privacy Systems

Originally published on LinkedIn. Lightly edited for clarity.

SOC teams want LLMs for summarization and triage, but they cannot send raw logs and alerts to a public API.

A private model changes the risk posture: you keep the data local, control retention, and decide what leaves the boundary.

Decide what the model is allowed to see

Start with scope. Most SOC use cases do not require full packet captures or raw identity data.

Define the minimum context the model needs and redact everything else. Think in tiers:

  • Tier 1: Alert metadata, signatures, severity, and timestamps.
  • Tier 2: Sanitized log excerpts with IDs and IPs masked.
  • Tier 3: Full raw events only when a human explicitly requests them.

Your redaction layer is the real security control. The model is just a consumer.

Choose a model and runtime with operational constraints

Running locally means you own latency, cost, and performance.

Make the tradeoffs explicit:

  • Smaller models are easier to run on CPU and are often sufficient for summarization.
  • Larger models improve reasoning but require GPU capacity and careful scheduling.
  • Quantization reduces memory but can change behavior.

Pick a runtime that supports offline operation, audit logging, and explicit model versioning.

Treat models like dependencies, not SaaS.

Build a simple, auditable pipeline

A durable pattern is:

  1. Normalize alerts into a consistent schema.
  2. Redact and classify sensitive fields.
  3. Run local inference for summarization, prioritization, or enrichment.
  4. Store output with provenance and version metadata.

The SOC should be able to answer, “Which model produced this summary?” without guesswork.

Keep the model inside the security boundary

Local LLMs are not magic. They are still software that can be exploited.

Treat the inference service as a sensitive system:

  • Isolate it on a private network segment.
  • Disable outbound network access by default.
  • Log all prompts and responses for auditability.
  • Apply the same hardening standards you use for other production services.

Accept the limits

Local models can help with triage and narrative building, but they do not replace detection logic or incident response.

Use them to reduce analyst toil, not to make the final call. Keep the model in an assistive role until you have strong validation.

2026 Perspective

Local models are smaller, faster, and easier to operate than they were even a couple of years ago, which makes this approach more practical.

The core discipline is unchanged: minimize what the model sees and treat outputs as untrusted. The SOC that wins is the one that keeps control of its data.