Skip to content
Falak Neo
Operations

Self-hosting

Running falak-neo-core and the console on your own infrastructure, including sizing, storage and upgrade guidance.

The sensor and core are Apache-2.0. Nothing about the architecture assumes our managed service, and self-hosted deployments have no sub-processors — nothing leaves your infrastructure.

What you operate

ComponentRequirement
falak-neo-core4 vCPU / 8 GiB to start; scales horizontally behind a load balancer
ClickHouseThe event store. Sizing below
PostgreSQL15+. Small — application state only
Object storageOptional, for the cold OCSF parquet archive
Certificate authorityFor sensor mTLS. Core can act as its own, or use your PKI / SPIRE

Sizing ClickHouse

Event volume is the number that matters. A rough model from production fleets:

events/node/day ≈ 3,500,000     # general-purpose K8s node, default policy
bytes/event     ≈ 480           # compressed, ZSTD, hot projection

For 100 nodes at 30-day hot retention:

3.5e6 × 100 × 30 × 480 bytes ≈ 5.0 TiB

Add 30% headroom for merges and projections. A build fleet runs 3–5× higher on file events, which is what the low-overhead policy exists for.

Cheapest lever first

Before buying disk, look at file_open filters. The default exclusions plus an include_only policy on build hosts routinely cut total volume by half with no detection loss — because build churn under /builds and /nix/store carries no security signal.

Deploying core

# docker-compose.yml (evaluation)
services:
  core:
    image: ghcr.io/falak-neo/core:1.7.2
    environment:
      FALAK_CLICKHOUSE_DSN: "clickhouse://falak:…@clickhouse:9000/falak_events"
      FALAK_POSTGRES_DSN:   "postgres://falak:…@postgres:5432/falak_core"
      FALAK_TLS_CERT:       "/certs/core.pem"
      FALAK_TLS_KEY:        "/certs/core-key.pem"
      FALAK_CA_BUNDLE:      "/certs/ca.pem"
      FALAK_SIGNING_KEY:    "/keys/action-signing-ed25519.key"
    ports:
      - "443:8443"   # gRPC ingest (mTLS)
      - "8080:8080"  # REST API

For production, the Helm chart:

helm upgrade --install falak-neo-core \
  oci://registry.falakneo.example/charts/core \
  --namespace falak-system --create-namespace \
  --values ./core-values.yaml

The signing key is the crown jewel

FALAK_SIGNING_KEY signs sensor configuration and response actions. Anyone holding it can instruct every sensor in your fleet to kill processes. Keep it in a KMS or HSM, not on the filesystem, and rotate it on the same schedule as your other production signing material.

Certificates

Two options.

Core as its own CA — simplest. Core issues client certificates during enrollment from a CA you generate once. Suitable for most deployments.

SPIRE / SPIFFE — if you already run SPIRE, sensors present an SVID and core validates against your trust bundle. The SPIFFE ID appears on the session object of every event, which is useful for workload-identity-based correlation.

Upgrades

Core and sensor versions are independent within one minor version. The supported sequence:

  1. Upgrade core first. It accepts the previous sensor minor version.
  2. Roll sensors gradually. Set desiredVersion on a subset and let the fleet converge.
  3. Watch the drift view. Do not proceed while sensors are reporting probe failures on the new version.

ClickHouse schema migrations run automatically on core start and are forward-only. Take a backup of the PostgreSQL state before a major version upgrade; the event store is replayable from the archive, but triage decisions are not.

Backups

DataStrategy
PostgreSQLStandard pg_dump or continuous archiving. Small — minutes, not hours
ClickHouseBACKUP TABLE … TO Disk(…), or rely on the cold parquet archive
Signing keyOffline, in escrow. Losing it means re-enrolling every sensor
CA materialOffline. Same reasoning

Observability

Core exposes Prometheus metrics on :9090/metrics. The ones worth alerting on:

MetricAlert when
falak_ingest_batches_rejected_totalRising — schema or clock problems in the fleet
falak_ingest_lag_seconds> 30s — core cannot keep up with the fleet
falak_agents_stale_total> 2% of fleet — sensors are not reporting
falak_probe_failed_totalAny increase — a visibility gap opened
falak_clickhouse_insert_errors_totalAny — events are being lost
falak_detection_eval_duration_secondsp99 > 500ms — a rule is pathological

Monitor the sensors, not just core

falak_agents_stale_total is the one people forget. A sensor that stops reporting looks identical to a quiet host, and an attacker who kills the sensor is counting on exactly that. Core raises a finding on a heartbeat gap for this reason, but a Prometheus alert on the aggregate is worth having too.

Something wrong or missing? Edit this page