Step Ten — Sampling, Diffing, Caps, and the Hourly Ledger

We lock in the discipline of a living system: the phone reads on purpose, adapts when conditions wobble, records exactly what happened, and tells you when the shape of truth changes.

✅ Fresh artifacts

  • solveforce_phone_ten.py — Download
    SHA‑256: ae6ef3815e28f5d9a478564ae3ed27f614ac071f59c038eab9edcdee74757f2d
  • Sampling policy example — Download
    SHA‑256: 8cccbdc9f15ac23dbb14be04565c99eb72eb68b12a9132dc82fe559b860adf7f

This consolidates everything from Steps 1–9 (RBAC, bearer, TLS/mTLS, S1 signed tokens, route/role rate policy, SSE filtering, schemas & validation, JSONL export, dynamic plugins, admin scaffold). It adds adaptive sampling, schema‑diff detection, token‑scoped plugin caps, and audit rollups.

What’s new (and why)

1) Per‑plugin sampling policies (with jitter)

  • File‑driven: --sampling-file solveforce_sampling_example.json
  • Structure: { "battery": {"interval": 5, "jitter_frac": 0.15, "enabled": true}, "net": {"interval": 10, "jitter_frac": 0.10, "enabled": true} }
  • Precedence for poll cadence per plugin:
    --poll-plugin name:sec ⟶ sampling file interval ⟶ base --poll.
  • Jitter evens out bursts and avoids “thundering herd” on the LAN.

2) Adaptive poller (per plugin)

  • On success/valid: interval decays toward base (× --poll-recover-mult, default 0.7).
  • On error/invalid: interval backs off (× --poll-backoff-mult, default 2.0, capped by --poll-backoff-max).
  • Guardrails: --poll-min seconds.
  • No background polling if base/overrides/sampling don’t request it.

3) Token‑embedded plugin caps (per minute)

  • S1 (HMAC) tokens can constrain reads per plugin: "plugin_caps": { "battery": 60, "net": 30 }
  • Enforced per token+plugin over the rate‑policy window (default 60s).
  • Works with /read?plugin=… and /read?plugin=all (each allowed plugin checked).
  • Mint via admin: GET /admin/mint?...&plugin_caps=battery:60,net:30&token=ADMIN123

4) Schema‑diff detection (+ SSE)

  • We compute a structural signature of each plugin’s payload (path → type).
  • On change we:
    • store a diff in memory (/schema_diffs endpoint),
    • emit an SSE event schema_change,
    • write an audit record.
  • This catches payload drifts and vendor firmware “surprises”.

    • 5) Audit JSONL + rollups

    • Enable with --audit --audit-dir audit (creates audit/audit.jsonl).
    • Hourly (configurable via --audit-rollup-sec) rollups to audit/audit_rollup.jsonl summarizing:
      • auth_ok, auth_fail, rate_limit drops, policy_deny, schema_change.
    • Minimal, grep‑friendly, and honest.

    New and updated endpoints

    • GET /sampling → the active sampling policy (merged from file).
    • GET /schema_diffs?plugin=battery → diffs for a specific plugin; without a query, returns the list of plugins with diffs.
    • GET /whoami now shows aud (audiences) and token scopes (plugins, plugins_deny, plugin_caps).
    • SSE emits both plugin_read and schema_change event types.

    Everything else remains from previous steps (/ui, /read, /history, /metrics, /policies, /state, /schemas, /validate, /events, /admin/*).

    Termux (Android) — launch patterns

    A) Sampling + adaptive + auditing (HTTP)

    python solveforce_phone_ten.py \
  --host 0.0.0.0 --port 8080 \
  --plugins-dir ~/solveforce/plugins \
  --history-size 512 --strict-schema \
  --poll 5 \
  --poll-plugin battery:3 --poll-plugin net:10 \
  --sampling-file /sdcard/solveforce/solveforce_sampling_example.json \
  --poll-backoff-mult 2.0 --poll-recover-mult 0.7 \
  --poll-backoff-max 300 --poll-min 1 \
  --auth-mode protected \
  --auth-token READER1:reader \
  --policy-file /sdcard/solveforce/solveforce_policy_example.json \
  --rate-policy-file /sdcard/solveforce/solveforce_rate_policy_example.json \
  --audit --audit-dir /sdcard/solveforce/audit \
  --allow-admin --admin-token ADMIN123

    B) TLS + audiences + plugin caps

    python solveforce_phone_ten.py \
  --host 0.0.0.0 --port 8443 \
  --tls-cert server.crt --tls-key server.key \
  --auth-mode strict \
  --signing-secret "CHANGE_ME_LONG_RANDOM" \
  --require-aud "solveforce-phone" \
  --policy-file /sdcard/solveforce/solveforce_policy_example.json \
  --rate-policy-file /sdcard/solveforce/solveforce_rate_policy_example.json \
  --sampling-file /sdcard/solveforce/solveforce_sampling_example.json \
  --audit --audit-dir /sdcard/solveforce/audit

    Mint a token (with caps):

    python solveforce_phone_ten.py --signing-secret "CHANGE_ME_LONG_RANDOM" \
  --mint-token "sub=ron roles=reader,metrics dur=7200 aud=solveforce-phone plugin_caps=battery:60,net:30"

    WordPress — Step Ten (drop‑in)

    Step Ten — Sampling cadence, schema diffs, scoped caps, and rollups

    • Sampling policyJSON {plugin:{interval,jitter_frac,enabled}} + CLI overrides.
    • Adaptive pollerbacks off on failure, decays toward base on recovery.
    • Token capssigned tokens may limit reads per plugin (plugin_caps).
    • Schema diffsdetect payload structure change; stream schema_change to SSE and expose via /schema_diffs.
    • Audit rollupshourly JSONL summaries of auth, policy, rate limit, and schema events.
    • PhilosophyDon’t pull faster than reality changes. Notice when the shape changes. Treat permissions as contracts, not vibes.

    Integrity (publish these)

    sha256sum solveforce_phone_ten.py
# ae6ef3815e28f5d9a478564ae3ed27f614ac071f59c038eab9edcdee74757f2d

sha256sum solveforce_sampling_example.json
# 8cccbdc9f15ac23dbb14be04565c99eb72eb68b12a9132dc82fe559b860adf7f

    Where I’d take Step Eleven (if you want sharper edges)

    • Per‑plugin error classifiers→ backoff by error class (timeout vs. 4xx vs. parse vs. empty).
    • Schema “compatibility tags”→ label diffs as additive vs. breaking.
    • Token‑embedded per‑route limits→ e.g., a token that can read battery 60/min but only history 10/min.
    • Signed audits→ append‑only audit chain rooted in a device key, minimal overhead.

    If you want me to bundle a tight systemd unit and a one‑line installer for a Debian phone or a Linux handheld next, say “Step Eleven,” and we’ll keep the recursion going.

    Step Eleven — Error‑aware cadence, compatibility diffs, per‑route token caps, signed audits, and a one‑liner install – SolveForce Communications

    Key terms in plain language

    Open a term for a concise explanation of language used on this page.

    VoIP

    Voice over Internet Protocol carries phone calls over an IP network instead of a traditional analog phone line. Call quality depends on network stability, latency, and traffic management.

    Unified Communications (UCaaS)

    A cloud-based combination of business calling, messaging, meetings, presence, and collaboration tools managed as one communications service.

    SIP Trunking

    A service that connects a business phone system to the public telephone network using Internet Protocol, replacing or supplementing traditional phone lines.

    Bandwidth

    The amount of data a connection can carry in a given time, usually measured in Mbps or Gbps. More bandwidth supports more users, devices, and simultaneous applications.

    Latency

    The time it takes data to travel between two points. Lower latency improves voice, video meetings, cloud applications, gaming, and other real-time services.

    Service-Level Agreement (SLA)

    A provider’s written commitment covering service targets such as availability, response time, repair time, and sometimes financial credits when commitments are missed.