SKI Framework v3.1.0-beta.1

Quick start

Prerequisites: Python 3.11+, Docker 24+, Git. The demo mode exercises every framework guarantee, including cryptographic signing and the Symbolic Verifier, using a deterministic FakeLLM backend. No GPU required.

Install from PyPI

pip install ski-sdk ski-schemas ski-kg-extractor ski-kg-validator ski-audit-ledger ski-model-deploy

pip install --pre ski-sdk ski-schemas ski-kg-extractor ski-kg-validator ski-audit-ledger ski-model-deploy

5-minute demo

# 1. Clone
git clone https://github.com/kpifinity/ski-framework.git && cd ski-framework

# 2. Generate secrets, TLS, and a signed demo KG
./scripts/setup.sh

# 3. Start the full stack (FakeLLM, no GPU needed)
cd reference-implementation
docker compose -f docker-compose.demo.yml up -d

# 4. Health check
curl http://localhost:8000/api/health
# → {"status":"ok","kg":"loaded","kg_signature":"valid"}

# 5. Submit a measurement and get a verdict
curl -X POST http://localhost:8000/api/evaluate \
  -H "Content-Type: application/json" \
  -d '{"tenant_id":"demo","readings":{"so2_ppm":142},"context":"energy.wellhead.monitoring"}'

Six packages

SKI ships six PyPI packages. ski-schemas and ski-sdk are independently versioned and will reach 1.0.0 on their own schedule. The four tools track the framework version.

How SKI works

SKI is a passive sidecar observer. It receives a measurement record, evaluates it against a sealed Knowledge Graph, and produces a V3VerdictEnvelope. It has no write path to operational systems, by architectural design.

The evaluation path has two independent components that must agree before a verdict counts:

1. The LLM examiner, reads the measurement, reasons against the KG, produces natural-language reasoning, and emits formalizable assertions.
2. The Symbolic Verifier, a deterministic algorithm that re-checks every assertion mechanically against the measurement record and the cited obligation.

If they agree: CLEAR or FLAG. If they disagree: DISCRETIONARY (human review). See the animated walkthrough for the full visual explanation.

Verdict envelope

Every evaluation produces a V3VerdictEnvelope (spec v3.0 §4.2). Key fields:

verdict                  # One of five categorical outcomes (see below)
reasoning                # LLM natural-language explanation (signed, not ground truth)
kg_citations[]           # node_id, role, source_document, source_clause
formalizable_assertions[] # obligation_id, metric, operator, observed, expected, unit, satisfied
verifier_result          # status (AGREED/LLM_CONTRADICTION), verifier_observed, verifier_expected
model_provenance         # model_weight_hash, kg_version_hash, prompt_template_id, decoder_seed
transcript_ref           # ledger sequence pointer for the signed LLM transcript
notes[]                  # taxonomy guard remapping messages, if any

The full schema lives in ski-schemas (envelope.py ↗). ski_model.v3.envelope re-exports from it for backwards compatibility.

Five verdict types

The taxonomy guard (v3.1.0-beta.1) ensures CLEAR can never be emitted with zero citations and zero assertions. An assertion-less CLEAR is deterministically remapped to NULL_UNMAPPED or DISCRETIONARY and recorded in envelope.notes.

Symbolic verifier

The Symbolic Verifier is a deterministic algorithm, not an LLM, that independently re-checks every formalizable_assertion the LLM produced. It verifies four obligation types (spec §3.3):

• must_be_below / must_not_exceed
• must_be_above / must_not_be_below
• must_be_one_of
• must_not_be_one_of

If the Verifier disagrees with the LLM on any assertion, or if the LLM's observed value doesn't match the measurement record, the result is LLM_CONTRADICTION and the verdict becomes DISCRETIONARY. A parity test pins both directions: a KG author cannot write a spec-valid obligation that silently verifies nothing.

Audit ledger

The ledger is append-only at the database trigger level, no UPDATE, no DELETE. Every entry contains:

• The signed V3SignedTranscript (ed25519 signature over the raw LLM output)
• A monotonically increasing sequence number
• A SHA-256 hash of the previous entry (hash chaining)
• The full V3VerdictEnvelope JSON

Sequence gaps trigger a Prometheus alert (ski_ledger_sequence_gaps_total). Tampering with any stored transcript is detectable by recomputing the ed25519 signature. Auditors can query the ledger directly and replay any verdict from its stored transcript.

Knowledge graph

The Knowledge Graph is a versioned, sealed artifact encoding regulatory obligations as machine-readable nodes. Each node includes: an obligation type, metric, operator, threshold, unit, and a citation to the source regulatory document and clause.

The KG lifecycle: extract via ski-kg-extractor → human expert review → validate via ski-kg-validator → sign (ed25519) → deploy. The runtime refuses to load an unsigned or signature-invalid graph. Every verdict envelope records the KG version hash.

Docker Compose

The reference implementation ships two Compose files:

cd reference-implementation
docker compose up -d
# Pulls qwen2.5:7b (~4 GB) on first run
docker compose logs -f ski-model

Kubernetes / Helm

The deploy/helm/ski chart enforces the SKI deployment constraints at render time, it fails if you try to configure multiple replicas (single writer by design), if no operator Secret is provided, or if no signed-KG ConfigMap is present.

# Create the signed-KG ConfigMap
kubectl create configmap ski-kg --from-file=kg.json=your-signed-kg.json

# Create operator Secret
kubectl create secret generic ski-secrets \
  --from-literal=SKI_SIGNING_KEY="$(cat keys/signing.key)"

# Install
helm install ski ./deploy/helm/ski \
  --set image.tag=3.1.0-beta.1

# Air-gapped: drops DNS inside the pod network
helm install ski ./deploy/helm/ski \
  --set networkPolicy.airgap=true

Default image: ghcr.io/kpifinity/ski-model, no --set image.repository required for standard deployments. Review values.yaml and the NetworkPolicy before production use.

Air-gapped environments

SKI's sovereignty guarantee is architectural: the full runtime is tested inside --network=none on every CI run. The air-gapped boot rig (conformance/sovereignty/test_air_gapped.py) boots the audit ledger, joins the SKI Model to a loopback-only namespace, replays a signed-KG workload, and verifies hash-chained ledger entries from inside the gap.

To run the Level 3 sovereignty suite locally:

SKI_L3_AIRGAP=1 pytest conformance/sovereignty/ -v

LLM backends

Conformance levels

SKI defines three conformance levels across 77 tests. The full conformance suite runs in CI:

pip install -r requirements-dev.txt
pytest conformance/ -v
# 77 tests: 30 Provenance + 30 Durability + 17 Sovereignty

For full compliance details including the EU AI Act crosswalk, see the Compliance page.

SKI Evals

SKI ships its own adversarial evaluation suite and publishes every run. A 50-case human-graded golden dataset evaluates the real production path, not mocks. The suite reports accuracy, FLAG recall/precision, NULL_UNMAPPED recall, assertion correctness, and LLM↔verifier agreement rate.

python -m evals.run --backend ollama
# or with vLLM:
python -m evals.run --backend vllm --endpoint http://your-vllm:8000

Full methodology and published run history: docs/evals.md ↗.

Benchmarks

The benchmark suite isolates framework overhead (scoping, citation validation, symbolic verification, risk-tier policy, ed25519 transcript signing) from LLM inference. Reference numbers on 2 vCPUs:

CI gates every build on p99 ≤ 100 ms (~250× headroom). Full methodology: docs/benchmarks.md ↗.

Observability, /metrics

The runtime exports a Prometheus-compatible /metrics endpoint (intentionally unauthenticated for scrape compatibility). It exposes aggregates only, no payload data, no verdict bodies, no reasoning text, no tenant identifiers.

ski_agreement_rate                 # LLM ↔ Verifier agreement
ski_kg_signature_verified          # KG signature validity (gauge)
ski_ledger_sequence_gaps_total     # Ledger tampering alert
ski_verdicts_total{verdict=...}    # Per-verdict-type counter
ski_evaluation_duration_seconds    # Latency histogram
ski_model_last_telemetry_timestamp # Heartbeat
ski_runtime_info                   # Version / KG hash info label

Grafana alert rules ship in monitoring/. A regression test parses the alert rules and asserts every referenced series is exported, the contract cannot silently drift.

Frequently asked questions

Which LLM does SKI use?

SKI is LLM-agnostic. The reference implementation ships with an Ollama backend (any local model via the Ollama API) and a vLLM backend (any OpenAI-compatible inference server). You supply the model — SKI orchestrates it, signs the transcript, and cross-checks the output with the Symbolic Verifier. Switching backends is a one-line config change. Custom backends can be added by implementing the LLMBackend protocol.

Is there a hosted or SaaS version?

No, and there will not be one. Data sovereignty is an architectural property of SKI, not a configurable feature. For that guarantee to be real, your operational data must never leave your infrastructure. SKI is on-premise only.

How large is the deployment footprint?

The reference implementation runs as a single Docker Compose stack. Minimum resource requirements for the SKI framework services (excluding the LLM): approximately 2 vCPUs and 4 GB RAM. The air-gapped boot rig has been validated on a single bare-metal node. LLM resource requirements depend entirely on which model you choose to run.

How long does it take to get a first verdict?

The 5-minute estimate in the quick start is accurate for demo mode, which does not require a model download and exercises every framework guarantee with a stub LLM. A production deployment including model download, KG configuration, and infrastructure setup is measured in hours, not days.

Can SKI connect to my existing SCADA or operational data systems?

SKI accepts readings as JSON payloads over an HTTP API. Any SCADA system, DCS, historian, or event stream that can make an HTTP POST can feed SKI. An OPC-UA read-only sidecar reference implementation is included in the repository under reference-implementation/integrations/opc-ua/.

Can I use SKI's Knowledge Graph for regulations outside the examples?

Yes. The KG schema is general-purpose — it is not locked to any specific regulatory domain. The reference KG covers an energy sector SO2 example; your domain experts translate your specific regulatory obligations into KG nodes, review and approve them, and the resulting graph is sealed and deployed. A full KG authoring and validation workflow is documented in Knowledge graph.

Changelog

Full changelog: CHANGELOG.md ↗. Latest release notes: v3.1.0-beta.1 ↗.

Security

To report a security issue, see SECURITY.md ↗. Do not open a public issue for security vulnerabilities. Responsible disclosure is the right path, we will acknowledge within 48 hours.

GitHub

kpifinity/ski-framework