TCDN-AI / Architecture

How a TCDN-AI request runs

TCDN-AI separates the part of an inference service that has to be centralised from the part that does not. A small coordinator owns identity, pricing, placement and settlement; the model itself runs on workers — machines operated by people and small operators — which get their weights from each other rather than from one datacenter. This page follows one request through that system, and says plainly where the design stops short of the guarantee a reader might assume.

01The pieces

Coordinator

The control plane. Authenticates the key, prices the job, places the credit hold, picks a worker, signs a short-lived ticket, verifies receipts, settles the job and writes the audit row.

Never receives prompt content and holds no session key, so it cannot read worker traffic.

Worker agent

Runs on the host machine beside the engine (vLLM or an Ollama/Ollama-compatible server). Verifies the ticket, decrypts the request, fetches any weights it does not have, streams tokens back and signs a receipt for what it produced.

Holds one job in memory for the length of the request; nothing about the prompt is persisted.

Seeder swarm

Serves model weights as shards, peer to peer. A worker fetches only the pieces it needs from peers that already have them, each shard checked against the signed manifest before it is used.

A sender that serves a bad shard is rejected and banned for that fetch; the worker retries elsewhere and falls back to an HTTPS origin if it must.

Gateway

The OpenAI-compatible surface — /v1/chat/completions and /v1/models. It is what a stock OpenAI SDK or curl client talks to, so nothing has to be installed to try the API.

In gateway mode the gateway holds the plaintext, so it is also the billing authority — see §5.

SDK

The Python client for SDK mode. Verifies the signed ticket against the pinned coordinator key, encrypts the request to the worker and decrypts the response, so the coordinator never sees the prompt.

Only a Python SDK exists; a TypeScript port is specified, not implemented.

Ledger (PostgreSQL)

Accounts, keys, nodes, jobs, holds, ledger entries, receipts, attestations, model prices and the audit log. Every balance movement is an integer micro-credit entry with a unique idempotency key.

All state that matters survives a coordinator restart; in-process replay caches do not (documented limitation).

02Where the bytes go

Two control edges, one data edge

The coordinator talks to the client (placement, ticket, ack) and to the worker (registry, receipts). The prompt and the completion travel on a separate connection between client and worker. That split is the reason the coordinator can settle a job without being able to read it.

Coordinator (control plane)Worker + seeders (compute, weights)Ledger / settlement

03One request, seven steps

  1. Place the job, before the prompt is sent

    A placement call carries the model, the maximum tokens, the privacy tier and a price cap — never the prompt. The coordinator authenticates the key, prices the job, posts a hold against the balance and selects an eligible worker.

  2. Hand back a signed ticket

    The coordinator returns a ticket for that one job: job id, worker endpoint, worker public key and an expiry. In SDK mode the client verifies the coordinator’s signature before it sends anything, so a tampered ticket is refused at the edge.

  3. Verify, then decrypt

    In SDK mode the request travels over TLS 1.3 with per-request X25519 key agreement, HKDF-SHA-256 and AES-256-GCM. The worker checks the ticket, decrypts the payload and refuses a confidential job outright if it is not itself attested.

  4. Fetch only the weights that are missing

    If the model is not cached, the worker pulls shards from the seeder swarm through a signed manifest. Each shard is hash-checked against the manifest before it reaches the engine.

  5. Stream the tokens

    Token frames go back to the client as they are produced; in SDK mode they are encrypted to the client’s session key. A worker that dies mid-stream is detectable, because the stream simply ends without the receipt that follows it.

  6. Sign a receipt for what was produced

    The worker posts a signed receipt to the coordinator: token counts in and out, timing, model and the hash of what it returned. Receipts are what prove which code ran, not what a client claims happened.

  7. Settle once, then split

    The receipt settles the job, capped at the hold placed before it ran. Settlement is split 70 % compute / 10 % seeder pool / 20 % platform, and the whole thing is one atomic, idempotent operation — a replay returns the first result instead of double-posting.

Ordering that matters

The hold exists before the job does, the receipt is verified against the node the job was assigned to, and the receipt settles the job — not the client’s acknowledgement. Billing on the acknowledgement would make revenue optional for the payer: a client that never acked would leave the job unsettled and the expiry sweep would release the hold, so the completion would be free. The ack is evidence: a token-count or output-hash mismatch flags the job for review while the charge stands.

04Money is integer micro-credits

Balances, holds, charges and payouts are integer micro-credits — no floating point ever touches a balance. One credit is 1,000,000 micro-credits and $1 is 100,000,000, so every price is exact to the micro-credit. A job can never cost more than the hold placed for it, and a receipt that claims more than the hold is capped at the hold.

Split
70 % compute · 10 % seeder pool · 20 % platform, from the same settled amount
Idempotency
Every ledger entry carries a unique key, so a retried settlement returns the original result instead of posting twice
Concurrency
A per-account advisory lock plus “available = balance − open holds” means two simultaneous placements cannot both spend the same credit
Refunds
Only through the admin refund path; there is no automatic reversal in the request flow

Credit is denominated in USD. Each pack is priced in Kenyan shillings with the dollar equivalent shown, and purchases on this deployment run in test mode during the alpha.

05Two ways in, and who can see the prompt

SDK mode (Python)

  • · Client encrypts the request to the worker, end to end
  • · Coordinator never holds the prompt or the completion
  • · Receipt is the billing authority
  • · Redundant execution cannot be dispatched by the coordinator, so sampled jobs are recorded as not compared

Gateway mode (/v1)

  • · Any OpenAI client works unchanged
  • · The gateway relays plaintext in transit; prompts and completions are never logged or stored
  • · The gateway measures both sides, so it is the billing authority; the lower of its count and the receipt is billed
  • · Standard tier only — a confidential parameter is rejected, not downgraded

A node whose receipts bill above the gateway’s measurement in more than a configured share of its settled jobs — once it has enough of them to judge — is suspended and its pending payouts are held for an operator. Both counts are recorded as evidence for review; neither is treated as a verdict.

06Privacy tiers

TierStatusWhat the operator running the job can see
StandardAvailableThe prompt in plaintext. Traffic is encrypted in transit, but the host that runs the model can read the request and the response.
ConfidentialNot available in this buildNothing: placement requires a live, verified attestation bound to the worker’s session key, and no real attestation verifier ships yet, so no eligible host exists.

Treat the alpha as suitable for non-sensitive work only. This is a design property, not a configuration an operator can turn on.

07Weights arrive peer to peer

A model file is split into shards and published with a signed manifest: shard sizes, their hashes and a Merkle root. A worker that needs a model fetches only the shards it is missing from peers that already have them, verifies each one against the manifest before it reaches the engine, and bans a sender that serves corrupt data for that fetch. The manifest is signed, so a peer cannot substitute a different file set.

Shard transport is a seam, not a single implementation. The ShardPeer interface is what the fetch logic talks to: manifest verification, Merkle proofs, integrity checks, peer banning, resume and quota handling are transport-agnostic and tested with in-process peers.

The honest gap

The HTTPS origin fallback ships and is tested. A production peer-to-peer transport (libtorrent, or a JavaScript sidecar) does not — the browser’s WebTorrent layer cannot be driven from a Python worker — so a real swarm is unexercised. Weight delivery beyond the tested origin path is design plus verified protocol logic, not a shipped swarm.

08Verification is probabilistic

A deterministic, secret-keyed sample of jobs (3 % by default) is chosen for redundant execution. The job is re-run on a different operator and the two transcript commitments are compared: a disagreement loses the disagreeing node reputation and can suspend it, and the decision, the faulty node ids and the transcript root are written to the audit log.

That is not a proof. It cannot catch a wrong answer every sampled executor agrees on, it says nothing about the 97 % of jobs that are never sampled, and the comparison is an exact hash, so two honest but non-deterministic runs are recorded as inconclusive rather than attributed. In gateway mode the second execution is dispatched automatically and paid from platform revenue; in SDK mode the coordinator never holds the plaintext, so the sample is recorded as not dispatched and nothing is compared.

Nothing here should be read as “verified inference” for an unsampled job.

09What happens when something breaks

SituationBehaviour
A worker disappears before returning anythingThe job expires with no receipt; the hold is released and nothing is charged. The client re-places — there is no output to keep.
A worker fails after the first tokenThe partial text is returned with an explicit partial-result error. Billing uses the tokens the gateway actually saw, not what the client asked for.
Two settlements raceOne wins on the unique idempotency key; the loser returns the original result. A settled job cannot be re-priced by a late receipt.
A shard fails its hashIt is rejected before it reaches the engine, the sender is banned for that fetch, and the worker retries another peer or the HTTPS origin.
The coordinator restartsBalances, holds, jobs and receipts are in PostgreSQL; the job is still there. In-process replay caches and nonces reset, which is a documented limitation.
A confidential job meets a non-attested workerPlacement finds no attested worker and fails closed. The worker would refuse the job independently, so there is no downgrade path.

10What this page does not claim

  • · No availability figure, uptime or SLA: hosts join and leave, and there is no measured number to quote.
  • · No confidential tier in this build — the interfaces exist, a real attestation verifier does not.
  • · No gateway streaming guarantee across a failure: an error after the first token arrives in-band, and the job expires and releases its hold.
  • · Gateway support is chat completions only; tools, JSON mode, seed and sampling penalties, and embeddings return an explicit unsupported-parameter error.
  • · Only a Python SDK exists; the TypeScript port is specified, not built.
  • · The gateway’s token count is measured, not produced by a tokenizer — a small tolerance applies, and an engine whose tokenisation differs is flagged rather than corrected.
  • · Multi-instance coordination is not shipped: rate-limit buckets and replay caches live in process memory, and the Redis-backed version is not implemented.

The full list — including the simulated attestation adapter, the dev-stub inference backend and the free-credit abuse controls — lives in the limitations document that ships with the code.

Next

TCDN-AI architecture — how one request runs on a peer-to-peer GPU network | TorrentCDN