How the SDK is put together

The pieces, what each one owns, and why the boundaries are where they are. Read this once and the rest of the documentation stops being a list of functions.

The layers#

                 ┌──────────────────────────────────────┐
your firmware ──►│  xcaicx.h   (stable C ABI)           │
                 ├──────────────────────────────────────┤
                 │  Engine    entitlements, stream caps │
                 │  Meter     per-module inference count│
                 │  License   Ed25519 verify, binding,  │
                 │            clock-rollback, CRL       │
                 ├──────────────────────────────────────┤
                 │  Pipeline  convert → infer → track   │
                 │            → zones/tripwires → events│
                 ├──────────────────────────────────────┤
                 │  Backend   reference | ONNX Runtime  │
                 │            | TensorRT | RKNN | Hailo │
                 └──────────────────────────────────────┘

Why a C ABI and not C++#

Camera vendors build with whatever toolchain their SoC BSP ships — GCC 7 on a Rockchip Yocto image, NDK clang on Android, MSVC on a Windows NVR. A C++ ABI breaks across every one of them: name mangling, exception tables, and the standard library's own ABI all differ.

So the boundary is C, and it is versioned structurally:

• Structs carry a leading size_t struct_size. Callers set it to sizeof the struct they compiled against, and the library uses it to decide which fields are actually present. New fields are only ever appended.
• Enum values are never renumbered. New values are appended.
• Functions are never removed within a major version.

That is why xcaicx_config_init() is mandatory before touching a config struct: it is what stamps struct_size correctly and keeps you forward-compatible. See Versioning & ABI policy.

Engine#

An engine is one licensed SDK instance. Creating it:

1. loads the token (from license_token or license_path),
2. verifies the Ed25519 signature against the compiled-in public key,
3. checks binding, term, grace, rollback and revocation,
4. checks that every module in cfg.modules is entitled by the token,
5. selects a backend,
6. opens state_dir and restores metering counters.

Any failure at that point returns a specific status rather than a generic one, because the field technician needs to know which. See Status codes.

The engine is thread-safe for submit/poll from different threads. A single stream must be driven from one thread at a time.

Streams#

A stream is one video source. max_streams comes from the licence, not from your configuration, so opening more than the SKU allows fails with LIMIT_EXCEEDED.

Per-stream settings:

Metering is attributed per stream by name, which is why the name shows up in logs and in usage records.

Frames#

A xcaicx_frame is a borrowed view of caller-owned pixels. The SDK never takes ownership and never frees data; the pointer must stay valid for the duration of the call only.

Supported formats are RGB8, BGR8, GRAY8, NV12 and I420. NV12 and I420 are first-class precisely because they are what ISPs emit — converting to RGB before handing frames over costs a memcpy on the boards least able to afford one.

Internally every frame is converted once into a packed-RGB plus BT.601 luma representation, then box-filter downscaled so the longest side is 320px for analysis. Box filtering rather than nearest-neighbour is deliberate: nearest aliases thin structures — wires, plate glyphs, hairline scratches — in and out between frames, which destabilises tracking.

The pipeline#

Each frame runs:

1. Convert — to RGB + luma, with stride handling.
2. Infer — every enabled module, each charged one metered inference.
3. NMS — suppress overlapping same-class boxes. Only within a class: a person standing in front of a vehicle is two detections, not one.
4. Track — greedy IoU association with velocity coasting.
5. Rules — zones, tripwires and dwell timers turn tracks into events.

Tracking#

A greedy IoU tracker with defaults max_age=15, min_hits=3, iou_threshold=0.30.

• Tracks coast forward on their last velocity each frame, so a fast mover is compared against where it should be rather than where it was.
• Velocity is exponentially smoothed (0.7 old, 0.3 new); raw frame-to-frame deltas are far too jittery to predict with.
• Association requires class agreement, so a person track cannot absorb an overlapping vehicle detection.
• A track is reported only once min_hits is reached and it was seen this frame. A coasting track is real to the tracker, but reporting a box where nothing was detected would put phantom objects into your event stream.

Greedy rather than Hungarian assignment because track and detection counts are both in the tens: the optimal algorithm would cost more to run than it saves in match quality.

Entitlements#

The four modules map one-to-one onto bits in the licence token's ent object. The engine refuses to start a module whose bit is false, with ENTITLEMENT_DENIED. This is enforcement in the runtime, not a sales-contract convention — widening an entitlement requires forging Ed25519, not editing a config file.

Metering#

One inference is counted per module per frame. Running detect and ppe over a 30fps stream is 60 metered calls per second.

Counters live in state_dir/usage.json, are checkpointed periodically and on shutdown, and resume across reboots — a reboot loop is not a way to get free inference. Full detail in Metering & billing.

The licensing authority#

A FastAPI service with three audiences:

• Devices — /v1/activate, /v1/usage, /v1/revocations. Unauthenticated at activation, because the batch token is the credential; bearer-authenticated afterwards.
• Operators — /v1/admin/*, behind an admin key.
• Monitoring — /healthz.

The signing key never leaves that process. Devices verify with the public key compiled into the SDK, so nothing on a camera can mint a licence. See Licensing REST API and Running the authority.

State on the device#

state_dir holds:

Writes are tmp-file + fsync + rename, so a camera yanked off PoE mid-write does not corrupt it. It must be persistent storage — see state_dir requirements.

Next#

• Licensing model — tokens, batches, activation, revocation
• Metering & billing — what is counted and how it reaches an invoice
• Backends & models — reference vs production inference