Backends & models

What the reference backend is for, what it is not, and how to get to production accuracy.

The two worlds#

The reference backend#

Classical contrast, edge and colour heuristics with no model files and no accelerator.

It is genuinely useful:

• It runs the whole pipeline — conversion, inference, NMS, tracking, zones, events, metering — so an OEM can validate frame plumbing, licensing and event handling before any model reaches them.
• It is deterministic, which makes CI meaningful.
• It has no dependencies, so it builds anywhere the compiler does.

What each module actually does#

Median and MAD rather than mean and sigma in the defect path is deliberate: a real defect must not be allowed to inflate the very statistics used to detect it. The z ≥ 6 gate is deliberately conservative, because on an inspection line a false reject stops production, which costs more than a cosmetic blemish caught at the next station.

Moving to production#

cmake -S core -B build -DXCAICX_WITH_ORT=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

cfg.model_dir = "/opt/xcaicx/models";
cfg.backend   = XCAICX_BACKEND_AUTO;   /* or pin one explicitly */

XCAICX_BACKEND_AUTO selects the best available backend at runtime and falls back to reference if nothing else initialises.

cfg.backend = XCAICX_BACKEND_ORT_CPU;
if (xcaicx_engine_create(&cfg, &engine) == XCAICX_ERR_MODEL_LOAD) {
    fatal("model bundle missing or corrupt — do not ship analytics in this state");
}

Available backend values#

Enum values are never renumbered, so a binary compiled against today's header keeps working — see Versioning & ABI policy.

Writing a backend#

core/src/infer/backend.hpp is the whole extension surface. Adding TensorRT, RKNN or Hailo means implementing that one interface and nothing else — no changes to the engine, the pipeline, the licensing or the metering.

class Backend {
public:
    virtual ~Backend() = default;

    virtual const char *name() const = 0;
    virtual xcaicx_backend kind() const = 0;

    // Called once at engine creation. `model_dir` may be empty.
    virtual bool init(const std::string &model_dir,
                      xcaicx_module_mask modules,
                      int num_threads,
                      std::string *err) = 0;

    // May this backend serve this module at all?
    virtual bool supports(xcaicx_module m) const = 0;

    // A human-readable note logged at start. Say what you cannot do.
    virtual std::string notes() const = 0;

    // One module, one frame. Append detections in normalised coordinates.
    virtual bool run(xcaicx_module m,
                     const Image &img,
                     float threshold,
                     std::vector<RawDetection> &out,
                     std::string *err) = 0;
};

Contract notes that are easy to get wrong:

• Coordinates are normalised to the analysis image, [0,1]. Do not return pixels.
• Do not run NMS. The pipeline does it, across all modules, suppressing only within a class.
• run() is called once per enabled module per frame. Metering has already been charged by the time you are called, so returning early on an unsupported module still costs the customer an inference — return true with no detections rather than failing.
• notes() is logged at INFO on every engine start. Use it to say what your backend cannot do; the reference backend uses it to declare that ANPR returns boxes without text.

Register it in backend_factory.cpp.

Model bundles#

model_dir holds the model files for the modules you have enabled. A bundle is tied to a backend — an ONNX bundle is not an RKNN bundle — and to a module set.

Ship the bundle inside the firmware image, not downloaded at first boot. A camera that needs the internet to become useful is a camera that fails commissioning in exactly the industrial sites you most want to sell into.

Accuracy, honestly#

There is no accuracy number on this page, and there is none on the website either.

A detection number without the cameras, the lens, the mounting height, the lighting and the test set attached is not information. The useful version of that conversation starts with your footage from your site, and produces numbers that mean something for your deployment. That is a conversation with us, not a datasheet row.

What the SDK does guarantee is the parts around the model: the pipeline, the tracking, the event semantics, the licensing and the metering behave identically whichever backend is underneath.

Next#

• Performance & tuning — fitting the budget once a real model is in
• Cross-compiling — building with ORT for your target
• Concepts — where the backend sits in the pipeline