RT-DETR-Lite is interesting because of the tile-grid constraint, not the COCO benchmark.

RT-DETR-Lite is the smaller member of Baidu's RT-DETR (Real-Time DEtection TRansformer) family, an end-to-end transformer object detector designed to run at real-time speeds on commodity GPUs. RT-DETR's whole pitch is that it drops the two parts of the YOLO-class pipeline that get expensive at higher quality, hand-tuned anchor boxes and post-hoc non-maximum suppression, and replaces them with a transformer encoder-decoder that emits a fixed set of object queries directly. The lite variants (RT-DETR-R18, RT-DETR-R34, and the smaller pruned configurations sometimes called RT-DETR-Lite or RT-DETRv2-S in derivative work) trade some COCO mAP for a tighter compute envelope so the model can actually finish in time on edge hardware. The reference implementation lives at github.com/lyuwenyu/RT-DETR.

Two reasons, and they are not the reasons the COCO benchmarks emphasize. First, the multiview retrofit case classifies a single composite frame containing up to 25 tiled sub-views, and the model has to find people and vehicles on every tile, including the ones with hardly any pixels. RT-DETR's transformer attention is more forgiving of small objects in cluttered backgrounds than NMS-based heads, which often suppress overlapping small detections in busy tiles. Second, RT-DETR's no-NMS output lets the per-class router downstream treat the detections as a stable stream rather than a flickering set, which matters for state classes like loitering or tailgating that are computed across frames. None of this is unique to RT-DETR-Lite specifically, the entire DETR family has these properties, but the lite variant is the one that actually fits the latency budget.

A typical legacy DVR draws a 1080p HDMI multiview at 30fps. The retrofit classifier has roughly 33ms per composite frame to do the entire pipeline, decode, model forward pass, de-tile to per-camera coordinates, run the per-class router. A 5x5 tile grid means each tile is 384x216 inside the 1080p composite, and the model must see all 25 simultaneously. If the model takes 25ms by itself you have 8ms left for everything else. That is why a sub-10ms-image-class model like RT-DETR-Lite or a small YOLOv8-N gets shortlisted, and why a heavier RT-DETR-R50 or DETR-R101 does not. The lite variant is not chosen for accuracy points, it is chosen because it leaves headroom.

There is nothing model-architectural that handles the tile grid; the tiling is handled by the surrounding pipeline. The model treats the 1080p composite as a single image and emits detections in composite coordinates. The pipeline records the tile-grid template at install time (5x5, 4x4, 3x3, or 2x2 against a known DVR resolution), then snaps every output bounding box to its tile by integer division on the composite resolution, and looks up the camera ID from the OCR'd tile name strip. The model never knows the tiles exist. What RT-DETR-Lite buys you on top of this is that the transformer's global attention sees the whole composite at once, so a person partially crossing a tile boundary is still a single detection, where an NMS-based detector with overlapping anchor proposals can split the same person into two boxes that get suppressed against each other.

Yes, in any practical configuration. The reference implementation targets CUDA, and the published latencies (around 4 to 9 ms per image at 640x640 on a T4 for the R18 backbone variant) are GPU numbers. On a CPU-only box, RT-DETR-Lite is not real-time at multiview composite resolutions. The pragmatic edge target is a small CUDA card, an NVIDIA Jetson Orin module, or an integrated NPU with an ONNX or TensorRT export of the model. The official repo ships TensorRT export instructions and the speed claims in the paper assume that path. If your edge form factor is CPU-only, you are looking at a quantized YOLOv8-N or NanoDet variant instead, and you will pay for it in small-target recall on the dim tiles.

RT-DETR's own paper benchmarks against YOLOv8 and the DETR variants on COCO, where the small RT-DETR-R18 beats YOLOv8-N on mAP at comparable latency. For the multiview case the more interesting axis is small-object recall on cluttered composites, which neither paper measures, because nobody runs that benchmark publicly. In practice on real DVR multiview frames, the lite RT-DETR variants and a tuned YOLOv8-S trade blows: RT-DETR-Lite is more stable across tile boundaries and better on partial occlusions, YOLOv8-S has a slight edge on sharp small objects in well-lit tiles. The original DETR is not a candidate, it is too slow at multiview composite resolutions, that is the whole reason RT-DETR exists.

It is one of three boxes. Box one is the HDMI capture, the multiview composite enters the edge device over a single HDMI cable from the DVR. Box two is the model, RT-DETR-Lite or a comparable lite variant produces detections in composite coordinates. Box three is the de-tile-and-route layer, which snaps every detection to its tile using the stored tile-grid template, attaches a camera ID and zone label, and sends the labeled detection into the per-class router (the part that decides whether anything fires based on class plus zone plus time of day). The model is the cheapest part of this stack to swap. If a better lite detection model ships next year, the rest of the pipeline does not change.

Yes. The official RT-DETR repository at github.com/lyuwenyu/RT-DETR is published under the Apache 2.0 license, which matches what most production teams want from a model dependency, permissive for commercial use with the standard attribution and patent clauses. The PaddlePaddle reference and the PyTorch reference both ship there. Pretrained weights for the various backbone variants (R18, R34, R50, R101, and the lite/scaled configurations) are linked from the repo readme. As of mid-2026 the v2 line is the actively maintained branch and the lite variants in v2 inherit the same license.

You can, but you should not, and the reason has nothing to do with the model and everything to do with the input. The DVR's motion engine is a per-tile pixel-difference threshold. By the time a clip is in the motion-alert stream it has already been filtered against that threshold, so any sub-threshold event (a slow approach in low light, a small distant person) is gone before the model sees anything. RT-DETR-Lite running on top of this stream cleans up false positives but inherits the DVR's recall ceiling. The HDMI direct architecture, where RT-DETR-Lite runs on every multiview composite frame at output rate, is what actually raises recall.

Cyrano runs a lite real-time detection transformer in the same family, plus a per-class context router on top, on an edge device that plugs into the back of an existing DVR over HDMI. One device handles up to 25 camera feeds (the 5x5 multiview), inference runs locally with no cloud round trip, and operators get a phone call with a 10-second clip when the router decides an event is HIGH THREAT given class plus zone plus time of day. At a 180-unit Class C property in Fort Worth the same architecture caught 20 incidents including a break-in attempt in the first month. Hardware is $450 one-time, software is $200 per month per property starting month two, and install on site takes about two minutes.