Bad labels cause AI shadow detections

A May 21 post on X from the account tagged @videoraiq circulated an example of an AI surveillance system appearing to detect shadows rather than a real threat target. The post included still frames in which detection boxes were placed over dark shapes on the ground instead of cleanly isolating the person in view. The images were presented as a case of bad annotation feeding bad model behavior. The post itself did not provide a full technical audit, but it pointed to a familiar computer-vision failure mode: a model learns whatever the labels reward. ### How can a model end up detecting a shadow instead of a person? Object-detection systems are trained on labeled images, and those labels tell the model what visual patterns to associate with a category. If the training boxes repeatedly include background artifacts — a shadow, glare patch, doorway edge or weapon-adjacent blur — the model can learn those cues instead of the intended object. Ultralytics, whose documentation covers computer-vision data collection and annotation, says data quality directly affects model performance and that poor labeling can produce false positives and false negatives. The problem is not limited to one post. A 2025 Fast Company report on AI surveillance cited a Transport for London test in which automated CCTV produced more than 44,000 alerts, with many false or misdirected, and described systems struggling to distinguish ordinary behavior from actual risk. ### What does “bad annotation” look like in practice? (docs.ultralytics.com) Bad annotation often means boxes that are too loose, too tight, inconsistent across frames, or attached to the wrong class. In surveillance footage, that can happen when labelers mark the whole dark silhouette around a person rather than the person’s visible body, or when they include cast shadows inside the training box so often that the shadow becomes a predictive feature. (fastcompany.com) Research on noisy labels in object-detection datasets has found that annotation quality strongly affects model performance, especially in complex detection tasks. Real-world video makes that worse. CheckVideo, a surveillance vendor describing field failures, lists shadows, reflections, passing headlights, tree movement and weather among common sources of false alerts in deployed systems. The company says systems trained for cleaner conditions often fail in messy live environments. (link.springer.com) ### Why would a surveillance system be especially vulnerable to this? Surveillance models operate on low-light footage, oblique camera angles, compression artifacts and constant background motion. Those conditions make it easier for a model to latch onto shortcuts — what machine-learning researchers often call spurious correlations — because the shortcut may be visually stronger than the object of interest. A dark, elongated shape on pavement can be more stable across frames than a partially occluded person. (checkvideo.com) Cloud providers’ documentation reflects the same dependency on labeled training data. Google says Vertex AI can be used to train custom computer-vision models for object detection, while Amazon Rekognition Custom Labels says users build specialized image-analysis capabilities by training on labeled images. Neither platform treats labeling as a minor setup step; it is the basis of what the model learns. (checkvideo.com) ### What would a team check after seeing an example like this? The first check is the dataset: whether shadows, reflections or empty background regions were repeatedly included in positive examples. The second is frame-by-frame review of annotation guidelines, especially for partially visible people, weapons, bags or hands. The third is evaluation on hard cases — dawn, dusk, wet pavement, backlighting and crowded scenes — before deployment. (docs.cloud.google.com) Ultralytics says data collection and annotation strategy is foundational to model success, and surveillance operators say false positives quickly erode trust in live systems. The May 21 X post does not establish how the underlying model was built or who deployed it. It does provide a concrete illustration of a broader risk in surveillance AI: if the labels are wrong, the model can be wrong in a very specific way. The next public evidence, if any, is likely to come from follow-up posts by @videoraiq or from additional examples showing the same frames with underlying annotations or model outputs. (docs.ultralytics.com)