CTC iOn EU - System performance issues (delayed message data)

CalAmp iOn – Message data delayed - 6/1/2026

Incident Date: 6/1/2026
Issue Started: 6/1/2026 02:46 am PT (Messages in iOn delayed)
Corrective action #1: 6/1/2026 04:50 am PT (Processing restored, processing backlog)
Corrective action #2: 6/1/2026 06:52 am PT (Controlled processing of backlog)
Event Cleared: 6/1/2026 12:29 pm PT (Backlog processed and data current)

Problem Statement

CalAmp iOn experienced a disruption to its message processing service. After processing was restored, a message delay continued until the backlog was completely processed.

Root Cause Analysis

An unexpected spike in device traffic volume exceeded the capacity thresholds of a message processing service within the CalAmp iOn platform. As traffic continued to increase, the service experienced resource exhaustion, which caused all message processing to halt.
Once processing was restored, a significant backlog of queued messages had accumulated. The platform continued to experience degraded throughput while working through this backlog in a controlled fashion, causing delayed message processing during the affected window. No data was lost. All messages were queued and delivered once processing was restored.

Corrective Action and Follow-Up

1. CalAmp increased the resource capacity of the affected message processing service to provide additional allowance for traffic spikes.
2. CalAmp enhanced internal monitoring and logging to enable earlier detection of degraded processing performance.
3. CalAmp added additional iOn message processing capacity to improve resilience and throughput.
4. CalAmp has initiated an architectural review of the technology stack to identify further opportunities to improve reliability, scalability, and fault tolerance.