Verkada Command
All Systems Operational
Login System Operational
Streaming Live Footage Operational
Streaming Historical Footage Operational
Sensors Operational
Access Control Operational
Alarms Operational
Events and Notifications Operational
Computer Vision Operational
Intercom Operational
Guest Operational
Mailroom Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Sep 21, 2023

No incidents reported today.

Sep 20, 2023
Resolved - This incident has been resolved.
Sep 20, 14:06 PDT
Monitoring - A fix has been implemented and we are monitoring the results.
Sep 20, 13:25 PDT
Identified - The issue has been identified and a fix is being implemented.
Sep 20, 13:19 PDT
Investigating - Some users may be experiencing issues loading Verkada Command. We are actively investigating this issue and will provide an update shortly.
Sep 20, 13:14 PDT
Sep 19, 2023

No incidents reported.

Sep 18, 2023
Resolved - This incident has been resolved.
Sep 18, 23:27 PDT
Update - We are continuing to work on a fix for this issue.
Sep 18, 20:52 PDT
Update - We are continuing to work on a fix for this issue.
Sep 18, 16:15 PDT
Identified - We are currently experiencing connectivity issues due to an ongoing Amazon Web Services outage in our primary region. Our engineering team is actively monitoring the situation. Some services may be intermittently unavailable. We apologize for any inconvenience and are working to resolve the issue as quickly as possible. Updates from AWS can be found here: https://health.aws.amazon.com/health/status
Sep 18, 16:06 PDT
Update - We are continuing to monitor for any further issues.
Sep 18, 15:35 PDT
Update - We are continuing to monitor for any further issues.
Sep 18, 14:46 PDT
Update - We are continuing to monitor for any further issues.
Sep 18, 14:41 PDT
Monitoring - A fix has been implemented and we are monitoring the results.
Sep 18, 14:35 PDT
Identified - The issue has been identified and a fix is being implemented.
Sep 18, 12:41 PDT
Investigating - Some users may be experiencing degraded performance loading certain parts of Verkada Command. We are actively investigating this issue and will provide an update shortly.

Requests to create camera archives are are not being processed.

Sep 18, 11:29 PDT
Resolved - This incident has been resolved.
Sep 18, 11:29 PDT
Update - We are continuing to work on a fix for this issue.
Sep 18, 10:55 PDT
Identified - The issue has been identified and a fix is being implemented.
Sep 18, 10:13 PDT
Update - We are continuing to investigate this issue.
Sep 18, 09:32 PDT
Investigating - We are currently experiencing issues with high latency on notifications. We are actively investigating this issue and will provide an update shortly.
Sep 18, 09:28 PDT
Sep 17, 2023

No incidents reported.

Sep 16, 2023

No incidents reported.

Sep 15, 2023

No incidents reported.

Sep 14, 2023
Resolved - This incident has been resolved.
Sep 14, 13:29 PDT
Monitoring - A fix has been implemented and we are monitoring the results.
Sep 14, 13:08 PDT
Identified - We have identified an issue with our person of interest notification pipeline. Some notifications may be delayed. We are actively investigating this issue and will provide an update shortly.
Sep 14, 11:58 PDT
Sep 13, 2023
Resolved - The backlog of events has been processed. Event processing latency is back to normal.
Sep 13, 21:22 PDT
Monitoring - A fix has been deployed and we seem to be making progress in processing the backlog of events. Event processing latency seems to be going down overall, though some customers might still be seeing high latency. We will continue to monitor and update on progress.
Sep 13, 16:40 PDT
Identified - The revert of yesterday's change did not fix the issue. We identified a dependency on a downstream service as another cause of high latency. We are removing this dependency and will update here when we have deployed the fix.

Also, customers should note that aux input-triggered lockdowns and unlocks will also be slowed down by this issue, as will panic button-triggered lockdowns in Alarms.

Sep 13, 13:36 PDT
Monitoring - There are significant delays in Access Control event processing. Recent events may not appear in Command, either in the door event view, the user event view, the Reports view, or Alert Inbox. These delays will also delay notifications for alerts based on any events whose processing is delayed.

The root cause was an attempted fix for intermittent slowness in event processing for some customers, which unfortunately made processing slower for all customers.

The change has already been reverted and we are monitoring. We will update here when event processing latency is back to normal.

Sep 13, 09:55 PDT
Resolved - This incident has been resolved.
Sep 13, 10:03 PDT
Monitoring - A fix has been implemented and we are monitoring the results.
Sep 13, 09:57 PDT
Investigating - We are currently experiencing issues with high latency on notifications. We are actively investigating this issue and will have an update shortly.
Sep 13, 09:01 PDT
Sep 12, 2023
Resolved - This incident has been resolved.

The root cause was a change to our event processing code meant to better spread processing load between nodes, which increased pressure on caches and, under high load, resulted in processing nodes timing out. We have fixed the issue.

Sep 12, 04:28 PDT
Identified - The issue has been identified and a fix is being implemented.
Sep 11, 08:34 PDT
Sep 11, 2023
Sep 10, 2023

No incidents reported.

Sep 9, 2023

No incidents reported.

Sep 8, 2023
Resolved - All LTE devices are functioning normally at this stage.
Sep 8, 13:53 PDT
Update - Most cellular backup modules should have normal connectivity at this stage. However, a small number of devices appear to still be experiencing connectivity issues. We are working with our upstream partner to resolve these cases.
Sep 6, 19:38 PDT
Update - Our provider has indicated that they have implemented a fix and impacted devices should be gradually coming back online. Further updates are available here: https://status.hologram.io.
Aug 30, 09:29 PDT
Update - Our provider has indicated that the previously identified fix is not viable. A new fix is being implemented, however the timeline for full resolution is not certain at this stage. Further updates are available here: https://status.hologram.io.
Aug 29, 20:55 PDT
Identified - Our LTE networking provider has identified the root cause of the issue and is actively resolving. They expect full resolution between 0700 and 0800 UTC. Further updates are available here: https://status.hologram.io.
Aug 29, 16:28 PDT
Investigating - Some cellular backup modules are unable to connect to the network due to an issue with our SIM provider. Our provider has found the root cause and is progressively fixing affected devices. They expect the issue to be resolved fully later this evening.
Aug 29, 12:46 PDT
Resolved - This incident has been resolved.
Sep 8, 09:29 PDT
Monitoring - A fix has been implemented and we are monitoring the results.
Sep 8, 08:47 PDT
Investigating - We are currently experiencing an issue with delayed notifications. We are actively investigation this issue and will provide an update shortly.
Sep 8, 08:24 PDT
Sep 7, 2023

No incidents reported.