Past Incidents

Monday 8th April 2024

Metrics [GLOBAL] Metrics query unavailable

The metrics query is currently unavailable as some indexing shared are offline. We are working to get them up as quickly as possible. There is no impact on ingestion pipeline and storage layer.

UTC 14:35 Services are back to normal

Access Logs [Global] Access logs ingestion lags

We have some lags on the ingestion of access logs. We are working on it.

EDIT 12:00 UTC : we have consume the lag for 75% of the access logs, we are working on the remaining ones.

EDIT 14:45 UTC : we are in-sync for 80% of access logs, we are consuming the remaining ones (eta 22h to consume the lag)

EDIT 16:41 UTC : we have consumed all remaining access logs

Sunday 7th April 2024

cleverapps.io domains Cleverapps.io certificate has expired

The *.cleverapps.io wildcard certificate failed to renew. We are currently renewing it.

Edit 15:05 UTC: we renew the certificate. It should appear on the load balancers in a few minutes. Edit 15:11 UTC: the certificate has been deployed on all cleverapps.io load balancers. Incident is over.

Saturday 6th April 2024

No incidents reported

Friday 5th April 2024

No incidents reported

Thursday 4th April 2024

Cellar Cellar North: Requests slowness / timeouts

We identified an issue where certain requests to the Cellar service on the North region might have timed-out or were slower than usual to respond. The issue has been fixed but we are looking for the underlying cause and we keep monitoring the situation.

EDIT 11:56 UTC+2: A storage node was unexpectedly unresponsive and incurred timeouts in various parts of the storage cluster. It started around 10:31 UTC+2 and went unnoticed until 11:26 UTC+2. Additional monitoring will be put in place to better handle this situation.

Wednesday 3rd April 2024

No incidents reported

Tuesday 2nd April 2024

No incidents reported