Some systems are experiencing issues
Scheduled Maintenance
[PAR] Security maintenance on 4 hypervisors

For security reasons, we will update the kernel of 4 Hypervisors in the Paris (PAR) region, more precisely in the PAR6 datacenter. Services (in particular databases) hosted on those hypervisors will be impacted : they will be unavailable between 5 and 10 minutes. Impacted hypervisors are:

hv-par6-008 hv-par6-011 hv-par6-012 hv-par6-020

Affected clients are directly and individually contacted by email with the list of impacted services, and options to avoid any impact. The maintenance will be planned in 2 operations of 2 hypervisors each, during the week of 18 to 22 Novembre 2024 between 22:00 and 24:00 UTC+1.

Past Incidents

Monday 19th August 2024

No incidents reported

Sunday 18th August 2024

No incidents reported

Saturday 17th August 2024

No incidents reported

Friday 16th August 2024

Infrastructure Read errors on telemetry cluster

The monitoring has detected errors on read queries of the telemetry cluster. We are investigating.

EDIT 21:30 UTC : We found out that the issue is related to indexes of the time series database, we are investigating the reason of the error.

EDIT 21:40 UTC : Some indexes had errors and have been rebooted, the estimate time to recover indexes is around 01:00 UTC.

EDIT 01:00 UTC : Indexes are still rebooting, the new estimate time is 03:00 UTC.

EDIT 02:47 UTC : Indexes are back online and query is available.

EDIT 07:30 UTC : We are running some maintenance operation, the query may be hanging a bit.

EDIT 08:00 UTC : We have shutdown the query to get some place to our maintenance query to run as fast as possible. We have found the root cause issue and we are fixing it, but to resolve read errors, we also need to achieve some clean up in parallel.

EDIT 09:40 UTC : We have turn on the query again, we have still maintenance queries running in the background.

EDIT 13:00 UTC : We have turn off the query, we are struggling the reads with the maintenance queries. To reduce the time of the recovery process, we took the decision to shutdown the read queries to keep the maximum compute space to the maintenance ones.

EDIT D+1 08:00 UTC : We have turn on the query again, the maintenance queries has finished during the night.

Thursday 15th August 2024

No incidents reported

Wednesday 14th August 2024

No incidents reported

Tuesday 13th August 2024

No incidents reported