Some systems are experiencing issues
Scheduled Maintenance
[PAR] Security maintenance on 4 hypervisors

For security reasons, we will update the kernel of 4 Hypervisors in the Paris (PAR) region, more precisely in the PAR6 datacenter. Services (in particular databases) hosted on those hypervisors will be impacted : they will be unavailable between 5 and 10 minutes. Impacted hypervisors are:

hv-par6-008 hv-par6-011 hv-par6-012 hv-par6-020

Affected clients are directly and individually contacted by email with the list of impacted services, and options to avoid any impact. The maintenance will be planned in 2 operations of 2 hypervisors each, during the week of 18 to 22 Novembre 2024 between 22:00 and 24:00 UTC+1.

Past Incidents

Sunday 10th September 2023

No incidents reported

Saturday 9th September 2023

No incidents reported

Friday 8th September 2023

No incidents reported

Thursday 7th September 2023

No incidents reported

Wednesday 6th September 2023

Infrastructure [JED] Hypervisors unreachability

We are currently experiencing unreachable hypervisors on the JED region. We are investigating the issue.

EDIT 18:50 UTC: The hypervisors are back online since 25 minutes now, all services were restarted by our monitoring.

Infrastructure [SCW] Hypervisor has crashed

An hypervisor has crashed, we are currently investigating the root cause

EDIT 18:45 - The hypervisor had a kernel panic. During the reboot operation the kernel has been upgraded and this issue should not occur again.

Tuesday 5th September 2023

Infrastructure [PAR] Network instabilities

We've seen network instabilities on the PAR region. It is currently resolved but we are still investigating the root cause.

EDIT 06:34 UTC: The problem is back with elevated packet loss. Our network provider is currently having an incident and is looking into the issue.

EDIT 06:46 UTC: Some DNS domains for services hosted on other regions may also have issues to resolve because their authoritative server is currently hosted on the PAR region.

EDIT 06:55 UTC: The incident is still ongoing and our network provider is still looking into the issue.

EDIT 07:20 UTC: Our upstream network provider is currently experiencing a DDoS attack. We are currently looking to use an alternative network transit to avoid going through the upstream network provider.

EDIT 07:47 UTC: We are seeing improvements for the last 20 minutes. We still are waiting for a confirmation of the issue resolution.

EDIT 07:58 UTC: We are seeing some loss again.

EDIT 08:15 UTC: The DDoS is still happening. It's partially mitigated. We still see some loss, but there is less impact globally.

EDIT 10:54 UTC: We still see loss from time to time, but much less that before. We keep an eye on the situation.

EDIT 15:45 UTC: Most of the ddos is mitigated, we didn't have any loss those past few hours, we still monitor the situation.

EDIT 2023-09-06 15:24 UTC: No more instabilities were detected since yesterday. The incident is now over.

Monday 4th September 2023

MySQL shared cluster New databases issues

When ordering a new database it can take time to reach them (databases are correctly created)

[EDIT] 21:38 UTC the root cause was identified and a path deployed