Some systems are experiencing issues
Scheduled Maintenance
[PAR] Security maintenance on 4 hypervisors

For security reasons, we will update the kernel of 4 Hypervisors in the Paris (PAR) region, more precisely in the PAR6 datacenter. Services (in particular databases) hosted on those hypervisors will be impacted : they will be unavailable between 5 and 10 minutes. Impacted hypervisors are:

hv-par6-008 hv-par6-011 hv-par6-012 hv-par6-020

Affected clients are directly and individually contacted by email with the list of impacted services, and options to avoid any impact. The maintenance will be planned in 2 operations of 2 hypervisors each, during the week of 18 to 22 Novembre 2024 between 22:00 and 24:00 UTC+1.

Past Incidents

Friday 29th December 2023

Cellar [NORTH] Partial Cellar requests timeout

Between 16:58 UTC and 17:03 UTC, the Cellar service on the North region timed out on some requests. The faulty component has been decommissioned and further investigations will be done to understand the source of the timeouts. The service is currently up and running.

EDIT 2023-12-30 00:51 UTC: The problem has been identified and resolved. The component is back in the pool and is working as expected. This incident is now over.

Thursday 28th December 2023

Access Logs [Metrics] Elevated queries error rate

We are seeing elevated error rate for metrics queries due to the underlying storage system. The problem has been identified and we are working toward its resolution. This can impact some of the grafana dashboards or API queries.

EDIT 09:44 UTC: The issue is not fully resolved yet but we are seeing improvements. We continue working on the issue.

EDIT 11:04 UTC: Queries are now working since 10:15 UTC, we continue monitoring to ensure everything is working as intended.

EDIT 15:43 UTC: Everything is back to normal, this incident is now over.

Wednesday 27th December 2023

No incidents reported

Tuesday 26th December 2023

No incidents reported

Monday 25th December 2023

Infrastructure [RBX] Unreachable hypervisor

An hypervisor is unreachable, we are investigating.

EDIT 03:17 UTC : There is no database affected on this hypervisor and applications has been redeployed.

EDIT 03:30 UTC : The hypervisor has been reboot and everything comes back to normal

Sunday 24th December 2023

No incidents reported

Saturday 23rd December 2023

Infrastructure [RBX] Unreachable hypervisor

An hypervisor is unreachable, we are investigating.

EDIT 3:37 UTC : The issue seems to be related with the following OVH incident : https://bare-metal-servers.status-ovhcloud.com/incidents/x135vv46x85l

EDIT 3:45 UTC : Applications on this hypervisor are currently redeploying and there is no such addons on it, we also have remove temporarely the A record from domain.rbx.clever-cloud.com to solve connection issues

EDIT 4:00 UTC : Applications have been redeployed, we are waiting after ovh folk to go further

EDIT 05:30 UTC : The hypervisor is reachable again, we are starting the recovery process

EDIT 05:45 UTC : The recovery process is over, everything works normally, the load balancer ip affected by the incident will be put later in the pool. for the record, the ip is 87.98.177.176 for domain.rbx.clever-cloud.com.