Some systems are experiencing issues
Scheduled Maintenance
[PAR] Security maintenance on 4 hypervisors

For security reasons, we will update the kernel of 4 Hypervisors in the Paris (PAR) region, more precisely in the PAR6 datacenter. Services (in particular databases) hosted on those hypervisors will be impacted : they will be unavailable between 5 and 10 minutes. Impacted hypervisors are:

hv-par6-008 hv-par6-011 hv-par6-012 hv-par6-020

Affected clients are directly and individually contacted by email with the list of impacted services, and options to avoid any impact. The maintenance will be planned in 2 operations of 2 hypervisors each, during the week of 18 to 22 Novembre 2024 between 22:00 and 24:00 UTC+1.

Past Incidents

Wednesday 31st May 2023

Access Logs The metrics storage layer is unavailable

The monitoring detect errors on the metrics / access logs storage layer. We are investigating.

EDIT 11:46 UTC : We have found the issue and fixed it. We are recovering the lag.

EDIT 13:19 UTC: The lag has been consumed, everyhting is operating normaly

Tuesday 30th May 2023

No incidents reported

Monday 29th May 2023

No incidents reported

Sunday 28th May 2023

Infrastructure [Montreal] Multiple hypervisors are unreachable

An hypervisor on the Montreal zone is unreachable. One of the FSBucket servers of the zone is hosted on it and is therefore unreachable too. This might impact PHP applications as well as any applications using an FSBucket hosted on this server.

We are awaiting information from our infrastructure provider regarding this incident.

EDIT 19:53 UTC: It seems like multiple servers are impacted at the same time, we believe it to be an issue with a specific OVH rack or room. Multiple services on the zone are thus impacted. We are looking at ways to mitigate the issues.

EDIT 20:05 UTC: The servers are reachable again since a few minutes. We are currently making sure everything is fine. OVH incident can be followed here: https://bare-metal-servers.status-ovhcloud.com/incidents/k664s90jxfj0

EDIT 20:15 UTC: Servers in the impacted rack couldn't reach each other up until now. It could have prevented some services to correctly work. It seems like OVH fixed it before we could report it to them. We continue to making sure everything is working as expected.

EDIT 20:36 UTC: The incident is over. We are redeploying all the applications of the zone to be on the safe side.

Saturday 27th May 2023

No incidents reported

Friday 26th May 2023

No incidents reported

Thursday 25th May 2023

Access Logs Metrics: Ingestion issue leads to missing data points

We are currently having an ingestion issue on our metrics cluster. The root cause has been identified and we are currently working on a fix. Until this incident is fixed, metrics data points might be missing from your metrics dashboards. Access logs are also impacted but will be re-queued later.

EDIT 14:14 UTC: Metrics ingestion is now back to normal. Access logs are being re-queued and are currently lagging a bit.

EDIT 14:20 UTC: Access logs have been ingested and are now up-to-date. The incident is now over.

EDIT 16:25 UTC: The problem came back, we are working on it.

EDIT 16:56 UTC: The problem is now solved again. Another root cause has been identified and has been fixed.