Some systems are experiencing issues
Scheduled Maintenance
[PAR] Security maintenance on 4 hypervisors

For security reasons, we will update the kernel of 4 Hypervisors in the Paris (PAR) region, more precisely in the PAR6 datacenter. Services (in particular databases) hosted on those hypervisors will be impacted : they will be unavailable between 5 and 10 minutes. Impacted hypervisors are:

hv-par6-008 hv-par6-011 hv-par6-012 hv-par6-020

Affected clients are directly and individually contacted by email with the list of impacted services, and options to avoid any impact. The maintenance will be planned in 2 operations of 2 hypervisors each, during the week of 18 to 22 Novembre 2024 between 22:00 and 24:00 UTC+1.

Past Incidents

Thursday 24th October 2024

Deployments Deployment system outage

At 09:30 UTC, we received an alert from the monitoring about deployments not going through. We checked and saw that the orchestrator failed silently in a way that wasn’t detected by the service controller. We restarted the orchestrator and the deployments started to be handled again. 09:33 UTC: end of alert.

We will find a way to detect this kind of failure and handle it before the deployments get blocked.

Wednesday 23rd October 2024

No incidents reported

Tuesday 22nd October 2024

MySQL PHP My Admin Error

PHP My Admin view in Console show error depending on your browser (issue with Chrome-based and Safari). We are investigating the issue.

Current workaround: it is know to work on Firefox

Users need to enable third-party cookies so it work properly again

Monday 21st October 2024

Pulsar Pulsar cluster is unavailable due to ZooKeeper instabilities

Following yesterday's incident (https://www.clevercloudstatus.com/incident/911), we took actions to solve the root cause issue.

When we are performing some actions the ZooKeeper cluster becomes unstable and fails. Access logs, logs and deployments stack are affected as well as all services interacting with the Pulsar cluster.

16:20 UTC : The cluster pulsar is up and running, the deployment stack, logs is running as well, we are restarting the access logs stack.

16:30 UTC : The access logs stack is up and running.

Sunday 20th October 2024

Pulsar Pulsar cluster is an unhealthy state

The monitoring report that pulsar is in a unhealthy state, we are investigating.

16:38 UTC: there seems to be an inconsistency in the underlying bookkeeper cluster. We are looking into it.

16:40 UTC: we are now looking into the zookeeper service that seems to fail.

17:30 UTC: we have fix the zookeeper issue, and we begin the recovery process of the cluster bookeeper and then pulsar.

18:10 UTC : we are rolling open the access to the pulsar cluster.

18:45 UTC : we have rolled open the access to the pulsar cluster to half of our hypervisors.

19:15 UTC : the pulsar cluster is running and available for everyone. We are running the recovery process of the platform to ensure that every applications is up and running as well.

21:30 UTC : we have finished to redeploy applications. We are investigating the access logs stack that got offloaders errors on pulsar-side.

22:10 UTC : we have finished to restart the access logs stacks.

Saturday 19th October 2024

No incidents reported

Friday 18th October 2024

No incidents reported