Some systems are experiencing issues

Past Incidents

Monday 27th May 2024

[Global][Dedicated] Application load balancers software maintenance, scheduled 6 months ago

Maintenance Window: 2024-05-27T09:00:00Z - 2024-05-29T20:00:00Z (UTC)

Scope:

  • We will roll out software updates on PAR region and dedicated load balancers

Expected Impact:

  • Brief disconnections or connection drops during the upgrade process.
  • Potential minor performance fluctuations.

Additional Information:

  • Please report any issues with a method for reproducing the problem (e.g., curl command for application load balancer issues).

EDIT 2024-05-28 13:15 UTC : We are beginning the maintenance. We are starting with cleverapps.io.

EDIT 14:10 UTC : We have updated cleverapps.io load balancers, we are updating the par region one.

EDIT 15:30 UTC : We have updated three of nine load balancers of par region, the updates are still running

EDIT 16:30 UTC : We have update six of nine load balancers of par region, the are still running for the last ones.

EDIT 17:00 UTC : We have finished to update the par load balancer, we will do the dedicated one starting tomorrow

EDIT 2024-05-28 08:00 UTC : We have seen an increase of tls error on par region, we have rollback 8 of 9 instances of this load balancer on the previous version which is not affected. We keep one instance with the issue the time to dig and found the the root cause.

EDIT 12:30 UTC : We have found the issue and written a patch, we are releasing it and then we will deploy the new version. The issue was limited to services under *.services.clever-cloud.com certificate only.

EDIT 13:30 UTC : We have deployed the new release on par region, we will start very soon with others regions and cleverapps.io

EDIT 14:50 UTC: We have deployed the new release on every region including dedicated ones and cleverapps.io. We will begin the dedicated load balancers very soon.

EDIT 17:20 UTC : We are finishing the last dedicated load balancers for today and we will terminate the others tomorrow.

Sunday 26th May 2024

No incidents reported

Saturday 25th May 2024

No incidents reported

Friday 24th May 2024

Access Logs [Global] Missing telemetry from access logs

We had an issue with the compute pipeline of telemetry from access logs which did not perform the computation since the end of the last week. We have fixed the issue, but we could not recover missing computations.

Metrics latencies and errors

We identified an increase of errors on read queries, writing isn't impacted.

Thursday 23rd May 2024

No incidents reported

Wednesday 22nd May 2024

Reverse Proxies [Global] Application load balancers software maintenance

Maintenance Window: 2024-05-22T09:00:00Z - 2024-05-24T20:00:00Z (UTC)

Scope:

  • We will roll out software updates on every region of Clever Cloud for all application load balancers

Expected Impact:

  • Brief disconnections or connection drops during the upgrade process.
  • Potential minor performance fluctuations.

Additional Information:

  • Please report any issues with a method for reproducing the problem (e.g., curl command for application load balancer issues).

EDIT 13:06 UTC : We are beginning the rolling of the WSW region.

EDIT 15:00 UTC : We have updated the WSW region. We are beginning the rolling of SGP, MTL and SYD regions.

EDIT 16:25 UTC : We have updated the MTL region.

EDIT 17:00 UTC : We have updated the SYD and SGP region. Next region tomorrow

EDIT D+1 08:10 UTC : We will begin the update of MEA and GRA-HDS.

EDIT D+1 09:20 UTC : We have finished the update of GRA-HDS, we are beginning RBX and RBX-HDS. The rolling update of MEA is still running.

EDIT D+1 11:15 UTC : We have finished the update of MEA, RBX and RBX-HDS.

EDIT D+1 15:00 UTC : We are beginning the update of SCW and dedicated regions.

EDIT D+2 13:00 UTC :We have finished the update of SCW and dedicated regions. We will perform the update of the PAR regions and dedicated application load balancers on a new status here : https://www.clevercloudstatus.com/incident/855

Tuesday 21st May 2024

No incidents reported