Past Incidents

Monday 13th January 2025

Cellar Paris: Service upgrade might increase requests latencies, scheduled 4 days ago

We are currently in the process of upgrading our Cellar Paris service. During this upgrade, customers might experience increased requests latencies when we restart certain components. No service interruption is to be expected. The upgrade should take place throughout the week starting on Monday 13/01/2025 around 09:00 UTC . We will update this maintenance once it is over or if anything comes up.

EDIT 2025-01-15 17:05 UTC: The maintenance is now over. No service interruption occurred.

Saturday 11th January 2025

No incidents reported

Friday 10th January 2025

Console Viewing a MateriaKV add-on can disconnect the user

16:50 UTC: We identified an issue where the user can be disconnected if the user try to see add-on informations of a MateriaKV. We are working on a fix.

EDIT 17:05 UTC: A fixed is deployed, if any disconnection happens, please inform the support.

Reverse Proxies [SGP] load balancer connectivity issues with configuration event bus

Load balancers are experiencing connectivity issues with the event bus when retrieving configuration; we are investigating.

EDIT 10:00 UTC : it seems that the issue is due to packets losse between SGP region and PAR region, we are investigating.

EDIT 11:00 UTC : the incident is related to an AS which is not in our network, we have contacted our network providers to mitigate the issue.

RabbitMQ shared cluster [Paris] Shared RabbitMQ service is experiencing degraded performances

The shared RabbitMQ cluster on the Paris region is experiencing intermittent degraded performances. We are investigating.

EDIT 10:27 UTC: The underlying issue has been found, a RabbitMQ client was reconnecting too fast / too often leading to a global increase of load on the cluster. The situation has been addressed. We continue to monitor the situation.

EDIT 2025-01-13 09:45 UTC: The situation is back to normal since 10/01/2025 10:30 UTC. The incident is over.

Thursday 9th January 2025

No incidents reported

Wednesday 8th January 2025

Cellar North: Service upgrade might increase requests latencies, scheduled 1 week ago

We are currently in the process of upgrading our Cellar North service. During this upgrade, customers might experience increased requests latencies when we restart certain components. No service interruption is to be expected. The upgrade should take place between Wednesday 2025-01-08 15:00:00 UTC and Friday 2025-01-10. We will update this maintenance once it is over or if anything comes up.

EDIT 2025-01-09 16:40 UTC: The upgrade is now over. Some queries were slower than usual today around 12:00 UTC but no requests were lost.

Tuesday 7th January 2025

No incidents reported

Monday 6th January 2025

No incidents reported

Sunday 5th January 2025

No incidents reported