Some systems are experiencing issues

Core

Infrastructure

Operational

API

Operational

Git repositories

Operational

Deployments

Operational

Reverse Proxies

Operational

Logs

Drains

Partial Outage

Services Logs

Operational

Access Logs

Operational

Products

Cellar

Operational

SSH Gateway

Operational

VPN

Operational

MySQL shared cluster

Operational

MongoDB shared cluster

Performance Issues

Let's Encrypt certificate generation

Operational

Unique IP for outgoing traffic

Operational

RabbitMQ shared cluster

Operational

FS Buckets

Operational

PostgreSQL shared cluster

Operational

Console

Operational

Elasticsearch

Operational

Customer support

Operational

Redis

Operational

Matomo add-on

Operational

Pulsar

Partial Outage

PostgreSQL

Operational

MySQL

Operational

Metrics

Operational

Heptapod Cloud

Operational

Other Components
cleverapps.io domains
Operational
Mails
Operational

Scheduled Maintenance

[PAR] Security maintenance on 4 hypervisors

For security reasons, we will update the kernel of 4 Hypervisors in the Paris (PAR) region, more precisely in the PAR6 datacenter. Services (in particular databases) hosted on those hypervisors will be impacted : they will be unavailable between 5 and 10 minutes. Impacted hypervisors are:

hv-par6-008 hv-par6-011 hv-par6-012 hv-par6-020

Affected clients are directly and individually contacted by email with the list of impacted services, and options to avoid any impact. The maintenance will be planned in 2 operations of 2 hypervisors each, during the week of 18 to 22 Novembre 2024 between 22:00 and 24:00 UTC+1.

Past Incidents

Friday 26th July 2024

No incidents reported

Thursday 25th July 2024

No incidents reported

Wednesday 24th July 2024

No incidents reported

Tuesday 23rd July 2024

Deployments Deployment outage

Application deployement take unbound time to proceed.
We are investigating the issue

09:30 UTC We notice deployment perturbation. 10:02 UTC We have found a cause of the perturbation and fixed it.

14:30 UTC We notice other issues with the deployment system. 16:30 UTC After further investigations, we found the cause of the perturbations. We applied a temporary fix. The deployments are back on tracks!

We are working on a stronger fix for the deployments.

Metrics Metrics latencies and errors

We identified a bottleneck on our FoundationDB cluster for warp10-c2. Writing is impacted and might occur a lag in read metrics. We enabled sampling on data.

6:32 UTC: We identified unusual usage that was harming the system

6:35 UTC: Unusual usage stopped, the storage layer is starting to recover

6:58 UTC: Storage layer fully recovered, we still investigate & watch over the system

7:45 UTC: System is back to normal

Monday 22nd July 2024

No incidents reported

Sunday 21st July 2024

No incidents reported

Saturday 20th July 2024

No incidents reported