Due to a high load, a storage unit is currently suffering from a high load, so the SimpleHosting is in degraded mode.
Errors may be encountered on some instances during this issue.
Our technical team is currently working on fixing this issue, we will analyze and do the necessary maintenance operations.
Please do not launch any operation like rebooting your instance.
UPDATE 24/07 19h51 : issue is now finished, we will identify and correct a software-related bug.
UPDATE 25/07 11h30 : a new issue has been encoutered on another storage unit, impacting part of our Simplehosting instances on Paris/France datacenter. We work on the issue, will take appropriate actions to identify surely the bug and fix the bug in a sustainable way to avoid contention once the incident is finished.
UPDATE 11h55 : the faulty storage unit has been restarted, please do not launch any operation on your PaaS instance or your IaaS instance (like restart)
UPDATE 12h50 : the situation is now stable, tech team is analyzing and identifying the whole issue to find the potential bugs.
We are currently experiencing a load problem with Gandimail and are working on a solution.
EDIT : Still on going problem, we are on degraded mode.
EDIT : services is back to normal.
EDIT jun 21 05:14 PM CEST : As a result of a problem resulting from excessive load on about 15% of our email storage systems, some customers may have had difficulties accessing their mailbox, or may have experienced delays in delivery of their emails.
On 20 June, we detected an anomaly one some storage units and our teams took steps to stabilise the problem. Unfortunately, these measures were ineffective and resulted in disruption to the webmail service between 14:00 and 16:32.
Between 16:32 and 19:00 only 15% of the storage units showed signs of this anomalous behaviour (connection difficulties or slow delivery). On 21 June, from 9:30, we again began experiencing the issue on certain storage units. We subsequently were able to identify the root cause of the problem and implement the necessary corrective measures. The situation was stabilised around 13:45. There was no loss to any data as a result of this incident.
An attacker is trying to target Gandi's Simple Hosting services with a distributed denial of service. Our teams are mitigating the attack. Service speed may be affected.
Update 19:25 CEST We filtered a web hos who was slow to respond to our requests to stop the attackers. The rest of the world can use Simple Hosting normally again.
Some Simple Hosting instances are currently unreachable. Our technical staff is currently working to identify the issue. It is not necessary to restart your instance. Further updates will be provided here shortly.
Following an incident that occurred during an update, some Simple Hosting websites stopped responding correctly (they display an error message). We have identified the problem and are correcting it at this time. It is neither necessary nor recommended that you reboot your instance.
We will post more information here when we have an update for you.
The following updates are in CET for February 19, 2013:
13:08 The origin of the problem has been found, we are verifying the solution and will apply it shortly
13:14 The operation is still in progress. 25% of the platform is impacted.
13:44 The script did not work. We are correcting it and testing it on several instances before launching it for all the others. We can confirm that no data has been lost.
14:25 The script works. We are applying it to all of the instances affected. This will take about one hour before all of them are fixed.
15:53 The update is taking longer than expected. Estimated time to resolution is 16:50-17:00.
Incicent resolved. There are the technical details:
The affected instances are all now restarted. Any residual issues we will handle on a case-by-case basis. The deployment of a migration script failed, and all Simple Hosting instances were affected. A configuration change that should have been applied on the next restart was instead applied directly to the Apache service, and the logs were rotated. In parallel, an automatic recovery was executed on the instances in the middle of a migration. The end result was that the instances were started with a partial update applied. Consequently, for us to correct this problem, we had to stop the majority of the instances, and determine which were in an inconsistent state. We then restarted the instances, and forced the migration of the incompletely updated systems. This took longer than expected, which is why our initial recovery estimates were inaccurate. No data was lost during this incident, and your instance should be fully functional.
Please kindly accept our apologies for this incident. This week, we will be discussing how to ensure that this never happens again in this way.
