We have a temporary emergency halt on the hosting storage system (filers). We recommend that you do NOT attempt to restart your server. The impacted servers should recover in the next few minutes. We will update you with further information as soon as possible.
[edit00:00]The services arefullyrestoredas of 21:20 CET. Most users were back to nominal function before 19:30,butsometook longerto start.Identifiable blocked systemswere managed and restarted manually.Pleaserestartyour servicesif theyare stillunavailable at this time,and contact supportif your serveris not available and cannot be restarted.
A storage unit is currently experiencing a slowdown. Our teams are currently working on a solution.
Update (09:45 GMT): The situation improved between 07:00 and 08:00 GMT. There were significant slowdowns between 05:00 and 06:50 GMT.
Update (January 25th 09:00 GMT): A storage equipment is currently experiencing slowdown. The incident is similar to the one yesterday. Our technical team is working on solving the issue.
Update (January 25th 10:00 GMT): The I/O situation improved. Our technical team is still working to find a complete fix to the issue.
Update (January 25th 10:22 GMT): A storage equipment is currently experiencing slowdown. The incident is similar to the one this morning. Our technical team is working on solving the issue.
Update (January 26th 11:26 GMT): The I/O situation improved. Our technical team is still working to find a complete fix to the issue.
Update (January 27th 19:11 GMT): A storage equipment is currently experiencing slowdown. The incident is similar to the incident of the week. Our technical team is working on solving the issue.
Update (January 27th 22:00 GMT): The I/O situation is now stabilized. Our technical team is still working to find a complete fix to the issue.
Update (February 2nd 03h30 GMT): Another incident affects one of our storage units. We're now rebooting the faulty equipment. We recently found a few corrective actions that we'll soon be able to take in order to solve this kind of issues.
Update (February 2nd 20:19 GMT): Another incident has occurred, and slowdown was noticed, however the situation is stable right now.
Update (February 6th 02:09 GMT): Slowdown on one of our storage units. Teams working on it.
Two storage units are concerned by these incidents, which are isolated slowdowns in read/write operations. We suspect that the problem is two-fold: a software problem (blocking of operations), and a hardware problem (some disk models are unusually slow).
When these slowdowns occur, the implementation of iSCSI that lets us connect your servers to their disks may be dysfunctional. The result is an "I/O wait" that is artificially high (100%) even if the storage is once again rapid.
We are currently working on these three problems by giving priority to the capacity of our system to re-establish service after a slowdown.
A storage unit is currently suffering from a slowdown
Our technical team is working on this issue which is due to a problem with the filer's software. Writing operations are currently slow on that unit. We will let you know as soon as possible when service has returned to normal. Please accept our apologize for any inconvenience this may have caused.
Thank you.
Update11:13: The problemis located andwe have a solutionto solve itif ithappened again.Performance isagainnormalon this equipment.We still have nota precise analysisof whatcausesthese slowdowns, andwill workto reproducethe incidentin our labin order tocorrect the problempermanently.
One of our mail filers has experienced a failure of its network interface card, impacting 800 mailboxes. Our engineers are on site to replace the defective card, and the service should be operational in the next couple of hours.
One of our storage equipment is experiencing slow input/ouput access (I/O).
All the virtual servers with a disk on this filer are impacted. Our technical staff is working on identifying the problem and to fix it as soon as possible.
As the incident is on a filer, we recommend that you do not attempt to reboot or restart your server.
A maintenance operation will be undertaken on the hosting service today (Thursday 8 December 2011) between 13:00 and 14:00 UTC. During this time, operations on the hosting platform (stop, start, restart, create, etc.) will be suspended. This maintenance activity will not impact already running servers.
One of our older generation storage systems encountered a fault. Our teams are on site investigating. We recommend that customers refrain from issuing start/stop/restart operations if their servers are unresponsive during this incident. The customer servers will regain access to their disks after the fault has been resolved.
One of our transit suppliers has scheduled emergency maintenance during the night of 24 to 25 November, starting at 23:00 GMT and ending at 02:00 GMT. During this maintnenance, the Gandi network may experience some instability for up to an hour.
We apologise for any inconvenience that may be caused.
On Monday, November 7th at 10:00 GMT, we launched an unscheduled maintenance operation on the RoundCube webmail platform, which rendered it unavailable for a about an hour. If you use a different webmail or download your GandiMail by POP, you will be unaffected.
Please accept our apologies for any inconvenience this may have caused.