We are pleased to announce that we will not stop and start your servers this Wednesday, December 21st, between 6:00 AM and 9:00 AM UTC (i.e. 10:00 PM PST on Tuesday December 20, 2016, and 1:00 AM PST on Wednesday December 21, 2016).
Our maintenance operation was designed to address a vulnerability in the Xen virtualization software. To respond, we decided to upgrade the Xen software and take advantage of the features of the new version, to allow you to gain performance and avoid these types of maintenance operations in the future.
However, many of you have contacted us to ask us to change our approach. We started by adjusting the schedule, but it was not enough for many of you.
As a result, we have investigated alternative solutions and we will not stop and start your servers.
We've contacted all affected customers via email. Thank you very much for your feedback on this matter.
A critical security issue in the virtualization software Xen will become public July 26 and the Xen team has already informed Gandi of the necessary patches.
Since this announcement, we have already preemptively deployed the patches required to correct the issue. We have been monitoring the particular security flaw and have determined we will need to stop/start certain Xen VMs in order to assure that no further possible attack vector will remain.
We will be contacting the affected customers directly in order to allow them to sufficiently prepare for this stop/start and those of you who have not received any message from us are therefore not affected.
In order to minimize downtime and also to help minimize the impact in general, we would advise all affected to schedule a stop/start of their platforms yourselves sometime between now and the cutoff date of July 26, 2016.
Any affected VMs that you have not yet stopped and started again by 12:00 AM PDT July 26, 2016 (07:00 UTC), we will stop/start at some point between then and July 28 at 9:00 AM PDT (16:00 UTC). Please expect around 30 minutes of downtime per stop/start.
Edit 7/21/16: Previously we used the term "reboot" instead of "stop/start." Rebooting isn't sufficient to apply the security patch. Your VM(s) need to be stopped and then started again in order for the patch to take effect.
ImageMagick announced a security vulnerability, registered as CVE-2016-3714, that allows malicious users to craft filenames to execute code remotely.
We have applied the appropriate fixes on Simple Hosting to protect customer applications using ImageMagick libraries.
If you're using ImageMagick in your application, make sure you restart your instance after 18:00 UTC (11:00 AM PDT) on May 4, 2016 in order to apply the patch.
Following this announcement we applied the necessary patches, thus reinforcing the existing security measures we had previously implemented. Over the past week, we have continued to study the vulnerability. As a preventative measure, we have decided that a reboot of KVM based VMs is required in order to ensure that all possible attack vectors have been mitigated.
We will contact affected customers directly via email to provide instructions on performing the reboot on their own. This preventive reboot will not affect customers we do not contact.
We will reboot the VMs of affected customers (who have not rebooted on their own) on November 19th. An outage of 30 minutes maximum is expected for each impacted VM.
If you have questions or encounter any problems regarding this issue, our support team is available to assist you.
A new critical security vulnerability will be publicly announced Thursday, October 29. The Xen team has already communicated fixes to Gandi. This flaw is found in the Xen virtualization software.
Following this announcement we applied the necessary patches, thus reinforcing the existing security measures we had previously implemented. Over the past week, we have continued to study the vulnerability. As a preventive measure, we have decided that a reboot of Xen-based VMs is required in order to ensure that all possible attack vectors have been mitigated.
We will contact affected customers directly via email to provide instructions on performing the reboot on their own.This preventive reboot will not affect customers we did not contact.
We strongly recommend that customers concerned by this to restart their VMs themselves, in order for them to verify that all of their services have been correctly restarted.
We will reboot the VMs of affected customers (those which were not rebooted by their owner) from Thursday, October 22 until Wednesday, October 28. An outage of 30 minutes maximum is expected for each impacted VM.
We have updated Linux kernel 3.12 (now 3.12.45) and published a new version (3.18) on our HVM platform. These new versions no longer support AUFS and might force some clients to take corrective measures for their services.
Starting today, every server that is created or rebooted on our HVM platform will automatically use version 3.12.45 of the Linux kernel, unless configured to use version 3.18 or a custom kernel.
Please note that these kernel versions do not include AUFS support. Docker users should take special notice, because AUFS has been the default storage driver for quite some time.
To continue to use Docker with this new kernel version, users must upgrade their docker client and images to use a different storage driver, such as btrfs or overlayfs (available for kernel version 3.18 only).
To use version 3.18, you can execute following Gandi CLI [4] command:
$ gandi disk update --kernel "3.18-x86_64 (hvm)"
You can also change the kernel from the web interface by following these instructions [3].
After the operation is completed, make sure you reboot your server and update your software packages and kernel modules [1].
Clients wishing to use a custom kernel can access more information on our Wiki page [2]. You can also access more information about kernel update history on our Changelog [5]
A new security vulnerability, CVE-2015-3456, was announced last week. The flaw is found in the QEMU virtualization software, and permits an attacker to gain access to a vulnerable host from a virtual machine located on that host.
Immediately following this announcement, we applied the necessary patches, thus reinforcing the existing security measures we had previously implemented. Over the past week, we have continued to study the vulnerability. As a preventative measure, we have decided that a reboot of certain VMs is required in order to ensure that all possible attack vectors have been mitigated.
This preventive reboot will only affect a small proportion of our customers. We will contact affected customers directly via email to provide instructions on performing the reboot on their own.
We will reboot the VMs of affected customers (who have not rebooted on their own) on Monday, May 25 at 11:59 p.m. PDT (that is: Tuesday, May 26, 2015 at 07:59 UTC).
For more information, see the following resources:
Earlier this month we started the process of unifying the SFTP service on Simple Hosting across our three data centers by merging their SFTP keys. Over the next few weeks, we're taking the next step towards a more unified infrastructure with migrations of the SFTP endpoints.
While most customers will not notice any disruption in service, we want to keep you informed of our operations so you can avoid any possible issues. Here's a schedule:
Datacenter
Endpoint
Date of migration
Baltimore
sftp.dc1.gpaas.net
December 30, 2014
Luxembourg
sftp.dc2.gpaas.net
January 5, 2015
Paris
sftp.dc0.gpaas.net
January 6, 2015
What are the possible issues?
Loss of connectivity
It's possible (though unlikely) that a running SFTP connection will lose connectivity during migration. This will happen very rarely, and will not have major consequences. Recovery will consist of simply reconnecting.
DNS / Firewall Issues
Since the IP address of each endpoint will change, DNS propagation problems may arise for some customers.
If you are having trouble connecting to the service immediately following the migration window, this may be the cause. Simply waiting for propagation should resolve it.
Also, if your firewalls or other security systems limit SFTP outbound to specific IP addresses, it will be necessary to adjust the rules on these systems to allow the new addresses (and disallow the old).
An incident has occurred on one of our storage units in the Parisian datacenter. Our technical team is working to resolve the issue as quickly as possible.
UPDATE - 2:57 AM (UTC) : the situation should be back to normal. Feel free to contact our support if you notice anything wrong.
We just suffered a major incident at one of our facilities. A faulty processor caused the shutdown of a storage unit.
As communications to the disk were interrupted, all operations (reboots, changes, etc) were suspended.
We restarted the unit, and all the services have begun recovering. Operations were queed and are being executed once again. No data was lost. Everything should be returning to normal.
This incident started at 16:19 CEST (07:19 Pacific time). The system was recovered at 16:57, and all queed operations were fully resolved at 17:25 CEST (08:25 Pacific time).
We do apologise for this interruption in service.
As a reminder, you can see the status of our services here:
https://www.gandi.net/servstat
You can also follow our twiiter feed from the Gandi Noc at @gandinoc.
This news feed is available at: https://www.gandi.net/news
