Feng SkyQualityServiceTechnical

Downtime, slowdowns, and backups.

By 4 December, 2012No Comments

Dear Feng Sky clients,

We have been exceptionally quiet at the website this past three weeks. Let us explain why:

On the 7th of November, 2012, we had an issue with one of our core servers (in summary, it broke down). So for the first time in 5 years we had to recover installations from daily backups.

While we were happy that we could recover most information, some clients did lose hours of input data.

The issues affected 26 clients, all of whom have been contacted directly to explain the situation and to work with them on managing the incident.

It was also very comforting to receive such a positive and supportive response from so many of you. We want to extend a big thank you from everyone at Feng Office for your understanding and support. We are proud to work for you.

We work constantly on improving our systems and be prepared to respond to this incidents without users even noticing it. But this time, this was not the case.

Since this incident, Feng Office took it as top priority to further improve our backups system. Our goal is to have almost real-time backups. In the mean time, this past two weeks we worked around the clock to migrate everyone to a new backup system that is making 5 times as many backups (5 backups per day). This, in turn, generated overload in some servers.

That has also affected our response times for support tickets during the past weeks.

We are very sorry for the inconveniences. We are doing our best to prevent any of these issues in the future.

We will follow up on this subject with an update on the next backup-system improvements

Again, to all affected clients, please excuse us for the inconveniences and the uncommon delays on our response times.

Thank you,

Conrado Vina
CEO
The Feng Office Team