The Ington Group Status - Incident history https://status.ington.net The Ington Group Wed, 1 Nov 2023 16:14:00 +0000 EU Infrastructure Service Disruptions due to Upstream Provider Type: Incident Duration: 1 day, 23 hours and 7 minutes Affected Components: ingtonbusiness-eu-germany-1, ingtonhost-eu-germany-1, ingtonhost-eu-germany-1 Nov 1, 16:14:00 GMT+0 - Identified - Our upstream provider for some of our EU infrastructure is experiencing service difficulties that are impacting our infrastructure. We are working with our provider to resolve this issue as soon as possible. **During this time, all Minecraft services in the EU region have been disrupted, and some business services in the EU region have been disrupted.** Nov 2, 21:27:37 GMT+0 - Identified - We are continuing to work with our provider to attempt to resolve this incident. If you have been effected by this incident, please reach out to us via a ticket in Discord or via support@ington.net so we can implement mitigation tatics. Thank you for bearing with us. Nov 3, 15:15:14 GMT+0 - Monitoring - Our upstream provider has implemented a fix, all services should return to normal operation shortly. If you are still affected, please contact us via our Discord or via support@ington.net as soon as possible. Thank you for bearing with us through this process. We are working with our provider to prevent issues like this from being possible in the future. Nov 3, 15:20:39 GMT+0 - Resolved - At this time all services have returned to normal operation. Thank you for your continued trust and loyalty. Type: Incident

Duration: 1 day, 23 hours and 7 minutes

Affected Components: , ,

<p><small>Nov <var data-var='date'> 1</var>, <var data-var='time'>16:14:00</var> GMT+0</small><br><strong>Identified</strong> - Our upstream provider for some of our EU infrastructure is experiencing service difficulties that are impacting our infrastructure. We are working with our provider to resolve this issue as soon as possible. **During this time, all Minecraft services in the EU region have been disrupted, and some business services in the EU region have been disrupted.**.</p> <p><small>Nov <var data-var='date'> 2</var>, <var data-var='time'>21:27:37</var> GMT+0</small><br><strong>Identified</strong> - We are continuing to work with our provider to attempt to resolve this incident. If you have been effected by this incident, please reach out to us via a ticket in Discord or via support@ington.net so we can implement mitigation tatics. Thank you for bearing with us..</p> <p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>15:15:14</var> GMT+0</small><br><strong>Monitoring</strong> - Our upstream provider has implemented a fix, all services should return to normal operation shortly. If you are still affected, please contact us via our Discord or via support@ington.net as soon as possible. Thank you for bearing with us through this process. We are working with our provider to prevent issues like this from being possible in the future..</p> <p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>15:20:39</var> GMT+0</small><br><strong>Resolved</strong> - At this time all services have returned to normal operation. Thank you for your continued trust and loyalty..</p> ]]> Wed, 1 Nov 2023 16:14:00 +0000 https://status.ington.net/incident/clog78d4w41989bhojltr1nve8 https://status.ington.net/incident/clog78d4w41989bhojltr1nve8 Degraded Performance in us-east Type: Incident Duration: 1 hour and 43 minutes Affected Components: ingtonhost-us-east-1, ingtonhost-us-east-2 Oct 15, 01:15:00 GMT+0 - Resolved - This incident has been resolved. Thank you for choosing Ington! Oct 14, 23:31:46 GMT+0 - Investigating - We are currently investigating an increase in panel connectivity errors to our US east servers. We will continue to update the status page as we identify and resolve the issue. Oct 15, 00:20:49 GMT+0 - Identified - We have identified the cause of the issue and have began rolling out a fix for the issue. We will continue to update this page as we progress Oct 15, 00:55:25 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. All services should return to normal operation. Type: Incident

Duration: 1 hour and 43 minutes

Affected Components: ,

<p><small>Oct <var data-var='date'> 15</var>, <var data-var='time'>01:15:00</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved. Thank you for choosing Ington!.</p> <p><small>Oct <var data-var='date'> 14</var>, <var data-var='time'>23:31:46</var> GMT+0</small><br><strong>Investigating</strong> - We are currently investigating an increase in panel connectivity errors to our US east servers. We will continue to update the status page as we identify and resolve the issue..</p> <p><small>Oct <var data-var='date'> 15</var>, <var data-var='time'>00:20:49</var> GMT+0</small><br><strong>Identified</strong> - We have identified the cause of the issue and have began rolling out a fix for the issue. We will continue to update this page as we progress .</p> <p><small>Oct <var data-var='date'> 15</var>, <var data-var='time'>00:55:25</var> GMT+0</small><br><strong>Monitoring</strong> - We implemented a fix and are currently monitoring the result. All services should return to normal operation. .</p> ]]> Sat, 14 Oct 2023 23:31:46 +0000 https://status.ington.net/incident/clnqo9c933235bjod1wd3v9kk https://status.ington.net/incident/clnqo9c933235bjod1wd3v9kk Backup System Migration Type: Maintenance Duration: 36 minutes Affected Components: Minecraft Panel, ingtonhost-us-east-1, ingtonhost-us-east-2, ingtonhost-eu-germany-1, ingtonhost-eu-germany-1, ingtonhost-us-west-1, ingtonhost-us-west-2 Aug 6, 03:00:00 GMT+0 - Identified - We are migrating our backup system to a more reliable and available solution. ALL PREVIOUS BACKUPS WILL BE INACCESSABLE, PLEASE DOWNLOAD ANY NEEDED BACKUPS FROM YOUR SERVER. **This will impact Ington Minecraft Services**. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns. Aug 6, 03:03:11 GMT+0 - Identified - Our maintenance is now in progress, we'll keep you updated on the status page. ALL PREVIOUS BACKUPS ARE NOW INACCESSABLE. **This impacts Ington Minecraft Services**. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns. Aug 6, 03:35:50 GMT+0 - Completed - We have completed our maintenance work. If you need a historical backup pulled for your server, please contact our support team before Thursday, August 10th, 2023. After that date, we will purge historical backups from our old backup system. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns. Type: Maintenance

Duration: 36 minutes

Affected Components: , , , , , ,

<p><small>Aug <var data-var='date'> 6</var>, <var data-var='time'>03:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are migrating our backup system to a more reliable and available solution. ALL PREVIOUS BACKUPS WILL BE INACCESSABLE, PLEASE DOWNLOAD ANY NEEDED BACKUPS FROM YOUR SERVER. **This will impact Ington Minecraft Services**. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns..</p> <p><small>Aug <var data-var='date'> 6</var>, <var data-var='time'>03:03:11</var> GMT+0</small><br><strong>Identified</strong> - Our maintenance is now in progress, we'll keep you updated on the status page. ALL PREVIOUS BACKUPS ARE NOW INACCESSABLE. **This impacts Ington Minecraft Services**. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns..</p> <p><small>Aug <var data-var='date'> 6</var>, <var data-var='time'>03:35:50</var> GMT+0</small><br><strong>Completed</strong> - We have completed our maintenance work. If you need a historical backup pulled for your server, please contact our support team before Thursday, August 10th, 2023. After that date, we will purge historical backups from our old backup system. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns..</p> ]]> Sun, 6 Aug 2023 03:00:00 +0000 https://status.ington.net/maintenance/clku8qf8l309529beocb3or4qv3 https://status.ington.net/maintenance/clku8qf8l309529beocb3or4qv3 Scheduled DDOS Maintenance Type: Maintenance Duration: 48 minutes Affected Components: API, ingtonbusiness-us-west-1, ingtonbusiness-eu-germany-1, ingtonbusiness-us-east-2, Website, Minecraft Panel, ingtonhost-us-east-1, ingtonhost-us-east-2, ingtonhost-eu-germany-1, ingtonhost-eu-germany-1, ingtonhost-us-west-1, ingtonhost-us-west-2, ingtonbusiness-us-east-1 Jun 25, 02:00:00 GMT+0 - Identified - We will be updating components of our public network's DDOS protection in order to enhance performance and security. **This will impact all Ington services**. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns. Jun 25, 02:57:18 GMT+0 - Identified - Our maintenance is now in progress, and we are making network changes. During this time, parts of all Ington service offerings may be unavailable. We will continue to update this status page as we progress. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns. Jun 25, 03:45:17 GMT+0 - Completed - We have successfully completed our maintenance work. We will reach out directly to affected Business Services clients with instructions on how to fully utilise our new DDOS protection. Standard Minecraft Hosting clients will see these benefits instantly, without any configuration changes. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns. Type: Maintenance

Duration: 48 minutes

Affected Components: , , , , , , , , , , , ,

<p><small>Jun <var data-var='date'> 25</var>, <var data-var='time'>02:00:00</var> GMT+0</small><br><strong>Identified</strong> - We will be updating components of our public network's DDOS protection in order to enhance performance and security. **This will impact all Ington services**. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns..</p> <p><small>Jun <var data-var='date'> 25</var>, <var data-var='time'>02:57:18</var> GMT+0</small><br><strong>Identified</strong> - Our maintenance is now in progress, and we are making network changes. During this time, parts of all Ington service offerings may be unavailable. We will continue to update this status page as we progress. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns..</p> <p><small>Jun <var data-var='date'> 25</var>, <var data-var='time'>03:45:17</var> GMT+0</small><br><strong>Completed</strong> - We have successfully completed our maintenance work. We will reach out directly to affected Business Services clients with instructions on how to fully utilise our new DDOS protection. Standard Minecraft Hosting clients will see these benefits instantly, without any configuration changes. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns..</p> ]]> Sun, 25 Jun 2023 03:00:00 +0000 https://status.ington.net/maintenance/clja9yujc18710wnn72bgrqnv6 https://status.ington.net/maintenance/clja9yujc18710wnn72bgrqnv6 Scheduled Internal Network Maintenance Type: Maintenance Duration: 1 hour and 18 minutes Affected Components: API, ingtonbusiness-us-west-1, ingtonbusiness-eu-germany-1, Website, Minecraft Panel, ingtonhost-us-east-1, ingtonhost-us-east-2, ingtonhost-eu-germany-1, ingtonhost-eu-germany-1, ingtonhost-us-west-1, ingtonhost-us-west-2, ingtonbusiness-us-east-1 Jun 21, 04:17:46 GMT+0 - Completed - We have successfully completed our internal network maintenance, and all services have been fully restored to operation. We keep our software and services up to date to provide the best possible services to our clients. Thank you for choosing Ington, please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns. Jun 21, 03:00:01 GMT+0 - Identified - We have begin the maintenance process, and will continue to provide updates as we progress. Jun 21, 03:00:00 GMT+0 - Identified - We will be updating components of our internal network in order to improve efficiency and security. This **will not** impact any public Minecraft Server hosting. This **may** impact Business Services clients, including dedicated managed machines and managed services. If you may be impacted, you will be reached out to over the next day. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns. Type: Maintenance

Duration: 1 hour and 18 minutes

Affected Components: , , , , , , , , , , ,

<p><small>Jun <var data-var='date'> 21</var>, <var data-var='time'>04:17:46</var> GMT+0</small><br><strong>Completed</strong> - We have successfully completed our internal network maintenance, and all services have been fully restored to operation. We keep our software and services up to date to provide the best possible services to our clients. Thank you for choosing Ington, please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns..</p> <p><small>Jun <var data-var='date'> 21</var>, <var data-var='time'>03:00:01</var> GMT+0</small><br><strong>Identified</strong> - We have begin the maintenance process, and will continue to provide updates as we progress..</p> <p><small>Jun <var data-var='date'> 21</var>, <var data-var='time'>03:00:00</var> GMT+0</small><br><strong>Identified</strong> - We will be updating components of our internal network in order to improve efficiency and security. This **will not** impact any public Minecraft Server hosting. This **may** impact Business Services clients, including dedicated managed machines and managed services. If you may be impacted, you will be reached out to over the next day. We make changes and updates like these to keep our infrastructure performing the best it can. Please reach out to [support@ington.net](mailto:support@ington.net) if you have any questions or concerns..</p> ]]> Wed, 21 Jun 2023 03:00:00 +0000 https://status.ington.net/maintenance/clj0ujt8c621745zn6sl8nh1uw https://status.ington.net/maintenance/clj0ujt8c621745zn6sl8nh1uw Scheduled Minecraft Panel Maintenance Type: Maintenance Duration: 3 days, 22 hours and 3 minutes Affected Components: Minecraft Panel Jun 1, 02:30:00 GMT+0 - Identified - We will be taking down the Ington Minecraft Panel for all users for the duration of this work. During this time, Minecraft servers that were started will continue to run, but all management actions (including FTP) will be unavailable. Please reach out to support@ington.net to start/restart/stop your server during this time. Jun 1, 02:32:14 GMT+0 - Identified - Our maintenance has begun! The Ington panel is now offline. Once again, please reach out to support@ington.net if you need to start/restart/stop your server. Jun 1, 04:04:02 GMT+0 - Completed - We have successfully completed the panel maintenance at this time! You should see faster loading times in the APAC (Asia-Pacific) region, as well as a theme more in-line with the Ington brand. Thank you all for choosing Ington, if you have any issues please email support@ington.net and we'll get you sorted. Type: Maintenance

Duration: 3 days, 22 hours and 3 minutes

Affected Components:

<p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>02:30:00</var> GMT+0</small><br><strong>Identified</strong> - We will be taking down the Ington Minecraft Panel for all users for the duration of this work. During this time, Minecraft servers that were started will continue to run, but all management actions (including FTP) will be unavailable. Please reach out to support@ington.net to start/restart/stop your server during this time..</p> <p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>02:32:14</var> GMT+0</small><br><strong>Identified</strong> - Our maintenance has begun! The Ington panel is now offline. Once again, please reach out to support@ington.net if you need to start/restart/stop your server..</p> <p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>04:04:02</var> GMT+0</small><br><strong>Completed</strong> - We have successfully completed the panel maintenance at this time! You should see faster loading times in the APAC (Asia-Pacific) region, as well as a theme more in-line with the Ington brand. Thank you all for choosing Ington, if you have any issues please email support@ington.net and we'll get you sorted..</p> ]]> Thu, 1 Jun 2023 02:30:00 +0000 https://status.ington.net/maintenance/clicaryyw603987booiwhe0ccsr https://status.ington.net/maintenance/clicaryyw603987booiwhe0ccsr Panel Outage Type: Incident Duration: 1 day, 10 hours and 35 minutes Affected Components: Minecraft Panel May 16, 02:56:32 GMT+0 - Identified - We have identified the issue and are working with our upstream provider to resolve it. May 17, 13:31:19 GMT+0 - Resolved - This incident has been resolved, and all panel functionality has been restored. May 16, 14:00:21 GMT+0 - Monitoring - We have implemented a fix with the support of our upstream provider and are monitoring its effectiveness. We're seeing improvements in some regions, however not all regions are fully operational at this time. Type: Incident

Duration: 1 day, 10 hours and 35 minutes

Affected Components:

<p><small>May <var data-var='date'> 16</var>, <var data-var='time'>02:56:32</var> GMT+0</small><br><strong>Identified</strong> - We have identified the issue and are working with our upstream provider to resolve it..</p> <p><small>May <var data-var='date'> 17</var>, <var data-var='time'>13:31:19</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved, and all panel functionality has been restored. .</p> <p><small>May <var data-var='date'> 16</var>, <var data-var='time'>14:00:21</var> GMT+0</small><br><strong>Monitoring</strong> - We have implemented a fix with the support of our upstream provider and are monitoring its effectiveness. We're seeing improvements in some regions, however not all regions are fully operational at this time..</p> ]]> Tue, 16 May 2023 02:56:32 +0000 https://status.ington.net/incident/clhpon6x565868uomd8iuqcn8 https://status.ington.net/incident/clhpon6x565868uomd8iuqcn8 Major DNS Outage Type: Incident Duration: 4 days and 21 hours Affected Components: ingtonhost-us-west-2, ingtonhost-us-west-1, Minecraft Panel, ingtonhost-us-east-1, ingtonhost-us-east-2, ingtonhost-eu-germany-1, ingtonhost-eu-germany-1 Apr 20, 01:56:00 GMT+0 - Investigating - We're investigating an issue impacting Ington's DNS chain across all of our public infrastructure. Many servers are inaccessible. Apr 20, 02:29:46 GMT+0 - Identified - We have identified a fix for this issue and are deploying it actively. Connectivity with individual nodes is being restored. Apr 20, 02:54:05 GMT+0 - Monitoring - We implemented a fix and are monitoring to address any further issues. Apr 20, 03:53:00 GMT+0 - Resolved - This incident has been resolved. Type: Incident

Duration: 4 days and 21 hours

Affected Components: , , , , , ,

<p><small>Apr <var data-var='date'> 20</var>, <var data-var='time'>01:56:00</var> GMT+0</small><br><strong>Investigating</strong> - We're investigating an issue impacting Ington's DNS chain across all of our public infrastructure. Many servers are inaccessible..</p> <p><small>Apr <var data-var='date'> 20</var>, <var data-var='time'>02:29:46</var> GMT+0</small><br><strong>Identified</strong> - We have identified a fix for this issue and are deploying it actively. Connectivity with individual nodes is being restored..</p> <p><small>Apr <var data-var='date'> 20</var>, <var data-var='time'>02:54:05</var> GMT+0</small><br><strong>Monitoring</strong> - We implemented a fix and are monitoring to address any further issues..</p> <p><small>Apr <var data-var='date'> 20</var>, <var data-var='time'>03:53:00</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved..</p> ]]> Thu, 20 Apr 2023 01:56:00 +0000 https://status.ington.net/incident/clgoi6xnq336061kjohyoc7hjfr https://status.ington.net/incident/clgoi6xnq336061kjohyoc7hjfr Minecraft Servers Not Responding Type: Incident Duration: 3 days and 11 hours Affected Components: ingtonhost-us-west-1 Mar 29, 04:03:00 GMT+0 - Investigating - We are currently investigating this incident. Mar 29, 04:16:52 GMT+0 - Identified - We have identified the cause of the issue and are working to resolve it as soon as possible. Affected Minecraft servers have been removed from the affected node for data security. Mar 29, 05:26:00 GMT+0 - Resolved - This incident has been resolved. We have restored all migrated Minecraft servers back to their original nodes, and restarted them. Thank you for choosing Ington as your provider! Type: Incident

Duration: 3 days and 11 hours

Affected Components:

<p><small>Mar <var data-var='date'> 29</var>, <var data-var='time'>04:03:00</var> GMT+0</small><br><strong>Investigating</strong> - We are currently investigating this incident..</p> <p><small>Mar <var data-var='date'> 29</var>, <var data-var='time'>04:16:52</var> GMT+0</small><br><strong>Identified</strong> - We have identified the cause of the issue and are working to resolve it as soon as possible. Affected Minecraft servers have been removed from the affected node for data security..</p> <p><small>Mar <var data-var='date'> 29</var>, <var data-var='time'>05:26:00</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved. We have restored all migrated Minecraft servers back to their original nodes, and restarted them. Thank you for choosing Ington as your provider!.</p> ]]> Wed, 29 Mar 2023 04:03:00 +0000 https://status.ington.net/incident/clft68s9f15042493om0xpn6fiw https://status.ington.net/incident/clft68s9f15042493om0xpn6fiw Server Migrations Type: Maintenance Duration: 3 days, 7 hours and 11 minutes Affected Components: ingtonhost-us-west-1, ingtonhost-us-west-2 Mar 22, 22:05:00 GMT+0 - Identified - Migrations now in progress. Mar 22, 22:00:00 GMT+0 - Identified - We will be migrating servers off of our US apollo node and on to our US zephyr node. Mar 26, 05:10:43 GMT+0 - Completed - Maintenance has completed successfully. Thank you for choosing Ington! Type: Maintenance

Duration: 3 days, 7 hours and 11 minutes

Affected Components: ,

<p><small>Mar <var data-var='date'> 22</var>, <var data-var='time'>22:05:00</var> GMT+0</small><br><strong>Identified</strong> - Migrations now in progress..</p> <p><small>Mar <var data-var='date'> 22</var>, <var data-var='time'>22:00:00</var> GMT+0</small><br><strong>Identified</strong> - We will be migrating servers off of our US apollo node and on to our US zephyr node..</p> <p><small>Mar <var data-var='date'> 26</var>, <var data-var='time'>05:10:43</var> GMT+0</small><br><strong>Completed</strong> - Maintenance has completed successfully. Thank you for choosing Ington!.</p> ]]> Wed, 22 Mar 2023 22:00:00 +0000 https://status.ington.net/maintenance/clfka1wao35849i9n10pc5tmz4 https://status.ington.net/maintenance/clfka1wao35849i9n10pc5tmz4