ChargeOver outage on Sep 9, 2020

What happened

ChargeOver reported a major outage starting Sep 9, 2020, 16:15 UTC, affecting Main Application, Payment Processing. It was marked resolved at Sep 9, 2020, 20:39 UTC.

• The incident lasted 4h 23m from first report to resolution.
• ChargeOver had reported no other partial or major outages in the 90 days before it.
• 1 other vendor we track reported a significant incident within 12 hours of it.

Latest update from ChargeOver

ChargeOver primarily uses the MariaDB database for data storage. The MariaDB process crashed on our primary database server at 11:09 CST. The process crashed with the following error message which is still being investigated: `InnoDB: Failing assertion: templ->clust_rec_field_no != ULINT_UNDEFINED` ChargeOver staff were immediately notified, and we began to investigate the issue. The database was automatically restarted immediately, and the database began to run automated data integrity checks to ensure that no data was lost, and no data was corrupted before beginning to service requests again. Although we had the ability to fail-over to a secondary database server, a decision was made to let the process complete, as the estimated downtime was very short. This automated data integrity check took much longer than originally expected. Our estimated time to recovery was less than an hour, and instead the database server took approximately 3 hours to do integrity checks and restart sa

Read ChargeOver's official incident report ↗

What our independent monitoring saw

Our independent monitoring of ChargeOver started after this incident, so this report is based on ChargeOver's status page.

Other outages around the same time

Significant incidents other vendors reported within 12 hours of this one.

• Partial outageStarling BusinessSep 9, 2020, 12:01 UTC

  Issues receiving calls

ChargeOver's overall reliability

Full ChargeOver reliability report and live status → · Most reliable ChargeOver alternatives →