Chris BoardJul 2, 2022 • 7 min read
I recently identified some issues with Crash Catch and want to let you know how it was identified and what I’m doing to ensure they don’t happen again.
As you may be aware, Crash Catch, is still relatively new and is currently running in a beta, therefore, issues are to be expected, however, the issues that I recently discovered should not have been there regardless of being in beta and I am embarrassed that these issues went unnoticed.
Over the last month, I’ve had a few signups of Crash Catch, each most signups have registered, created a project and not gone any further, with one or two registering and then not even getting as far as a creating a project.
Before we get into the issues, let me breakdown some of the monitoring and error handling that is done by Crash Catch.
First of all there are multiple components as listed below:
Within the engine and the API should any error occur then an alarm is raised and I then receive an email so I am aware of any fault that happens in real time so I can investigate.
When I noticed people signing up and not going any further than creating a project, I never received dany alarms, and looked through the logs and couldn’t see any errors in the log file (I thought maybe something was failing but had forgotten to an alarm submission).
I then suspected there was something wrong with the registration process so I tried signing up myself and creating a project and everything worked as expected.
I emailed each of the signups but never got a reply with any feedback or details of any issues.
A month later another few signups again, the same thing where they sign up and then no further progress made.
I decided to retest the registering again, this time I got an error. When you login, the app saves a global state and this is updated as part of the registration when the registration response is returned from the API.
The second issue was around Flutter projects. The majority of people that signed up created a Flutter project and then no further action, there was no crash submitted.
I looked through the logs for the project id, and found that the new user had registered the project, and then successfully added the library to a flutter project and sent an initialisation request which was successful but no further action taken on that project.
I decided to retest registering a new flutter project and seeing what happens. I created a project that initialised and sent a crash straight away (similar to the example in the docs) and found that although I successfully initialised the project with the library, no crash was sent to the Crash Catch backend.
I found that this was an odd timing issue. On certain devices, it appeared to work and on others it failed. This was due to a timing issue where the crash being sent was attempted before another part of the library completed a task, which meant the crash was never attempted to be sent.
This then identified another issue. If the crash was able to be submitted, there was then an issue on the engine side, although this took some time to identify as I initially dismissed the crash processing being at fault as no alarm was raised.
This was unfortunately a decision that I made during the development. I had a concern that, maybe not relevant, where if invalid data was sent to the engine API, I didn’t want to raise an alarm, as I could potentially receive a lot of alarms if someone, or something is trying to send an invalid request. This meant that I didn’t get alerted to there being a problem with the library submitting data.
The issues identified above have already been fixed, so you will now be able to register and send flutter crashes (make sure you are using the latest version tagged on GitHub).
If you have been following me on Twitter, you may be aware that I am in the process of redesigning Crash Catch. The web interface hasn’t changed all that much since the initial release, and I using it myself, have found certain aspects to be a little clunky and overly complicated in places, so I am currently going through a redesign.
As part of this project I am going to further test and make some improvements to the backend, these are detailed below:
I want to apologise again that these issues were present, and unnoticed for so long. I strive to make software, web apps etc, that are reliable and trustworthy and unfortunately this has not been the case.
I have contacted everyone who signed up in the last month to let them know these issues have been identified and resolved, but if you signed up earlier on and were also affected by these issues, then please send me an email at [email protected] with the email you used to signup and I can reactivate your trial.
As for the redesign, I am in the final stages of testing and hope to have a release in the not too distant future, but you can follow me on Twitter as I’m providing updates there.