Analyzing JavaScript Errors and Browser Updates
Our friends at ZingChart asked to do some data crunching on some of our data. They wanted to understand if evergreen browser releases impacted error rates overall. We were happy to provide them with some aggregate metrics regarding the recent Chrome and Firefox updates.
This post was originally written by Derek Fletes and posted to the ZingChart blog. We appreciate the time they took analyzing the data.
ZingChart is constantly on the lookout for relevant real world examples that allow us to exhibit real data with our own charts. We recently had the pleasure of being able to work with TrackJS to visualize the impact browser updates have on JavaScipt errors.
What is TrackJS?
TrackJS is a really useful software that allows developers to comprehensively track JavaScript errors on their web pages or apps. It essentially pinpoints the source of your errors for you by showing you the page, browser, and operating system your error occurred on. It’s useful when it comes to fixing errors fast so that your user experience is never anything but perfect.
But, enough about our friends over at TrackJS. The background was just necessary in order to understand the nature of the data we have to present.
Data
So what data do we have exactly? TrackJS passed us over some user data after two Google Chrome and Mozilla Firefox releases. For this data set, we have:
- Google Chrome 50 leading up to and after Google Chrome 51 release
- Google Chrome 51 release and after
- Firefox 46 leading up to and after Firefox 47 release
- Firefox 47 release and after
Chrome and Firefox Key Stats
- Google Chrome usage is about 78% of Internet users as of August 2016
- Mozilla Firefox usage is about 16.8% of Internet users as of August 2016
Source: W3Schools
Google Chrome
Below we have Google Chrome (with a super lively Google color scheme).
Mozilla Firefox
Below we have Mozilla Firefox.
Both
Below is the data combined so you can compare and contrast the number of JavaScript errors for both browsers.
Analysis
What is there to say about the data we have collected and visualized? There are actually some takeaways that you could use to your advantage in upcoming projects.
Note: This is merely a point for a discussion. I realize this isn’t a huge sample size, but it definitely shows some interesting unexpected trends. The data is here for you to analyze!
Chrome
- At its peak error count, Chrome 50 had an average of 8 errors per IP
- At its peak error count, Chrome 51 had an average of 16 errors per IP
- ~Google Chrome 50 errors dramatically increased on the day of Firefox 47’s release.~
Firefox
- At its peak error count, Firefox 47 had an average of 7 errors per IP
- At its peak error count, Firefox 46 had an average of 13 errors per IP
- Firefox 46 errors dramatically increased once Google Chrome 51 was released.
In Comparison
Firefox 47 and Google Chrome 51 were both released on a Thursday. Overall errors decreased over the weekend and increased dramatically on Monday. This caused a huge number of people to revert to previous versions during the first week launches.
It takes about a month for older version browser errors to decrease.
Older browser errors dramatically increase again around three weeks after new version launch.
Developers (that use TrackJS) are pretty quick to start evening errors out. You can see a dramatic decrease in errors around three weeks into a new version launch.
Dramatically less browser errors occur during weekends. (Probably because people refuse to go near their computers and deal with real life on the weekends.)
Conclusion
As we have seen, browser errors are going to occur. Browsers seem to have interesting patterns that seem to show trends:
- Browser releases impact other competing browsers
- Previous version use jumps as new release errors jumps
- Developers seem to jump on errors within a month, so as a developer, you need to pay attention to updates!
These are just a few of the many possible observations that could have been pulled from this data. If you have any more insight be sure to let me know in the comments! If you want to check out some more data visualization check out our gallery!
Also, a quick shoutout to TrackJs for providing us with some cool data! If you want some help tracking your JavaScript errors more precisely you should check them out!
