The Top Four Exception Tracking Services

Written by: Matthew Setter
12 min read

Software development is both challenging and complex. Specifically, whether we're developing landing page apps or Linux kernel drivers, developing software often requires recording and analysis of vast swathes of information.

Of that information, application exceptions are essential. Well, if you want to find out why a user encountered a problem, why an error occurred, and why an application crashed.

Take the example below. The exception's message shows that a PDOException was thrown, because a database table, tblmovies, wasn't found. You can also see the original point of execution, right through to the class and line that threw the exception (edited for readability).

PDOException thrown with message "SQLSTATE[HY000]: General error: 1 no such table: tblmovies".
Stacktrace:
#28 PDOException in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/vendor/zendframework/zend-db/src/Adapter/Driver/Pdo/Statement.php:186
#27 PDO:prepare in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/vendor/zendframework/zend-db/src/Adapter/Driver/Pdo/Statement.php:186
#26 Zend\Db\Adapter\Driver\Pdo\Statement:prepare in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/vendor/zendframework/zend-db/src/Adapter/Driver/Pdo/Statement.php:212
#25 Zend\Db\Adapter\Driver\Pdo\Statement:execute in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/vendor/zendframework/zend-db/src/TableGateway/AbstractTableGateway.php:239
#24 Zend\Db\TableGateway\AbstractTableGateway:executeSelect in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/vendor/zendframework/zend-db/src/TableGateway/AbstractTableGateway.php:209
#23 Zend\Db\TableGateway\AbstractTableGateway:selectWith in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/src/Movies/src/Services/Database/MovieTable.php:48
...
#5 Zend\Stratigility\Middleware\ErrorHandler:process in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/vendor/zendframework/zend-expressive/src/Middleware/LazyLoadingMiddleware.php:60
#4 Zend\Expressive\Middleware\LazyLoadingMiddleware:process in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/vendor/zendframework/zend-stratigility/src/Next.php:128
#3 Zend\Stratigility\Next:process in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/vendor/zendframework/zend-stratigility/src/MiddlewarePipe.php:102
#2 Zend\Stratigility\MiddlewarePipe:process in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/vendor/zendframework/zend-expressive/src/Application.php:374
#1 Zend\Expressive\Application:run in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/public/index.php:29
#0 {closure} in /Users/settermjd/Documents/Business/Projects/Zend Expressive Course/book-code/iteration-five/public/index.php:30

Do your applications record this information -- ideally logging it to a central store? If they are:

  • Is the information readily accessible?

  • Can it be efficiently parsed?

  • Can you trace the problem back to an individual commit (and commit author)?

  • Does it provide additional metadata and heuristics that help make it more meaningful?

  • Does it contain personally identifying information (PII), credentials, or any other form of information that should be kept secure?

If it doesn't satisfy all -- or at least most -- of these questions, then you'll want to read this post and learn about four online services that do. We'll start off, in no particular order, with Raygun.

Raygun

[caption id="attachment_5964" align="aligncenter" width="2272"][

]1

Raygun[/caption]

Trusted by the likes of Coca-Cola, Microsoft, Slack, and 3M, Raygun provides a modern and professional UI for tracking application exceptions. Raygun is easy to get started with, thanks to the detailed and well-thought-out instructions available.

In addition to the setup documentation, Raygun provides extensive documentation for all aspects of their service. It includes:

  • Setting up audit logs.

  • Managing teams and users.

  • Configuring integrations (which include Bitbucket, PivotalTracker, Visual Studio Team Services, Zendesk, Slack, YouTrack, and Trello).

  • APIs and webhooks.

Raygun supports a range of languages, platforms, and frameworks. These include PHP, .Net, Android, iOS, macOS, WordPress, Java, Python, Cold Fusion, and C++.

Raygun's offering is broken down into two primary sections: crash reporting and real user monitoring. Crash reporting aims to help find, understand, and fix the exceptions which cause applications to crash. For every exception that is sent to Raygun, it is broken down and organized so that it shows, among other things:

  • When it occurred

  • The page that it happened on

  • The exception's message

  • The country of origin

The exceptions can be filtered based on a range of criteria, including browser, operating system, host, and machine name. Real user monitoring, on the other hand, provides details about how the application is performing, including metrics, (where applicable) such as:

  • Average load time

  • Page load time

  • The number of visitors and views

This feature helps users keep track of how each deployment affected the application, whether positively or negatively. The user dashboard can be fully customised to suit the needs of your project and team and can show details such as:

  • Load time

  • Global session distribution

  • Live user

  • Session count

  • Total users

  • New users

  • Returning users

  • Average load time

Another convenient feature is Pulse, which shows live users of the site, where events are displayed in near-realtime. Within that view, you can see details such as:

  • Load time

  • Number of sessions

  • Number of users

  • A 24-hour session trend

  • The most recent requests

  • Median load times

  • Most requested pages

  • Recent crashes

If you drill down to the page level, you can get comprehensive information on why a page is performing poorly, including DNS lookup, latency, SSL overhead, page generation, and transfer time.

Pricing

Raygun's pricing is broken down based on what your subscription includes. You can access either the entire platform or crash reporting or real user monitoring. At first blush, this seems like a very flexible pricing structure.

Plans start as low as $19 per month and range as high as $1,499 per month. As you'd expect, the features increase as the plan price increases. Check out the pricing page and see what combination best suits your needs. And don't forget, you can sign up for a 14-day free trial.

!Sign up for a free Codeship Account

Airbrake

[caption id="attachment_5965" align="aligncenter" width="2272"][

]4

Airbrake[/caption]

Airbrake is used by companies such as SendGrid, Groupon, SoundCloud, Salesforce, and TED, and complies with the following standards: HIPAA, PCI & SOX.

Airbrake supports a large number of languages and platforms, including Java, JavaScript, PHP, Go, Node, Swift, Magento, Django, .Net, and Ruby on Rails. It has integrations with a range of standard services, including Bitbucket, GitHub, PivotalTracker, FlowDock, Campfire, Jira, HipChat, and assembla.

In contrast to Raygun, Airbrake doesn't appear as feature-rich, nor is its UI as fleshed out. For example, whereas Raygun supports logins via Facebook, Google+, Twigger, and GitHub, Airbrake only supports GitHub.

What's more, Airbrake's documentation isn't as extensive. That said, however, the service itself is still excellent, and it provides detailed setup instructions to get you started, regardless of your language or platform.

Looking more closely, Airbrake replaces an application's default error handler with an open source error and exception notifier. It catches the standard types of errors, including 404s, and 500s, login problems, and other application-specific errors.

The notifier then provides comprehensive details about application exceptions to Airbrake, which in turn parses and organizes that data to provide greater visibility, and details of the exceptions that occurred. The dashboard data includes such information as:

  • The number of times an exception has occurred

  • The environment in place at the time the error occurred

  • A full backtrace

  • Parameters in place at that moment

  • How many times an exception occurred since the last deploy

  • The exceptions that a deploy fixed

Exceptions can be filtered by an environment, prioritized by error count, environment, and users. Airbrake makes it easy to turn exceptions into tickets so that they can be tracked through to resolution, and deployed to various environments.

Pricing

Airbrake's pricing structure is flatter and a lot simpler than Raygun's. It supports four options. These start as low as $49 a month and go up to $249 a month. I like this flatter pricing structure can be a good thing, as you don't have to pick from a wide array of options.

If you're keen to have a look at Airbrake, check out the pricing page for further pricing and feature details. And don't forget to take advantage of the 30-day free plan.

Sentry.io

[caption id="attachment_5966" align="aligncenter" width="2272"][

]9

Sentry[/caption]

Sentry is one of the most impressive looking of all four options and is used by such notable companies as Dropbox, Airbnb, Paypal, and Microsoft. Written in Python, it supports an API that can handle events from any language and application.

Sentry supports a wide array of languages and platforms, including JavaScript, Ember, Python, Ruby on Rails, PHP, iOS, Android, Go, .Net, Java, Objective-C/Swift, and Perl. It integrates with most of the commonly used developer services including Slack, GitHub, Trello, Heroku, GitLab, Twilio, PagerDuty, Datadog, and YouTrack.

Like most of the services in this list, it has extensive, searchable, documentation, organized in a very logical manner. In addition to that, Sentry supports a community forum and viewing and creating issues on their GitHub repository.

I find that the Sentry dashboard is very well laid out, very clean, and very well organized. You can quickly get an overview of all your projects, drill down into one specific project to find out more about it, move between the issues within a project, such as unresolved and needs triaging.

A key feature set which appealed to me is the security and privacy settings. There, you can enable enhanced privacy and data scrubbing, which ensures that personally identifiable information and other types of sensitive information are scrubbed from the logs, so that it can never, accidentally leak out.

Looking at the core functionality, exceptions that are sent to Sentry can be interrogated based on a range of criteria including:

  • The environment

  • Browser

  • Operating system

  • The route that triggered the exception

  • When the exception occurred

  • The software release

  • Exception severity

  • Server name

  • When it was last seen

  • Number of active users affected by the exception

  • Error type (HTTP error, 500, and 404)

  • The number of times the exception has occurred

  • Whether it's resolved or not

Sentry also supports a concept called breadcrumbs. As the name implies, it's:

A trail of events which happened prior to an issue.

Breadcrumbs contain the following features:

  • Message: A string describing the event. The most common vector often used as a drop-in for a traditional log message.

  • Data: A mapping (str => str) of metadata around the event. This is often used instead of message, but may also be used in addition.

  • Category: A category to label the event under. This generally is similar to a logger name, and will let you more easily understand the area an event took place, such as auth.

  • Level: The level may be any of error, warning, info, or debug.

From all this, you can see that it's one of the more comprehensive of the offerings in today's list. One other notable plus is that it supports both an open-source and fully-hosted offering. Given that, you can pay nothing and host it yourself, or you can outsource it to Sentry. The choice is yours.

Pricing

Of all the services, I like Sentry's pricing structure the most, likely because they support a hobbyist account, which is forever free. It's handy if you're just experimenting with the service, or dabbling with a project which may or may not go anywhere.

They support a range of plans, starting with the Small plan at $26 a month, right up to large, at $449 a month. If your needs exceed that, like the others, they also support an Enterprise plan.

An extra nicety of Sentry is that each plans limits are not hard limits, as additional events can be sent for $0.00024 per event on monthly subscriptions or $0.00019 with yearly plans. For more pricing and feature information, check out https://sentry.io/pricing/ and be sure to take advantage of the hobbyist plan if you're keen to trial the service.

Rollbar

[caption id="attachment_5967" align="aligncenter" width="2272"][

]11

Rollbar[/caption]

Rollbar is used by companies such as Kayak, Heroku, Salesforce, and Instacart. It supports languages such as Ruby, PHP, Node, Python, and Java, and platforms such as iOS, and Android. In addition to these, it integrates with most of the standard services, such as Slack, HipChat, Logstash, Mailgun, Jira, Trello, Sprintly, Help Scout, and PagerDuty.

Rollbar complies with a range of industry standards, including HIPAA, ISO 27001, and Privacy Shield:. It is protected by 2048-bit SSL encryption, Single Sign-On, and 2FA.

Like all of the services, there is a setup wizard to help you get started, and documentation is extensive. In addition to this documentation, support is available via email, IRC, phone, and live chat.

Rollbar supports several features to help ensure information privacy. These include encrypting data at rest, PII, PCI data filtering and removal, auditable logs of user activity, and both internal and independent PEN testing.

What's particularly handy about Rollbar is that you can trace problems often back to the commit which introduced them, as it has full integration with GitHub, Bitbucket, and GitLab. This information can be found via the dashboard, via the API, or by using RQL (Rollbar Query Language).

To quote the documentation:

Rollbar Query Language (RQL) provides a rich SQL-like interface to the data in Rollbar. RQL supports running SELECT queries on two logical tables, item_occurrence, and deploy. Basic GROUP BY, ORDER BY, LIMIT, and aggregation functions are available, as are arbitrary expressions in the WHERE clause.

For example:

SELECT request.user_ip, count(*), count(server.host), avg(body.message.seconds), max(timestamp)
FROM item_occurrence
WHERE item.counter = 840
  AND timestamp > unix_timestamp() - 60 * 60 * 24
GROUP BY request.user_ip
ORDER BY count(*) DESC
LIMIT 100

The dashboard, while simple, shows a copious amount of information. You can see, by default:

  • The top 10 issues in the last 24 hours

  • The top 5 active items in the last 24 hours

  • The top 5 active items in the last week

You can filter the report by the environment. You can look at a list of all items, viewing how many times they occurred, a 24 hour trend, the last time it occurred, which environment it occurred in, and the language.

Drilling down into an item shows you the entire history of that exception. You can see the browser, operating system, request method, request URL, server hot, and the exception message.

The exception can also be linked to a version control system issue, allowing for it to be resolved and pushed through to deployment, with a full history. The information is comprehensive, yet not presented in an overwhelming way.

Pricing

Like the other services, Rollbar offers a range of plans in quite a flat structure. They have a free for personal project. And the paid plans start at $49 per month and go up to $599 per month for the premium plan.

In addition to these, you can get a custom arrangement if your needs aren't matched by one of the default options, or you have special security and compliance needs.

In Conclusion

That's been a look at four of the market's best online exception tracking services. Speaking personally, Rollbar and Sentry are my favorites, because of the depth of functionality and the professional appeal of their dashboards. In a nutshell, they're so comprehensive! However, that level of information may also be overkill for your situation.

Regardless, if you're keen to begin tracking exceptions that you're application's encountering, or to migrate away from a homegrown solution and outsource it to a dedicated, professional, service, I strongly encourage you to consider one of the four we've covered today. I'd suggest that at least one of them will suit your application's needs, allowing you to devote more resources to your application.

If you're already using one of them, share your feedback in the comments. If you're using a service not listed here, let us know what it is and what your experience has been like. The more we know, the better off we'll all be.

Stay up to date

We'll never share your email address and you can opt out at any time, we promise.