A fictional story about software development.
Sarah pulled up her LogCorp dashboard to start debugging. Customers were having sporadic issues checking out from the Acme site, but no one knew why. “This logging tool is so complex” she thought, “I have to re-learn this stupid query language every time.”
After several minutes of hacking, she worked out a query to see the errors. There were indeed checkout errors, but they only read "Checkout Failed". She needed more context for that error. She needed to know who the user was and what was going on before the failure.
She pulled up the source code. “We’ll just add some context here”, she remarked as she added logging statements in a few places. “More logging never hurt anyone”. A quick review and it the new logging was deployed.
Sarah spun in her chair a few times while waiting for the new logs to roll in. After a few minutes she re-ran the arcane search query and found… an error.
Not a checkout error, a logging error! The new system logs had pushed them over their event quota on LogCorp. New log events had stopped being captured and queries were locked. They couldn’t get new data, and their existing logs were held hostage, along with the answers to the current checkout outage.
LogCorp Dashboard with Licensing Error
“Oh crap!” exclaimed Sarah, “You couldn’t have picked a worse time LogCorp.” She began fumbling through her drawers, she knew here was a LogCorp Account Rep business card in there somewhere. Finding it, she hastily dialed the 800 number.
A cheerful voice answers, “Hello, this is Simone from LogCorp.”
“Hi, yes, this is Sarah from Acme. We’re having an issue right now with our systems, but we’re over our quota and locked out of the logs. Can you help us with that?”
Simone recalled her sales training. This is what LogCorp called a “Sales Event”. The customer was in a desperate situation, and they needed to get access. It was the best time to push for an upgrade.
“Oh I’m sorry to hear that! I can absolutely unlock that for you, we just need to get your contract updated to the next plan.”
“Um, what?” stuttered Sarah.
“You’ll need to upgrade or enable metered billing for the extra event data to unlock the account.”
“Right now?! We’re in a bit of a panic over here.”
“Let’s get it handled quick then. Do you have your credit card handy?”
Frustrated with the situation and the unexpected bill, Sarah re-ran the LogCorp query. She could see her logs again, lots of them. “It looks like there is a rejected promise during checkout”, she said to herself as she scrolled through the results. “Something about this call to an analytics service”.
Checking the source code again, she saw a call to an analytics service that Marketing must have added. It was failing silently and blocking the checking. She deleted the line and pushed out a new change with the temporary fix.
Watching the tests run, she considered that maybe it’s time to look for a new job. Somewhere that didn’t use LogCorp.
Afterword
I’ve heard and experienced a variant of this story many times. Legacy logging services often price themselves based on the number of events. The more you log, the more errors you track, and the more analytics you gather, the more you have to pay. It fundamentally punishes developers for adding proper instrumentation to their systems.
At TrackJS, we want to be on your team. We’ll track all the errors you throw at us, without blocking them or locking you out. When you have a bad day and run into lots of errors, we have your back. And it’s the same price.
Our business grows when you do. We price our services based on how busy your site is, not on the number of errors you have. When you do well and get more users, we’re ready to grow with you. We don’t cut you off. We don’t hit you with surprise bills. We don’t meter your usage. We’re on your side. Let us help you.
Read more »
