My spouse traveled to Canada for a few days. She just went a few miles over the border into Vancouver, BC.
She neglected to add an international data plan to her mobile number before she left. Because of this, she racked up $300 of data charges in 24 hours.
Every wireless carrier has at least one, and you have to add it to your account before you travel outside the country, and then delete it when you return home. But, why? My carrier knows when I’m out of the country! In fact, multiple systems between my cellphone and my account know it!
We had more fun with a vendor today.
We license a vendor’s services for corporate information, like annual revenue and office locations. Their name shall be kept confidential. I’ve written about them before.
About two weeks ago, we noticed a slowdown in our API calls into their system.
We asked them about it, and they replied that they would take a look. A bit later, they said they had found the problem and were working on a solution.
Today, after working on new code, I ran my unit tests. A few tests make calls to this vendor. (Yeah, I could have mocked out the calls. But there are good reasons to not mock out calls in unit tests.) I was surprised to see those tests now fail.
Curiously, they failed because the API calls returned the response, “Customer Disabled”.
I switched to a browser window and tried a part of our product that used their API. I found that our product now failed with the same error. Uh oh.
I e-mailed the vendor and asked what’s up. Their answer:
We found that our service was being slowed down by your API calls. So we disabled your API key.
I am not kidding. Continue reading after you’ve caught your breath.
At IP Street, most of our technology stack is open-source. Something happened last week that threw our components’ different design philosophies into stark relief.
We use Solr (with Zookeeper) for many of our search and pivot tasks, and Redis as a Swiss Army Knife. They do different things and have different consistency requirements. You can easily critique any juxtaposition as comparing apples to oranges. I think it’s instructive, because Solr and Redis are both high-performance, production-quality, and powerful tools.
Working on them within the same day, I experienced exact opposites in configuration philosophy!
Let’s meet contestant number 1
Solr is a powerful search engine. Their Cloud feature lets you shard and scale your index, and Solr will do the internal shard and node routing. Or you can direct your queries to the appropriate node for a small performance win. Being
short-handed understaffed frugal with our peons worker bees people, we let Solr do the routing. “Here’s a document, store it.” “I want this document.” “Here’s a pivot within a search, do it and assemble the results for me, pronto.” Etc.
Solr nodes are peers, though internally there are leaders and replicas. Solr uses Zookeeper, an Apache technology for distributed persistent configuration. Nodes do the right thing when other nodes come and go.
My dead Drobo saga’s conclusion…
- Grades: Drobo customer support: A+. DiskWarrior: F. Disk Rescue 3: A-.
- Don’t consider your Drobo to be hot-swappable. Ever.
- Buy Disk Rescue 3 and have it on hand.
- Run Disk Utility and do a Verify Disk once a month. If that’s too often for you, do it once a quarter.
I’m migrating my files and apps to my new MacBook Pro. A highly anticipated improvement was connecting my Drobo S to a USB 3.0 interface, instead of my previous laptop’s USB 2.0 bus.
During my migration, the Drobo Dashboard advised me that a Drobo firmware update was available. I did the update, which -boom- bricked my Drobo.
After trying rebooting, power-cycling the Drobo, and plugging it into the other USB socket, I’m at a point where Drobo Dashboard says the Drobo is healthy. But OS X won’t mount it. Disk Utility says:
Unable to bootstrap transaction group 6000: cksum mismatch
No valid commit checkpoint found
The volume xxxxxxx was found corrupt and needs to be repaired.
Problems were found with the partition map which may prevent booting
Error: This disk needs to be repaired. Click Repair Disk.
I then run Repair Disk, and it tells me the same thing! So Repair Disk can’t repair the disk!
I bought DiskWarrior (for $109, I’ll have you know) but it can’t repair a disk that isn’t mounted. They’ll ship me a physical CD-ROM of my purchase, so I can try booting from it. Oh, but wait, my MacBook Pro/Retina doesn’t have a CD-ROM drive!
I am not a happy camper.
We license a vendor’s services for corporate information, like annual revenue and office locations. Their name shall be kept confidential in this story.
We access their API via http calls. They call it a REST API. But like 95% of the “REST” APIs in the world, it’s not REST at all, and in fact nowhere near REST. The term “REST” has
been corrupted to be become synonymous with, “web API”.
But whatever. It’s an API accessed with http calls.
One of service calls has a parameter called, “countryCode”, which was documented as an ISO 3166 country code.
Boy, what a roller coaster! Shortly after opening a position for a Senior Devops engineer, we had a funding “event” and now the opening’s gone. What’s worse, I had to lay off one of my developers, right before before the end-of-year holidays. It was stressful for all involved.
We’re doing some interesting things with name relationships at work, and these present fun development challenges. I’m trying to spend as much time as possible in Emacs, because the less-fun work issues always occur when I’m not coding.
I upgraded our codebase to version 3 of Celery, just to get us off version 2. I’m still hankering to replace Celery, but it must have known it was living on borrowed time because it’s been behaving lately, so I’ve decided to fry some bigger fish. But the moment Celery starts acting up again…
I just turned 55. How the hell did that happen?!?