“I wouldn’t give a fig for simplicity on this side of complexity,
but I would give my right arm for the simplicity on the far side of complexity.”
― Oliver Wendell Holmes
Yesterday, we got stuck on a very interesting bug related to Elasticseach. On my local instance, I would delete my index to rebuild the datastore from scratch, but somehow it would instantly come back. Pascal looked at it for a while, then realized that I was running a newer version of Elasticsearch (0.9.6), which had a brand new cluster-based index recovery features.
What happened was that my Elasticsearch server was automatically discovering another Elasticsearch server running on our network (the one installed for participants in the bug bash) and had the auto-healing feature turned on. As a result, whenever I would delete my index, a new one would be fetched on the fly from the other server.
It’s a really cool feature when you’re in production, but during development, it’s not exactly what you want, so we turned it off, and we carried on fixing other bugs. And we were left wondering what would have happened if we did not have someone like Pascal on our team, because it’s definitely not the kind of bug that I could fix on my own…