Evernote Users Lose Few Days of Notes, Evernote Recovers Gracefully

Evernote experienced some server issues between July 1 and July 4 that caused a few thousand Evernote users to lose notes created in that time frame. Fortunately, it sounds like most of them were able to recover their lost notes thanks to Evernote reaching out and walking them through how to upload local copies of those notes. In all, just over 6,000 users were victims of the incident. If you didn’t receive a note about the incident last month, then your notes are absolutely fine.

In a blog post today, Evernote explains why the hiccup didn’t have severe consequences:


Every user’s data is stored on a “shard”. A shard is made up of a server together with a redundant fail-over server. If there is any problem with a server, the system automatically fails over to the second server in the shard. We currently have 37 shards. Shard 22 was the one that had problems last month. The data in each server is stored on a RAID 1 (fully redundant) array. All data is also backed up on-site and off-site. A full copy of your notes are also stored on the Windows and Mac clients (and the iPhone and iPad clients for Premium users who enable that option). This means that every note in Evernote is stored in at least six redundant locations: the disk on the primary server, the RAID mirror, the fail-over server on the shard and it’s RAID mirror, the on-site backup and the offsite backup. Most users also have another one or two full copies on their local clients. This makes data loss in Evernote extremely rare. The problem with shard 22 was a very idiosyncratic intermittent combination of hardware problems with both the primary server and the fail-over mechanism. Basically, the shard kept failing over back and forth between two servers over the time period causing some of the data created during that time to get overwritten. Everything created before the failure was easy to recover from backup.

Evernote has a pretty good handle on keeping your notes safe, but it’s always important to take it upon yourself to make sure that data stored in the cloud is also backed up locally somewhere. Evernote’s a pretty stable company with lots of customers and corporate partners, so I doubt they’re going anywhere soon, but stranger things have happened. The very nature of Evernote all but ensures that I have at least a couple of spare copies of my notes at hand offline in case the company’s servers go haywire.

This is yet another example of why I’m a big fan of Evernote. I have Evernote installed and synced on three PCs, two MacBook Pros, my iPad and iPhone. Except for the relatively rare occurrences of entering a note via Evernote.com, I create a local copy of a note every time I add a note. This is not the case with all cloud services, many of which don’t have companion desktop and mobile applications.

If you’re an avid Evernote user you might want to follow the company’s EvernoteStatus on Twitter or subscribe to the company’s status page.