Replication Refactoring

Replication Refactoring:
The following represent a stream of thoughts while I was refactoring the replication behavior for RavenDB. It may not be very meaningful, since I am mostly using this to sort things out in my head.
Previously in RavenDB, we created what are known as tombstone documents in RavenDB to represent deleted documents. That caused problem under some specialized conditions (if you had a lot of deletes, you had a lot of tombstone documents to go through). So we changed the way we store tombstone documents to use a separate list that isn’t part of the document storage.
That means that now we need to update the part that accepts a replication request and have it understand what is going on. This is a bit complex because, ignoring concurrency, we have the following states:

• Item does not exist in the current database.
  
  
  • Item was deleted (mark it as deleted? leave it alone?)
  
  
  • Item was created/updated (persist it)
  
  
  
  


• Item was deleted locally.
  
  
  • Item was deleted (should we generate a conflict if the history is not similar?)
  
  
  • Item was created/updated (should generate a conflict if the history isn’t consistent).
  
  
  
  


• Item exists locally:
  
  
  • Item was deleted (need to check for history for conflicts).
  
  
  • Item was created/updated (need to check for history for conflicts).
  
  
  
  


• Item was modified locally.
  
  
  • Item was deleted (need to check for history for conflicts).
  
  
  • Item was created/updated (need to check for history for conflicts).
  
  
  
  


• Item is conflicted:
  
  
  • Item was deleted (generate conflict).
  
  
  • Item was created/updated (generate conflict).
  
  
  
  

We also need to make sure that we have the right behavior when you load a conflicted document, which make for interesting behavior when you have conflicts with deleted documents.
The first thing to do was to actually simply things, instead of having all of those options, I decided to branch early, so we would have one branch for when we replicate a delete and one branch when we replicate a create/update.

• Replicating delete




• Item does not exists – ignore


• Item is conflicted – add to the conflicts (save as standard tombstone document, not as a separate item. That makes it possible to handle the client side conflict resolution easily).


• Item was deleted locally – merge histories and do not generate conflict even if there is difference.


• Item wasn’t modified locally – delete it


• Item was modified locally – create conflict

There were actually to major decisions that made it drastically simpler. The first was to have the delete yes/no branching early. The second is more subtle, but probably even more important. The reason for this change was to remove the cost of storing the deleted tombstones as documents. The problem is what to do when we are actually getting a conflict. Previously, we would save the conflict into the document store, and that would be accessible using the standard RavenDB tooling. But how do you store a conflict between a deleted item and a locally modified item?
I decided that for that specific scenario, we are going to continue storing them as documents. That means that externally, nothing will change. This drastically reduced the complexity in the codebase and the problem set that I had to resolve.
Everything now works, and we are good to go.


DIGITAL JUICE