Resolving Delta source ended unexpectedly SVN issue

A couple of days ago, I started working on migrating a new Libcloud website to our Apache SVN website repository.

During this migration, I made a bunch of commits to the SVN repository. Those commits were composed of additions (adding source code and generated static files for a new website) and deletions (removing old website source code and data).

At some point I’ve updated the content, re-generated the website and wanted to commit the modified files again.

Everything looked fine while the changes / deltas were being transmitted, but just when the server supposed to ACK all those changes, I have received the following error:

Transmitting file data ............svn: Commit failed (details follow):
svn: Delta source ended unexpectedly

I have never received this error before, but I suspected this might be related to some weird consistency issues I saw when I was originally adding new website files and removing old ones.

When I was doing that, I ran svn update a couple of in between the commits and after one large commit, I have received old updates from the server even though I shouldn’t, because the local repository should have all the changes and represent a latest state.

One of the commits I have made was big and contained a lot of changes so I immediately suspected this might be related to Apache GEO load balanced SVN setup and some weird replication / out-of-sync issues.

I waited a couple of minutes, ran svn update again and the issue seemed to have resolved itself (I have received all the changes which were already present locally before the weird sync thing). It looked like I was either being redirected to the same SVN server again or all the changes have now been fully replicated to the other server (citation needed, I’m not actually 100% sure that current replication is asynchronous and not synchronous).

Anyway, now back to the original issue.

After I have received this error message, I have tried a couple of things:

  1. I have checked the output of svn status
  2. I have tried checking out an earlier revision using (svn update -r <rev>)
  3. I have tried a clean checkout (svn co <repo url> clean-checkout)

None of those things worked. svn status only showed modifications (M) to the existing files (no additions) and after I have re-added the modified files (yes, those files did not include .svn directories) after trying #2 and #3 I have received the same error during the commit phase.

At this point, I ran out of the ideas and I didn’t want to bug the ASF infrastructure team just yet, so I have tried Googling around for a solution. I came back empty handed, but at least a consensus seemed to be that this is caused by a consistency issue between a local checkout and a remote repository, something I have originally suspected.

To try figure out which file or folder is an causing this issue, I wrote a little script which commits all the files one by one.

(It’s worth nothing that other project members probably weren’t too happy, because this resulted in ~100 commits and each commit results in an notification email being sent to our commits@ mailing list.)

#!/bin/bash

FILES=$(svn status generated/ | awk '{print $2}')

for file in ${FILES}; do
    svn commit ${file} -m "Regenerate website."
done

During those commits, all the files except 3 were committed successfully. For those three offending files, the server returned an error message which said that those files were not part of the repository. This was weird, because svn status showed those files being modified and no additions.

To try to resolve this issue, I have tried removing those files (svn rm), adding them back (svn add) and committing the changes.

# backup to-be removed files (exclude .svn directories)
svn rm generated/blog/tags/dir1
svn rm generated/blog/tags/dir2
svn rm generated/blog/tags/dir3
# re-added changed files back
svn add generated/blog/tags/*
svn commit -m "Regenerate website."

This resolved the issue and this time I have received no errors while transferring the deltas.

I’m still not exactly sure what caused this issue (there was obviously an inconsistency between a local and a remote state), but I was at least happy I have resolved it and I don’t need to deal with svn anymore.

Edit (January 1st, 2014): Justin confirmed that Apache SVN setup is using asynchronous replication which explains the first issue I have encoutered. He also noted that I could, to avoid replication issues, explicitly choose to only work with master by using svn relocate.