Subversion to TFS 2012 Migration: Conclusion

If you’ve been following this series of articles, I’ve been attempting to migrate from Subversion to TFS 2012 over the last six months or so and have given up on the idea.  It seems as though we are using a non-standard repository pattern which means the odds are stacked against us.

I’m not a newbie when it comes to troubleshooting TFS, I used to work in Microsoft Developer Support and it was my specialisation, but it shouldn’t be this hard to move from one platform to another.  I know that the Subversion adapter is Alpha but I also attempted to use SVNtoTFS and the trial version of Timely Migration’s tool too without luck.  I’m not sure if the way we’ve used Subversion makes it difficult to move history (update: likely yes) or if the tools just aren’t up to scratch with TFS 2012 but we need our version history and as we can’t get it into TFS, that’s that.

After a lot of effort, we’ve given up on migrating to TFS and instead we are going to use GIT and Stash, we’ve already licences for Jira and Bamboo too so will be investigating these as they are new products to me.  We’ll be self hosting on Linux which should be an experience too and as I’m a Linux newbie I’ll be blogging my experience as I go in order to help other Windows veterans get to grips with things, as well as a reference for myself.

Update: Before leaving the company I had a play with TFS 2013 Preview and had success with the migration, I was made redundant before completing the process though would suggest people try this route.

Subversion to TFS 2012 Migration: Part 3 – Troubleshooting

We hit more problems, as per my last article that means another post!  This post will detail the steps to troubleshoot a merge issue but the techniques I learned are good examples for generic troubleshooting of the TFS Integration Platform.  Our issue;
A set of files aren’t being checked in after a merge in SVN.

Looking at the Subversion log I can see that the files should be checked in during a merge at a specific changeset.  Looking at the TFSIP log you should see entries similar to  “Processing ChangeGroup”, searching the solution for this string takes us to the MigrationEngine.cs file.  I added an IF statement after the processing trace is added such that a break point can be hit when a certain changeset is reached;

TFST - Breakpoint

Adding a breakpoint at line 347 means that you’ll be able to step into the code when a problematic changeset is reached and troubleshooting can begin.

During troubleshooting this issue I randomly hit the following error on build from the  Interop.Subversion project, I resolved by cleaning only that project then rebuilding the solution;

error C1859: ‘Debug\Interop.Subversion.pch’ unexpected precompiled header error, simply rerunning the compiler might fix this problem        D:\tfsip\IntegrationPlatform\Adapters\Subversion\Interop.Subversion\AprPool.cpp

I ended up hitting the same issue as mentioned in part 1 where a manual merge was required and for the life of me I couldn’t remember how I fixed it the first time around.  Follows are my notes from each attempt;

Fail 1:

“1039” issue – Manual rename of folder in TFS and check in;

Source = SVN version 1039
Target = TFS checkin 942

 

Fail 2:

Undo pending change
Delete empty folder from folder B and checkin -> 942
Move from folder A to B and checkin -> 943

Fail 3:

Get latest, undo pending change then open in Windows Explorer.  Manually copy the files from source folder to target then within Source Explorer delete the source folder and add new items to target.  Check in an cross fingers.

1039 -> 942

Fail 4:

Rename B/WebSite to Website2 then checkin
Move A to B and check in

Manually map 1039 to 944
Hit start, map all conflicts to 944

Success:

Resolve Conflict in TE, Rename Server and check in
Map 1039 to 943 in TFS IP.

The simple step I should have taken, and wasn’t aware of, was to open Team Explorer and have a look at the list of pending changes.  This helps a lot and was the troubleshooting step that worked around this issue.  Migration continued

Around a thousand SVN revisions later we hit another issue, this time a runtime conflict; “Checkin failed for unknown reason”.   I opened VS to check pending changes and found missing files, for some reason the Subversion adapter wasn’t getting a few thousand of the files for this revision from the repository.  I manually got that revision from SVN and pasted into the TfsIpData folder and checked in to attempt a manual merge as before.  I managed to check it in but when I kicked off the migration again it threw a few hundred conflicts.

It failed, and I’m sorry to say this is where the story ends as we are cutting our losses with the migration and moving to GIT and Stash.  More on that in the next post.

Subversion to TFS 2012 Migration: Part 2

If you followed the steps in the previous article and everything went smoothly, good for you!  My migration hit a few snags, here are the big issues we hit and how we worked around them.

The first issue we hit was due to the fact we use HTTPS and hit an issue with the certificate, this isn’t immediately obvious from the Integration Platform UI however.  Follow the steps in Martin’s blog entry and you should be up and limping in no time!

That issue out of the way the migration started in earnest and we hit our first code bug in the adapter, it relates to the Subversion version control analysis provider and occurred when a null change set is detected in Subversion.  We added a check for null changesets at line 355 in SubversionVCAnalysisProvider.cs to allow analysis to continue.

SVNVCAP

 

Another issue resolved, migration was resumed.  This was when we hit a really odd issue, we started getting “path not found” errors, upon further investigation we discovered that for some reason there were folders in TFS that included “%20” rather than spaces.  The same folders in Subversion never had these characters so it seems the issue was with the adapter.  We tracked the issue to the BatchedItem.cs file and made the following change to the code to URL decode each path when the string has a value assigned, at which point we had to restart the migration from scratch as the damage was done.  The TFS11 (TFS 2012. codename “dev11”) adapter inherits an awful lot from TFS 2010, hence the TFS 2010 adapter needs updating despite us migrating to TFS 2012.

SVNURLENC

 

The last major issue we hit was another issue with missing files;
“Microsoft.TeamFoundation.VersionControl.Client.CheckinException: TF10141: No files checked in: resolve the conflicts and try again.”

It seems to be caused by a random deletion in Subversion that was somehow out of order and the adapter couldn’t handle it.  Setting the source and target to the same value, that of the TFS checkin rather than Subversion, resolved this “conflict” and the history appears in tact in TFS.  I’ve had to start the migration again and can’t get passed this error anymore so this may not be accurate, see part 3 for an update.  Regarding the confusing conflict resolution system in the Integration Platform, head back to Martin’s blog for an explanation.

The last hurdle wasn’t related to the Integration Platform but I’ll mention it hear for completeness, this blog is as much for my reference as for others after all.  We kept seeing a CRC error popping up in the logs similar to this;
“The CRC in GZip footer does not match the CRC calculated from the decompressed data.”

It seems that this was related to the network adapter on either the TFS box or my desktop, investigations are ongoing but it’s one or the other.  There was an issue in TFS 2010 regarding large files in IIS but it has been fixed and presumably included in TSF 2012, further reading can be found here;

http://social.msdn.microsoft.com/Forums/en-US/tfsbuild/thread/3000b6f7-f029-46e6-98c8-f0f7a21e5fec
http://social.msdn.microsoft.com/Forums/eu/tfswebaccess/thread/c9b773f9-3606-4013-a49a-a72dec34c97e

We haven’t quite finished out migration yet so there may yet be more issues we hit, if so I’ll add another post in this series.  Good luck and I hope this has helped.

Subversion to TFS 2012 Migration: Part 1

When I started at VoiceVault I mentioned I used to support Team Foundation Server in a previous life, it came to light that it has functionality that may benefit the company and make things a bit easier to manage than Subversion.  I was then tasked with investigating migration from Subversion to TFS, my first stop was to have a look at the TFS Integration Platform crafted by the ALM Rangers.  Alas, it wasn’t one of the projects I was involved in while I was a member… 

I thought I’d make notes regarding the issues I hit, along with the solutions, in the hope it’ll help others following the same path.  As the Subversion adapter wasn’t released and is still in “alpha” state it can be a bit tricky to get it up and running as you need to build the platform yourself first.

First of all, get the bits from here;
http://tfsintegration.codeplex.com/

Once downloaded and extracted, you should find a readme file in the “IntegrationPlatform” folder which you should read. Twice.  Maybe three times, just to be sure.  It includes information regarding building the platform specific to setting up the build environment as well as the required prerequisites to build the Subversion adapters.  I’d recommend grabbing the latest bits for each of the prereqs rather than the specific versions listed.  I’ll not add links here as they’ll be out of date in no time.  There are also a couple of batch files in the same location which automates a few of these steps, further details are in the readme file.

Once you’ve followed the above, open the solution in Visual Studio and attempt a build.  You’ll likely hit a few little issues, I recall I did, but they should be simple enough to work around and if not grab one of your devs to give you a hand.  The first time through I hacked and slashed at projects until I removed a number that I thought weren’t required, then once I messed it all up I started again.  One thing you can do is to not build the installers, I deselected these projects from the configuration manager at the solution level as I’d never be installing the platform, just building and copying across the binaries. 

With regards to the database, you can find it in the “Tfs_IntegrationPlatform” project in the “Framework” folder.  You should be able to deploy it from there.  A non-Express instance of SQL Server is recommended as if you’ve a large repository to migrate you’ll hit the size limit of Express quite quickly.  The connection string you’ll need to change for the tool to connect to the database is in the MigrationToolServer.config file, located in the “Tools\MigrationConsole” project.

Right, once you’ve got this far the fun really begins.  Load the TfsIntegrationShell.exe application in the “IntegrationPlatform\Binaries\Debug\Bin” folder, if you’ve a database deployed and the connection string updated you should be presented with the start screen.  Click “Create new” and navigate to the configurations folder, you should find an example Subversion config, then click open.  Select the source and targets then the dialog should change.  You’ll see a tab at the bottom of the screen labeled “xml”, if you have a number of users you wish to map then you’ll need to click on this and edit the <UserIdentityMappings /> section, an example is below; 

    <UserIdentityMappings EnableValidation=”false”>
      <UserIdentityLookupAddins />
      <DisplayNameMappings DirectionOfMapping=”LeftToRight”>
        <DisplayNameMapping Left=”tom” Right=”tfs\tom” MappingRule=”SimpleReplacement” />
        <DisplayNameMapping Left=”dick” Right=”tfs\dick” MappingRule=”SimpleReplacement” />
        <DisplayNameMapping Left=”harry” Right=”tfs\harry” MappingRule=”SimpleReplacement” />
      </DisplayNameMappings>
    </UserIdentityMappings>

Click save to database, things will happen, then click “Start” under “Current Migration” in the left hand menu.  If you are lucky, after a while, your migration will be complete.  If you aren’t then the next article in the series may be of some help as it details some of the issues we hit, along with the fixes and links to the source materials for further reading.