Git, FTP and Cygwin

So I’ve been trying to get to grips with scripting for git to allow me to commit then deploy via FTP on Windows. I discovered git-ftp which looked promising but it was Linux flavoured, I followed the rabbit hole all the way to installing Cygwin and attempting to build it for Windows before realising I could just use PowerShell with Posh-Git and PSFTP (PowerShell FTP) instead. More to follow on that in the next post however though I’d share a few things I learnt about Cygwin first, no point in my learnings going to waste, this post is mostly a link dump.

Git-FTP can be found here and information regarding installing packages into Cygwin can be found at the source and there is a packpage manager, apt-cyg, that I never tried but looked useful.

A few issues I hit were that I sit behind a proxy at work so following this guide to configure Cygwin to have internet access.  The guide works for http access and if you need https access, just used this command;

export https_proxy=https://user:pass@host:port/

To access my existing git folder which sits in my Documents folder I mounted my C: drive in Cygwin then created a symbolic link;

mkdir /c
mount c: /c
cd ~
ln -s /c/Users/Username/Documents

This creates a folder named Documents in your home folder which maps to your Documents folder in your Windows user folder.  I found this in a cheat sheet on a Stanford users site, very useful so my thanks to them.

All fun learning, more info on my PowerShell solution in the next post.

SSHFS From Windows to Ubuntu

SSHFS is a file system that works over SSH, this allows for a secure connection to remote file systems and in my case will allow me to use the Windows based tools I’m familiar with against files on a remote Linux machine.  I’m planning on getting to grips with Linux but it’s daft not to use the tools you know and this will likely prove very useful with the Raspberry Pi too.

The best method of using SSHFS with Windows I’ve found is outlined here, I’ve not tried with anything but a password yet but I’ve now mounted the disk of a user I have in a virtual Ubuntu server I’m running on my laptop.  Seems to work a treat though not tried it over the network yet, with an upcoming move from Home Server to Ubuntu at home soon that will be likely be heavily tested.

Generating a C# Class from XML

After a two week hiatus from work and blogging, spent mostly at the ever excellent Reading Beer Festival, I’m back at work and doing something so simple I’ve utterly forgotten how to do it.  As this blog started as a way for me to keep track of random things I learn over the years, it’s time to write this one down.

Here’s the issue; I’ve created an XML file and want to create a C# class library to work with it in code.  It’s really easy once you remember how, through the use of xsd.exe.

  1. Create xml file
  2. Run “xsd.exe filename.xml” to create an XSD file
  3. Run “xsd.exe /classes /language:CS filename.xsd” to generate a .CS file

One thing that always makes me chuckle is this line that is included at the top of each CS file;
“This code was generated by a tool.”

In this case, a forgetful one at that.

Installing Atlassian Stash on a Headless Ubuntu Server

Today’s post relates to my experience installing Atlassian Stash on a headless Ubuntu box. My first attempt ended with me trying all sorts of nonsense which left the server messy as hell, including an accidental installation of Unity…

The trickiest part for me was the installation of Java as the version available via the package manager, OpenJDK, didn’t seem to play ball with Stash.  No idea why but one of the helpful fellows from Atlassian, Stefan Saasen, pointed me in the right direction with this article which I’ve used as basis for the steps below, the browser plugin steps were omitted for obvious reasons.

First of all you need to grab the tar file for Java from here then get it onto your server, the easiest way I found was to use an FTP application in SSH file transfer mode.  Then connect to the terminal and run through these commands;

  1. Decompress the tar file noting that the file name will reflect the version you download and may differ from mine;
    tar -xvf jdk-7u2-linux-x64.tar.gx
  2. Move the JDK directory to /usr/lib;
    sudo mkdir -p /usr/lib/jvm
    sudo mv ./jdk1.0.7.0_17 /usr/lib/jvm/jdk1.7.0
  3. Now run these lines, no idea why, explanation in the comments please!
    sudo update-alternatives –install “/usr/bin/java” “java” “/usr/lib/jvm/jdk1.7.0/bin/java” 1
    sudo update-alternatives –install “/usr/bin/javac” “javac” “/usr/lib/jvm/jdk1.7.0/bin/javac” 1
    sudo update-alternatives –install “/usr/bin/javaws” “javaws” “/usr/lib/jvm/jdk1.7.0/bin/javaws” 1
  4. Correct the permissions for the executables;
    sudo chmod a+x /usr/bin/java
    sudo chmod a+x /usr/bin/javac
    sudo chmod a+x /usr/bin/javaws

If you are installing on a fresh server, that should be all you need for Java to work.  If you’ve a few JVMs present on your machine, see the original article for steps to configure the default.  For Stash, you can now follow the instructions here.  One thing I found quite useful was the wget command that a colleague showed me, it makes downloading via the terminal quite easy, getting the Stash installer for example can be done like so;
wget http://www.atlassian.com/software/stash/downloads/binary/atlassian-stash-2.3.1.tar.gz

Note, the version I’ve listed there may not be the latest so you should grab it via here;
http://www.atlassian.com/software/stash/download

For editing the setenv.sh file, and text files in general, I’ve found nano really useful.  I’ve not ventured into the murky depths of the text editor holy war yet but this does the trick for now.  I had to run the same chmod against the Stash home directory I created but after that I was good to go.  Open a browser to the path listed in the console and use the wizard to continue setup.

jedibowler on Git Hub

As I’m implementing Git at work, I’ve decided to start using it at home too and for the site.  As such I’ve dusted off an old account I’d forgotten I’d created and you’ll find me on github as jedibowler.  The first of my projects to be posted is the code for the network enabled temperature/humidity logger from a few weeks back which can be found here.

Subversion to TFS 2012 Migration: Conclusion

If you’ve been following this series of articles, I’ve been attempting to migrate from Subversion to TFS 2012 over the last six months or so and have given up on the idea.  It seems as though we are using a non-standard repository pattern which means the odds are stacked against us.

I’m not a newbie when it comes to troubleshooting TFS, I used to work in Microsoft Developer Support and it was my specialisation, but it shouldn’t be this hard to move from one platform to another.  I know that the Subversion adapter is Alpha but I also attempted to use SVNtoTFS and the trial version of Timely Migration’s tool too without luck.  I’m not sure if the way we’ve used Subversion makes it difficult to move history (update: likely yes) or if the tools just aren’t up to scratch with TFS 2012 but we need our version history and as we can’t get it into TFS, that’s that.

After a lot of effort, we’ve given up on migrating to TFS and instead we are going to use GIT and Stash, we’ve already licences for Jira and Bamboo too so will be investigating these as they are new products to me.  We’ll be self hosting on Linux which should be an experience too and as I’m a Linux newbie I’ll be blogging my experience as I go in order to help other Windows veterans get to grips with things, as well as a reference for myself.

Update: Before leaving the company I had a play with TFS 2013 Preview and had success with the migration, I was made redundant before completing the process though would suggest people try this route.

Subversion to TFS 2012 Migration: Part 3 – Troubleshooting

We hit more problems, as per my last article that means another post!  This post will detail the steps to troubleshoot a merge issue but the techniques I learned are good examples for generic troubleshooting of the TFS Integration Platform.  Our issue;
A set of files aren’t being checked in after a merge in SVN.

Looking at the Subversion log I can see that the files should be checked in during a merge at a specific changeset.  Looking at the TFSIP log you should see entries similar to  “Processing ChangeGroup”, searching the solution for this string takes us to the MigrationEngine.cs file.  I added an IF statement after the processing trace is added such that a break point can be hit when a certain changeset is reached;

TFST - Breakpoint

Adding a breakpoint at line 347 means that you’ll be able to step into the code when a problematic changeset is reached and troubleshooting can begin.

During troubleshooting this issue I randomly hit the following error on build from the  Interop.Subversion project, I resolved by cleaning only that project then rebuilding the solution;

error C1859: ‘Debug\Interop.Subversion.pch’ unexpected precompiled header error, simply rerunning the compiler might fix this problem        D:\tfsip\IntegrationPlatform\Adapters\Subversion\Interop.Subversion\AprPool.cpp

I ended up hitting the same issue as mentioned in part 1 where a manual merge was required and for the life of me I couldn’t remember how I fixed it the first time around.  Follows are my notes from each attempt;

Fail 1:

“1039” issue – Manual rename of folder in TFS and check in;

Source = SVN version 1039
Target = TFS checkin 942

 

Fail 2:

Undo pending change
Delete empty folder from folder B and checkin -> 942
Move from folder A to B and checkin -> 943

Fail 3:

Get latest, undo pending change then open in Windows Explorer.  Manually copy the files from source folder to target then within Source Explorer delete the source folder and add new items to target.  Check in an cross fingers.

1039 -> 942

Fail 4:

Rename B/WebSite to Website2 then checkin
Move A to B and check in

Manually map 1039 to 944
Hit start, map all conflicts to 944

Success:

Resolve Conflict in TE, Rename Server and check in
Map 1039 to 943 in TFS IP.

The simple step I should have taken, and wasn’t aware of, was to open Team Explorer and have a look at the list of pending changes.  This helps a lot and was the troubleshooting step that worked around this issue.  Migration continued

Around a thousand SVN revisions later we hit another issue, this time a runtime conflict; “Checkin failed for unknown reason”.   I opened VS to check pending changes and found missing files, for some reason the Subversion adapter wasn’t getting a few thousand of the files for this revision from the repository.  I manually got that revision from SVN and pasted into the TfsIpData folder and checked in to attempt a manual merge as before.  I managed to check it in but when I kicked off the migration again it threw a few hundred conflicts.

It failed, and I’m sorry to say this is where the story ends as we are cutting our losses with the migration and moving to GIT and Stash.  More on that in the next post.

TaW: Week 6 – Arduino Based Syma S107G Tx and Xbox 360 Controller Interface

After the last two TaW items have been less than successful, and as a last minute tie in to the No ‘Air Ambulance Challenge I’ve decided to upgrade one of my old projects.  Rather than an RC car I’ve gone with a Syma S107G 3ch helicopter and standing in for the Netduino is an Arduino.  I had a go at implementing the IR protocol in the .NET Micro Framework but it proved a little tricky.  More information can be found in this post along with a video demo.

Xbox 360 controller interface to enable control of a Syma S107G.
Xbox 360 controller interface to enable control of a Syma S107G.

Subversion to TFS 2012 Migration: Part 2

If you followed the steps in the previous article and everything went smoothly, good for you!  My migration hit a few snags, here are the big issues we hit and how we worked around them.

The first issue we hit was due to the fact we use HTTPS and hit an issue with the certificate, this isn’t immediately obvious from the Integration Platform UI however.  Follow the steps in Martin’s blog entry and you should be up and limping in no time!

That issue out of the way the migration started in earnest and we hit our first code bug in the adapter, it relates to the Subversion version control analysis provider and occurred when a null change set is detected in Subversion.  We added a check for null changesets at line 355 in SubversionVCAnalysisProvider.cs to allow analysis to continue.

SVNVCAP

 

Another issue resolved, migration was resumed.  This was when we hit a really odd issue, we started getting “path not found” errors, upon further investigation we discovered that for some reason there were folders in TFS that included “%20” rather than spaces.  The same folders in Subversion never had these characters so it seems the issue was with the adapter.  We tracked the issue to the BatchedItem.cs file and made the following change to the code to URL decode each path when the string has a value assigned, at which point we had to restart the migration from scratch as the damage was done.  The TFS11 (TFS 2012. codename “dev11”) adapter inherits an awful lot from TFS 2010, hence the TFS 2010 adapter needs updating despite us migrating to TFS 2012.

SVNURLENC

 

The last major issue we hit was another issue with missing files;
“Microsoft.TeamFoundation.VersionControl.Client.CheckinException: TF10141: No files checked in: resolve the conflicts and try again.”

It seems to be caused by a random deletion in Subversion that was somehow out of order and the adapter couldn’t handle it.  Setting the source and target to the same value, that of the TFS checkin rather than Subversion, resolved this “conflict” and the history appears in tact in TFS.  I’ve had to start the migration again and can’t get passed this error anymore so this may not be accurate, see part 3 for an update.  Regarding the confusing conflict resolution system in the Integration Platform, head back to Martin’s blog for an explanation.

The last hurdle wasn’t related to the Integration Platform but I’ll mention it hear for completeness, this blog is as much for my reference as for others after all.  We kept seeing a CRC error popping up in the logs similar to this;
“The CRC in GZip footer does not match the CRC calculated from the decompressed data.”

It seems that this was related to the network adapter on either the TFS box or my desktop, investigations are ongoing but it’s one or the other.  There was an issue in TFS 2010 regarding large files in IIS but it has been fixed and presumably included in TSF 2012, further reading can be found here;

http://social.msdn.microsoft.com/Forums/en-US/tfsbuild/thread/3000b6f7-f029-46e6-98c8-f0f7a21e5fec
http://social.msdn.microsoft.com/Forums/eu/tfswebaccess/thread/c9b773f9-3606-4013-a49a-a72dec34c97e

We haven’t quite finished out migration yet so there may yet be more issues we hit, if so I’ll add another post in this series.  Good luck and I hope this has helped.