“A man is not dead while his name is still spoken.” – Going Postal, Chapter 4 prologue
A touching tribute, one I’ve heartily embraced.
http://www.gnuterrypratchett.com/
Category: Dev Stuff
Ending Illegal Fishing Using Games Technology
As we’ve just had our big media launch of the project I thought I’d share some information about the project I’ve been working on as part of my job at the Satellite Applications Catapult here in the UK.
For the passed 18 months we have been working on a project with Pew Charitable Trusts with the goal of building a system to use satellite derived data to track and deter illegal fishing at sea. Around one in five fish sold today is illegally caught so it’s a certainly a big problem to tackle.
Our system uses a live feed of vessel positions, currently provided by ExactEarth, using satellite AIS (Automatic Identification System) combined with a few other data sources to create a near real-time and historical view of fishing activity around the world.
We’re using the Unity games engine for data visualisation and as the interface to the system, both on our video wall and desktop machines, and we have a tablet version in the works. The video wall runs at 11536×3252 and Unity runs a treat!
We’ve a lot of vessels tracked at any point in time, all of which are rendered and animated on the screen at an accelerated rate for analysis. We are building the system using MMOs as inspiration as we’ve all seen how it’s possible to organise and work in a large group of people in raids to achieve a common goal. Having analysts working globally with large sets of data, it seemed a good model of interaction to follow.
A few screenshots and a video of the software in action are below, as the son of an engineer and ecologist I’m happy to say I’m proud of the work we’re doing and this is only the beginning!
More information on the project, can be found here;
http://www.pewtrusts.org/en/multimedia/video/2015/project-eyes-on-the-seas
We’re considering doing a live presentation of the system in a few weeks using Twitch or similar, if anyone would be interested in a demo or if you’ve any thoughts or comments please feel free too leave them below.
Keegan
Dad’s Clock
Work has been mental for a few months so despite doing odds and sods on Hugo and a buttload of work on our illegal fishing project at work (hopefully more on that soon) I’ve not posted anything in a while so thought I’d post about a clock I’m building for my Dad.
My dad’s an engineer, heavy fabrication mostly, designing and building access platforms for the nearby oil refinery for over 40 years and since he drove me school on my first day on his crane he’s been a big influence.
As a reference to his engineering heritage and my current work with satellite data I’m building an Arduino based clock that uses the GPS time signal to set the time and uses a set of voltage panels as the face. The GPS receiver is the Adafruit Ultimate GPS board and I’m using a DS1307 based board for the real time clock. The Ultimate GPS board is a bit OTT but it does allow me to receive a time signal indoors, as I only need one signal to get the time rather than the multiple signals needed for a full fix it works quite well.
The code is a work in progress but you’ll find it on my github page.
Geoserver, Leaflet Angular Directive and viewParams
Catchy title, I know…
In the big GIS project I’m working on we have a large set of data, 100 million rows+, which needs to be rendered as an overlay. This turned out to be tricky as I’m not really a GIS developer and learning as I go. I figured it out through trial and error so I’m writing this post to help those like me, hopefully I’m using the correct keywords I was looking for so that SEO picks this up!
Our frontend architecture is as follows;
To create the tiles in Geoserver we ended up creating a parametric SQL layer, more info on that here, and we are adding it as a WMS layer. What I discovered in the end is that to pass the viewParams through to Geoserver you need to add a layerParams option with viewParams within it, for example with start and end as SQL parameters;
layerParams: {
viewParams: "start:'2014-01-01T00:00:00.000Z';end:'2014-01-31T00:00:00.000Z'"
}In a WFS call the equivalent would be to append the following and one thing to note in both cases is to properly escape the characters;
VIEWPARAMS=start:'2014-01-01T00:00:00.000Z';end:'2014-01-31T00:00:00.000Z';
Hope this helps!
JavaScript IDE, Git and Deployment
So I was working on a set of scripts to automatically commit and deploy HTML, JavaScript and CSS stuff using PowerShell however as I’ve just discovered WebStorm which does all this and allows for live preview and debugging of code I’m not going to bother anymore!
After installing it I quickly managed to configure auto-deployment to a Linux server via SFTP and live previews in Chrome. Having spent the last few months getting to grips with JavaScript programming I have to say I’ve missed having a good IDE, live preview of code that updates on the fly really is a brilliant touch and should make things far easy moving forward as I rewrite our UI using AngularJS.
Note:- I had an issue testing my SFTP connection when I configuring the deployment location and received the following error;
“Conection to ‘192.168.0.10’ failed. Invalid descendent file name “Accept:”.”
It turns out that I’d managed to somehow create a file called “Accept:” in my home folder in Ubuntu and the WebStorm IDE doesn’t appear to deal well with filenames that contain ‘:’, at least I think that is why. All I know is I deleted the file and was then able to connect without issue. Hope this helps someone as it took a while to figure out this morning in my pre-caffeinated fugue.
Git, FTP and Cygwin
So I’ve been trying to get to grips with scripting for git to allow me to commit then deploy via FTP on Windows. I discovered git-ftp which looked promising but it was Linux flavoured, I followed the rabbit hole all the way to installing Cygwin and attempting to build it for Windows before realising I could just use PowerShell with Posh-Git and PSFTP (PowerShell FTP) instead. More to follow on that in the next post however though I’d share a few things I learnt about Cygwin first, no point in my learnings going to waste, this post is mostly a link dump.
Git-FTP can be found here and information regarding installing packages into Cygwin can be found at the source and there is a packpage manager, apt-cyg, that I never tried but looked useful.
A few issues I hit were that I sit behind a proxy at work so following this guide to configure Cygwin to have internet access. The guide works for http access and if you need https access, just used this command;
export https_proxy=https://user:pass@host:port/
To access my existing git folder which sits in my Documents folder I mounted my C: drive in Cygwin then created a symbolic link;
mkdir /c
mount c: /c
cd ~
ln -s /c/Users/Username/DocumentsThis creates a folder named Documents in your home folder which maps to your Documents folder in your Windows user folder. I found this in a cheat sheet on a Stanford users site, very useful so my thanks to them.
All fun learning, more info on my PowerShell solution in the next post.
SSHFS From Windows to Ubuntu
SSHFS is a file system that works over SSH, this allows for a secure connection to remote file systems and in my case will allow me to use the Windows based tools I’m familiar with against files on a remote Linux machine. I’m planning on getting to grips with Linux but it’s daft not to use the tools you know and this will likely prove very useful with the Raspberry Pi too.
The best method of using SSHFS with Windows I’ve found is outlined here, I’ve not tried with anything but a password yet but I’ve now mounted the disk of a user I have in a virtual Ubuntu server I’m running on my laptop. Seems to work a treat though not tried it over the network yet, with an upcoming move from Home Server to Ubuntu at home soon that will be likely be heavily tested.
PowerShell Guid Generation
Learning WIX at the minute and needed a Guid for the product ID, a quick search found an easy solution using PowerShell. To make it easier to paste into another application, this was my solution;
[System.Guid]::NewGuid().toString() | clip
This puts the Guid straight into the clipboard, ready to be used.
Generating a C# Class from XML
After a two week hiatus from work and blogging, spent mostly at the ever excellent Reading Beer Festival, I’m back at work and doing something so simple I’ve utterly forgotten how to do it. As this blog started as a way for me to keep track of random things I learn over the years, it’s time to write this one down.
Here’s the issue; I’ve created an XML file and want to create a C# class library to work with it in code. It’s really easy once you remember how, through the use of xsd.exe.
- Create xml file
- Run “xsd.exe filename.xml” to create an XSD file
- Run “xsd.exe /classes /language:CS filename.xsd” to generate a .CS file
One thing that always makes me chuckle is this line that is included at the top of each CS file;
“This code was generated by a tool.”
In this case, a forgetful one at that.
Installing Atlassian Stash on a Headless Ubuntu Server
Today’s post relates to my experience installing Atlassian Stash on a headless Ubuntu box. My first attempt ended with me trying all sorts of nonsense which left the server messy as hell, including an accidental installation of Unity…
The trickiest part for me was the installation of Java as the version available via the package manager, OpenJDK, didn’t seem to play ball with Stash. No idea why but one of the helpful fellows from Atlassian, Stefan Saasen, pointed me in the right direction with this article which I’ve used as basis for the steps below, the browser plugin steps were omitted for obvious reasons.
First of all you need to grab the tar file for Java from here then get it onto your server, the easiest way I found was to use an FTP application in SSH file transfer mode. Then connect to the terminal and run through these commands;
- Decompress the tar file noting that the file name will reflect the version you download and may differ from mine;
tar -xvf jdk-7u2-linux-x64.tar.gx - Move the JDK directory to /usr/lib;
sudo mkdir -p /usr/lib/jvm
sudo mv ./jdk1.0.7.0_17 /usr/lib/jvm/jdk1.7.0 - Now run these lines, no idea why, explanation in the comments please!
sudo update-alternatives –install “/usr/bin/java” “java” “/usr/lib/jvm/jdk1.7.0/bin/java” 1
sudo update-alternatives –install “/usr/bin/javac” “javac” “/usr/lib/jvm/jdk1.7.0/bin/javac” 1
sudo update-alternatives –install “/usr/bin/javaws” “javaws” “/usr/lib/jvm/jdk1.7.0/bin/javaws” 1 - Correct the permissions for the executables;
sudo chmod a+x /usr/bin/java
sudo chmod a+x /usr/bin/javac
sudo chmod a+x /usr/bin/javaws
If you are installing on a fresh server, that should be all you need for Java to work. If you’ve a few JVMs present on your machine, see the original article for steps to configure the default. For Stash, you can now follow the instructions here. One thing I found quite useful was the wget command that a colleague showed me, it makes downloading via the terminal quite easy, getting the Stash installer for example can be done like so;
wget http://www.atlassian.com/software/stash/downloads/binary/atlassian-stash-2.3.1.tar.gz
Note, the version I’ve listed there may not be the latest so you should grab it via here;
http://www.atlassian.com/software/stash/download
For editing the setenv.sh file, and text files in general, I’ve found nano really useful. I’ve not ventured into the murky depths of the text editor holy war yet but this does the trick for now. I had to run the same chmod against the Stash home directory I created but after that I was good to go. Open a browser to the path listed in the console and use the wizard to continue setup.