help. I’m being assimilated!

The largest source of problems in a project is us bumbling humans, and we’re not just talking about the bugs that we introduce!. Damn those pesky humans.

Automate Everything

Ok, so how do we help obsolete ourselves? Build automation is a really good start. As Steve McConnell once said “The build is the heartbeat of the project. If the hearts not beating the project is dead.” If that metaphor’s good enough for Steve, its good enough for me. Automated building is one of the best tools to keep that heart beating.

So, one of the keys to having a project run as smoothly as possible is automating the builds. There are many reasons for wanting to automate the build :

1. Repeatability
   You get the same output artefacts every time. No forgotten steps, miss-typed commands, forgotten copies. Trusting the process to give you the same outputs means you no longer have to worry about the end result working.
2. Create different builds automatically
   even though its a good idea to run different configurations locally, you can perform pre-configured builds for a specific purpose without messing with your local setup. Debug,Release,Nightly etc, different build configurations can be run on demand, or on schedule.
3. Reduced Interruptions.
   It’s subtle, but often overlooked, you get to reduce the number of people that interrupt you to get the latest version. Reducing the number of interruptions to your flow keeps up your development momentum, and its not very often you get to do that!
4. Tests can be run automatically.
   One of the great things about modern development is that there are plenty of Unit test frameworks available. Unit tests , no matter how time consuming can be executed against the build. Really time consuming operations can be run overnight, on a scheduled build.
5. Build Farms.
   Real men have build farms. More Pc’s projects a manly image. Ok, the wife might dispute that, she just seems to call it clutter. But having a good automated build system means that you can distribute the build amongst multiple PC’s, reducing time consuming builds and utilising those old 386’s you’ve got holding the doors open. Well, ok, it does actually have to qualify as a pc.
6. Time.
   Another important factor. All the time you used to spend running the various builds & unit tests you can now spend on developing the product. Multiply that by the number of members in your team.

So, it all sounds good in practice, but how do we achieve these benefits in reality?

An Automated Build System is for the project, not just for Christmas

Firstly this is really an up-front decision. The more complex the project gets the more time consuming it will be to switch systems. So I recommend making the automated build tool one of the first choices in the project.

Choosing which system you’ll implement depends largely on your budget.

Microsoft Team Foundation Server (TFS)

TFS provides all the automation you’d need and more, but is pretty costly, and requires a reasonable amount of infrastructure and admin to get up and running smoothly. Its not something you’d consider implementing in small teams and projects, unless it’s maintained outside your project. You can read more about TFS on MSDN

Two others worth considering are CIFactory and Team City.

CIFactory

It’s quite amazing , but CIFactory is the work of one man – Jay Flowers. Ok, fair enough he did have a little help where the CIFactory community had submitted various add on packages, but the all of the CIFactory framework was done by Jay. When you look at what it does, that’s a phenomenal piece of work by one guy, who wasn’t doing it as his day job either!

I implemented CIFactory on the last project, which started about a year ago, and was really happy with how it went. The install was straight forward, and the packages it provided for things like Unit testing , and code coverage were all up to date and matched what I was already using day to day, so at the time it was a perfect fit. The range of pre-configured packages is pretty good too, including nCover, mbUnit Analytics, and even Simian!

The really nice thing about CIFactory is that once it’s installed the whole system builds itself as well as your projects. All of the CIFactory tools and scripts are held in the same source hierarchy as your code, and the beauty of this is it’s completely self contained. To recreate your build system, all you have to do is re-extract the entire source tree to a HDD and point IIS back at the folder where the report pages are. It really is a nice clean and ultra flexible system. That may not seem like a major benefit, but for small teams spending any more time than necessary on tooling is obviously a big negative.

Now for the bad news.

When starting a project for the new client this time round I naturally went back to the slick tool I used over a year ago, and found little had changed. This is absolutely NOT a dig at Jay! Repeat, NOT disrespecting ANY of Jay’s work. His body of work is amazing, and I still had CIFactory installed and running with few modifications (as in the image), but it did take me more than a day to sort out. And to be perfectly honest staring at pages of nAnt scripts and XML files isn’t particularly pleasurable. Not only that, but many of the automatically installed packages now have broken links, and the pre-installed packages are now out of date. After trawling the CIFactory pages, Jays blog, and the forums, it was kind of obvious that CIFactory has probably reached the end of its natural lifespan, activity seemed to be very low on all fronts. I expect that there will still be the hardcore users that like it for its pure configurability (and it’s probably still the best for that) but it just seems like too much effort when there are easier alternatives.

Team City

Yes, I’ve discovered Team City. If you’re a ReSharper users (if not why not? you have to be start raving BONKERS) then you’ll understand why I’m in love with TC. It has all of JetBrains slickness packaged up in an easy to use, and easy to setup automated build system. JetBrains really are the Dogs Danglies, and I’d be more than happy to bear children to them.

Ok, enough with the sycophantic JetBrains stalking, trust me, Team City is about the best thing you’ll come across for automated building that doesn’t require you to sacrifice your first born to get installed and working. Not only that, but there are Plug-ins for the most popular IDE’s and task tray notifications that you can download from the interface. To be fair, CIFactory (Built on Cruise Control) also had task tray notifications with cctray.

What’s even more amazing is that the standard version is FREE! yes. Read that again FREE!

Ok, there’s FREE and there’s FREE, so this version of FREE has licensing restrictions, the most limiting of which is you can only install 3 build agents, but the other restrictions aren’t really that restrictive. So, given that there are only 2 developers that need to build our new project, I installed the latest version of TeamCity on our build server. I then set up our projects with the minimum of fuss. A minor problem occurred because the Solution builder that’s included can’t build Deployment projects. This isn’t a problem with Team City per se, only devstudio can build web deployment projects. This was easily fixed by using Team City’s command line build runner and building the offending setup project directly with devenv.exe! Even though I hadn’t really used TC before I had it building the projects (command line builder included) in about 3 hours. That more like it!

A feature of both CIFactory and TC is that they release the build artefacts from at the web page – so anyone with a browser can download the build output.

CIFactory does this by zipping up the project using a nAnt file set that looks at the project’s output folder. This is a bit of a problem if you want to include other files in the output. No big deal, but you have to go in and get your hands dirty with the XML properties files for the deployment stage.

With TC, the artefacts are held inside your project, exactly where you left them!. Generally these are generated in your projects bin/release folder by default. TC Allows you to expose any path or file relative to your projects root as an artefact, so you can include anything you like by simply providing TC  with a nAnt style wildcard spec.

Once the project is built the resulting artefacts are available for download.

  Here we have the build output from the deployment project.

Now anyone with a browser can retrieve the latest version of your project without interrupting you.

One thing that TC does do that CIFactory doesn’t is distribute the build. I used all 3 licences for the build agents for our mini build farm. The first machine is the dedicated build machine where the TC server is running. This machine also functions as a build agent.

Notice that the Build machine CPU rating. It’s  core2 duo E6600. Note my machine’s CPU rating. We just WONT go into that. Keep in mind the build machine is running inside a VM, with Oracle11, SQLServer 2008, and IIS running in it. <sigh>.

Once the build agents are set up, TC distributes a running build according to a set of rules you specify. Our TC server is set to distribute to the fastest build agent first. Another good choice would be the first Idle agent too. Since there are only 3 of us and the build doesn’t take that long at the moment, I don’t think the distribution strategy matters a great deal.

Here TC is distributing the build across the agents.

One nice feature of all of TC’s Ajax enabled interface is that you still manipulate controls whilst the server/agents are building. For example, as soon as the artefacts start becoming available you can begin download.

Another feature of TC is that you can see the build statistics on a per agent basis,

I guess this would be more useful in a large farm, but i guess its still useful to know which agents are successfully building the project. If they’re not it might point to environmental problems on that particular machine.

When TC is performing a GET to extract the sources, well, with subversion anyway, it actually does an export of the source tree, so you get an unpolluted source hierarchy from which you can extract your artefacts. You can also control who does the checking out, the agents or the server – checking out on the server reduces the load on the source control system, since only one machine is doing the full checkout – but then the sources are distributed to each agent as TC calls upon each agent to build.

All in all TC is pretty amazing, and the fact that you can install it for small dev teams for free makes it my current best friend !

Coming next, a step by step guide on how to set up a complete system from start to finish. Using Subversion as the source repository, and then installing TC on top of that, all with the canonical hello world project!

Here’s the courtesy link to his website

Either way, its the one that checks files as they’re accessed.

McAfee.. helping keep files clean, and product deadlines away.

Image via Wikipedia

I’ll never understand how some people in IT departments don’t grasp the value of a developers time. Its a false economy to provide anything but the absolute fastest PC for a developer. I’m not really sure where the real problem lies. Is it that non-techie people rise to management positions without realising what precisely their reports are doing?

Take the current client. We’re working on incredibly old PC’s – P4’s with 2gb of ram and 40gb.  DevStudio is simply un-useable. I don’t care what the minimum supported spec says, it’s painful. We’ve managed to get some new PC’s ordered, but in most other places its simply not that easy. We still have to justify why we’d like large monitors for example.

Who creates those kinds of obstacles? How they *know* what size monitor I’ll need?

There’s plenty of supporting evidence for productivity gains for using multiple monitors, check out Jeff Attwoods blog for well over 10 posts on this subject alone. In fact check out Jeff’s own setup.

Personally I use dual 24”  Samsung 245B’s. But try explaining this to some guy in procurement who’s sitting in front of his own single little 19” monitor and you’re in for a struggle. Maybe its because monitors are something tangible that they can actually compare the dimensions, so maybe the question “why would anyone need a larger monitor than mine?” creeps into their heads, whereas they can’t easily measure something like ram size or cpu speed.

With our current PC’s the edit-compile-debug feedback loop takes way too long, and when the build is finished the PC takes at least 20 seconds to become responsive again. The project isn’t very large, a web project with a separate DAL and a unit test project. Still it takes what seems like a decade to complete a single build. By comparison my home machine takes less than 5 seconds.

Consider building the code an average of 100 times a day. This is not unreasonable, it equates to roughly one compile every 5 minutes. Not a massive amount when a project is in full flow. Yet if I’m wasting 20 seconds for those 100 builds then that soon mounts up to a total of over 1/2 an hour. If the build is taking 1 minute instead of 20 seconds I’m wasting over an hour a day waiting for my PC to catch up. Sum that over a week and you’re losing almost a whole day. Over a month, that’s almost a week. in my case this is currently WAY worse. A typical build will take over 3 minutes.

Suddenly when you start totalling up all those wasted seconds it starts costing significant amounts. At an average contract rate of £40 an hour that’s over a thousand pounds a month you’d be burning, all over the cost of a 500 quid PC. If the contract lasts for year.. well, you do the math.

But there’s something more insidious here, and i fell into the trap this very morning. I block copied some code <yes, gasp>, a single line, setting up an OracleParameter for a query. Now this parameter did fit entirely on the width of my tiny monitor. The trouble is as I edited the start of the line the text pushed off the right hand side of my monitor. I could no longer see the end of the line. This in itself was not a real problem except that I’d forgotten to edit & remove a bit of code assigning a value, from the end of each of those block copied lines. It took me two subsequent edit-compile-test loops which on this PC took almost 15 minutes to complete, before I discovered what I’d messed up.

Not having the correct equipment leads to a increase in defects.

I’d wasted 15 whole minutes looking for something that I wouldn’t even have done in the first place if I could see the line ends. This only gets worse as you use languages like XAML or ASP where the inserted code sometimes extends beyond even the end of my 24” widescreen. Admittedly solving it is only a code format away but wrapping lines of code sometimes makes it harder to comprehend, and even then its another step that takes time away from you solving the problem at hand.

It’s a false economy believing that using anything but the best possible PC’s for developers is a saving.. The real question is how do you justify all of this to the purse string holders if they don’t realise it themselves?? Sometimes even demonstrating the financial penalties isn’t enough!

Oracle isn’t my Friend.

So I dig out reflector and run the ODP.Net assemblies through one of the reflector add-ins that brings it back to something more readable.

On inspection it seems that they’re mixing the domain values for db nulls. the OracleString performs a conversion when the column is null and actual returns the string “null”! Poor? Well I think so.

So, the next step is to look through the designer code that the ODT spits out when you drag a stored procedure onto a data set design surface. It appears to work ok, until you try using any procedure that involves varcar2 out parameters. Then it chokes. Nice.

The problem is it looks like their generated null checks aren’t working correctly. They’re testing the returned objects against NULL, and then if the object isn’t null they’re comparing the objects Value with System:DBNull. This obviously wont work since an they always return an Oracle type and the value throws an exception if the Oracle type is null. Not only that, but their implicit casting in the generated code is broken too, although that seems to be a symptom of their checks for null failing and going on the read the Value in order to cast it to a real .Net type.

The fixes are pretty simple, but it does involve editing the Table Adapters designer .CS file and fixing their ropey code.

I encountered these problems a couple of years ago, and swore never to trust them again, but heh, this is 3 years later, surely they’ve fixed the casting bugs?

Apparently not.

So, if you have the choice stick with SQLServer. For all MS’s flaws they seem to emit bug free code, and they do fix stuff.

<rant over/>

What I do find a bit puzzling is that other than Sheffield station there’s no way to find out which trains stop at your station, or what time it will arrive. Where there’s more than one train stopping at your destination you have no way of knowing which one will be quicker other than to ask. Even Sheffield’s fancy display board doesn’t show you the arrival times, but it does, however, show you which stations the trains will stop at.

So for instance, although there are trains that go direct from Sheffield to Leeds these can take up to an hour and 40 minutes. However, the cross country trains that start at Plymouth and end in Edinburgh also stop at both Sheffield and Leeds and yet only take 50 minutes. The trouble is you have no way of finding that out other than asking, or being a regular and already knowing!

It all seems a bit odd to me. Anyway, its no longer taking me 3.5 hours to commute! Yay!

As for the work, it does seem there’s more ASP.NET that I expected, but the data that’s driving the site is pretty interesting to work with, and because we’ve got a pretty switched on Oracle developer it’s making my life really easy! He has complete control over the organisation of the data, and if we need something that he hasn’t thought of it’s a simple task to ask him to add it, no politics involved which is a great relief.

T-Mobile Internet

For the T-Mobile internet dongle, apart from the  slightly patchy connections whilst on the train I’ve actually used less than 80mb, which is pretty good really. I was thinking that I’d be going over my tariff limit but on reflection i doubt I’ll even come close.

That 80mb consists of email, some web research and a bit of online development  i.e. uploading files to a couple of my sites. I have noticed that as soon as the connection is made, the tmobile connection manager’s transfer count starts climbing and reaches 160K of transfers, without me actually doing anything. my daughter has borrowed it too, so I was expecting the whole allowance to be used but she only ended up using 28mb for a whole night of the usual facebook/bebo/tagged rubbish !

So, Good news I dont think I’ll end up going over the limit at all!

Image by Neil T via Flickr

Ok, so after after a short break over Christmas, I though I’d better pull my socks up and start looking for a new contract. As everyone returned to work on the 5th of Jan, I wasn’t expecting anywhere near the response I got after posting my CV online! Either my C.V. is absolutely amazing or the IT market is ridiculously understaffed! I’d like to think it was the former, but I suspect it’s the latter! Just over 3 days into the first week and I was getting over 25 calls a day. by Thursday I’m signed up for a 6 month contract in Bradford!

Since trying to get to Bradford by car would be a real PITA , I’ve opted for a more comfortable commute and gone by train. So not only can I get a good hour and half of work done I can actually blog more often too Finally might get a chance to write as many articles as I keep promising!

I did buy a nice little T-Mobile internet dongle too, so I can be connected whilst travelling – T-Mobile seemed like to best deal because of their usage policy. If you happen to go over their limits, there’s no real penalty. Unlike all the other operators which charge over £1 a megabyte! £1 a megabyte! That’s totally ridiculous! If I accidentally went over my limit and watched a day of youtube videos (well ok, you’d have to be pretty brain dead to watch *anything* on youtube) I could end up with a bill of over £1000! . Anyway. after a few days of email / blogging and googling, I’ll post an update on how long my usage cap is likely to last!