In IT, what is a good measure of success?

Whether you adopt ITIL, MOF or CoBIT, eventually you’re going to come across KPI’s, or Key Performance Indicators.  The idea is that you can measure success by identifying indicators (quantitative, directional, financial) which are important to the organisation, and then use these to identify both opportunities to improve, and measure improvement over time.

And why not find a way to measure improvements so you can prove that the all the effort you’re putting in to changing your IT Services is working?  The problem is the assumptions on which most KPI’s are based.  The idea is that the indicator is (a) important to the organisation, and (b) measurable in some way.

I have two problems with that.

1. ‘The Organisation’ isn’t a thing, it’s a concept, which means finding out what is important to the organisation is difficult at best.  You might be able to identify what’s important to the head of Finance, the person in charge of Estates, or your most senior Academic, but ‘The Organisation’ isn’t something that has an opinion.  Even if the Senior Management Team/Faculty Board/Council can reach an consensus on what is important, this doesn’t necessarily lead to the definition of meaningful indicators.
2. There is a tendency to focus on the indicators that are easily measured, but these are only part of the picture.  Call response times, calls fixed within SLA, system downtime are all favourites of the IT Service Manager.  However, the effectiveness of IT service delivery is not measured by downtime alone, it’s measured by it’s impact on the effectiveness and efficiency of business processes. If users don’t engage with the systems and use all the functionality they practically can, then your IT Service isn’t performing, but no common quantitative measurement will detect that, unless there’s some serious analysis of business processes going on.

In my experience, there is no substitute for knowing your users.  By this I don’t mean personally knowing every single user (that’s a big ask) but at two levels; in theory and in practise.

The theory

This means understanding functionally what users do, what their role in the organisation is, how they interact with their colleagues and outside bodies.  Understanding their role means understanding their individual requirements, and understanding how users interact gives you the wider organisational requirements.

The practise

You have to have some personal connection with a proportion of your users, to give you a more complete understanding which theory cannot.  Many chose to use committees or user groups to connect with users, which are of limited value, because they’re almost exclusively made up of people who make up committee and user groups everywhere.  Those users are more vocal, inclined to push their point of view rather than their peers, and not representative of the majority of users.  Such groups have their place, but supplement them by interacting with users in their environment, to make sure you get a balanced view across the organisation.  This isn’t something one person does on their own, but something the whole IT department should do (or at least the user facing part).

If you can combine an understanding of how users interact with your systems, with the statistics of traditional KPI’s; then you might have an understanding of how your IT Services are performing.

Whats the point of IT Strategy?

I’ve always struggled with the construction of a meaningful IT Strategy, not because I don’t understand the concept, but because nine times out of ten my employer or client doesn’t.

Every time I’ve written an IT Strategy it’s been a ‘bottom up’ process, with me feeling a bit like a toddler presenting his parents with a carefully prepared masterpiece, only to have them pat me on the head and stick on the fridge with the rest of the project plans, acceptable use policies and service statements.

So why bother?  What’s the point?  Even that question is difficult to answer, as the purpose of an IT Strategy depends on what you think an IT Strategy is.  To avoid this blog post being a very short and fruitless exercise let me start by taking  a stab at what an IT Strategy actually is.

What is an IT Strategy?

An IT Strategy at it’s minimum must contain the following;

1. Refer to the organisation’s high level aims, whether they be it’s mission, core values, strategy or five year plan.
2. Define high level aims for IT Services and Systems which directly support the organisation’s high level aims (see 1), and relate each aim directly to the  organisation aim it’s intended to support.
3. Break down the high level aims down into measurable goals or objectives which you can identify as completed
4. A summary of current state of the organisation’s IT, focusing on where change needs to occur to meet the high level aims and objectives (2 and 3)
5. Define a governance structure which will be responsible for delivering against the aims and objective and reviewing the strategy.

Miss any of these out and you won’t necessarily fail, but you significantly reduce your chances of delivering.  The idea is that the strategy is self defining and self sustaining, with review processes built in to define when objectives are completed and when new objectives are defined.

So, now we’ve defined a generic IT Strategy, what benefits does such a document bring?

The benefits of an IT Strategy

• It ensures your aims are in line with your organisation’s aims, which delivers a number of benefits including buy-in from senior management, quicker approval of project plans and major purchases
• It gives you something to deliver against, without which it’s difficult to get credit or recognition for the hard work the IT department does (IT is now ubiquitous, it’s a commodity like electricity, and it should ‘just work’).
• It’s a valuable planning aid which allows you to direct human resource and finances to the right areas
• It forces your organisation to think strategically about IT

All of these benefits are real, and if you can get your institution to own this strategy (so it’s not the IT Departments Strategy, but the Institution’s IT Strategy which it is responsible for) then this helps prevent a lot of the problems some IT Managers and Directors struggle with (disengagement, distrust, a feeling that it would be a lot easier to just outsource the whole thing).

Are those benefits enough?

Having defined the benefits, unfortunately for many institutions the question might still remain unanswered, or the answer is “there is no point”.  I know from past experience that it is possible deliver excellent IT Services without an IT Strategy ever being committed to paper.  This is the way many very good IT departments operate, and it’s not impossible nor without it’s merit.  In this scenario, where no official strategy has been defined (for the business or for IT) the IT Manager has to ‘read’ his organisation, understand it’s needs, and then deliver the appropriate services.

The advantages of operating without an IT Strategy

It’s a more natural approach than trying to express complex and ever changing concepts to paper, effectively the strategy doesn’t exist on paper, it exists in the minds of the business senior management team, including the IT Manager.  Because it’s not fixed, it’s easier to react to changes in requirements, finances or personnel.

The disadvantages of operating without an IT Strategy

The downsides are that it’s difficult to demonstrate success against objectives, and this approach relies heavily upon the IT team correctly interpreting their organisations needs and acting accordingly.  Defining some basic KPI’s (call wait times, time to respond, time to fix) will allow you to show improvements in service, and getting buy-in from the key people in the organisation to project plans should ensure you remain in harmony with your organisation rather than drifting apart (and suffering the consequences).

Do, or do not.  There is no ‘try’

Trying to force an IT Strategy upon an organisation which isn’t capable of coming up with one is counter-productive, leading  to wasted time and effort producing a document which is redundant as soon as it’s finished.  You have to work within the decision making processes your organisation follows, whether they are highly structured and formalised, or informal and harder to pin down.

In an organisation which expects IT Strategies, where the demand and support for such a document comes from above, and where other departments are doing the same thing (and understand what you’re trying to achieve) it’s a critical business tool, and one you shouldn’t be without.

So if you’re reading this and thinking “I don’t have an IT Strategy, should I have one?”, ask your boss, or your senior management team.  They’ll either look blankly at you (at which point, carry on as you are, but work hard at understanding your organisation’s needs), or say “Don’t you have one?!”, at which point, use the excuse that you’ve never been asked for one, but had started work on a document and were looking for senior management input (which should get you out of hot water).

Never forget, malware costs money

Sometimes it’s easy to become complacent and assume that you’ll never get a virus infection, or if you do dealing with it will be simple. Last weekend I was witness to what can happen when a friend, who I consider a more technically minded and well prepared user, suffered from a serious virus infection.

A common scenario

This friend runs a business and relies heavily on being able to use their PC for work. If they couldn’t use their PC, it could reduce their income in real time, so the PC is critical. The first thing they noticed were searches being redirected to another URL, which aroused suspicion. Following this they found they couldn’t access database services, then they couldn’t update or run their anti-virus products, and finally they couldn’t boot into safe mode.

This last sign was bad news, as it indicated not just malware, but malware which was changing the way Windows itself worked, a kernel mode rootkit. To remove these viruses there are only two options;

• Completely reinstall the operating system, applications and restore the file data from backup (having scanned it for malware)
• Try and detect and remove the rootkit and any associated malware by booting an alternative operating system from trusted media, and mounting the infected volume from there.

This second option isn’t an available to most users, but it can be completed with some measure of success, certainly enough to allow the user to boot their system and either copy data off to portable media ready for a rebuild, or get short term access to the system if they need it urgently.

I’ve used two rescue CD’s  successfully;

• Sophos command line scanner SAV32CLI. Note: you’ll need a Windows boot CD, BartPE or UBCD4Win disk to use this option.
• Avira AntiVir Rescue System. A live boot CD which allows you to update to the latest virus definition before scanning.

There are other rescue CD solutions, I have no experience of the following two but I they should do the trick;

• Kaspersky Rescue Disk
• F-Secure Rescue CD

None of these rescue disks options can be guaranteed to be safe, the risk of an infection remaining undetected is too high, but they do offer a quicker recovery option than completely rebuilding the entire system.

The cost

In total, the user lost at least 7 working hours, and was unable to maximise his income for this period because he lost access to his business systems.  The total cost was probably hundreds of pounds, but could potentially have run into the thousands or tens of thousands, especially if the malware had managed to transfer from the PC to his company website, taking it offline, or infecting site visitors.

The Solution

There are two things all business should do, especially those whose income is directly tied to their ability to access business systems;

1. Protect. Keep your PC operating system and applications up to date by patching them, in particular those applications which are known to be at risk (at the moment that would be web browsers, Java, Adobe Flash, Adobe Reader), and run up to date anti-virus.
2. Mitigate.  Assume at one point you will be infected; backup your data to an external drive, have a backup PC or laptop which is of a sufficient specification to run the applications you need to run your business (not necessarily fast, but well enough), and keep the installation media and license keys for the software you use, so you can setup your backup PC should it be needed

None of the steps above should carry a significant cost, you can often use your old PC or laptop as the backup system, and the installation media for your software should have come in the box when you purchased it.  For the sake of a couple of hours preparatory work you could save yourself days of disruption, the damage caused by lost data, and potentially hundreds or thousands of pounds.

Great expectations

True contentment depends not upon what we have; a tub was large enough for Diogenes, but a world was too little for Alexander.  Charles Caleb Colton

Too often, IT Service fails or is perceived to have failed due to unrealistic expectations.  Expectations (not the financial or technical realities of service provision) are the rule against which your service will be measured.

Too high or too low, both are bad

You could deliver the most reliable, fully functional, user friendly and secure IT services ever; but if the users expectations exceeded that, you perception will still be that you failed.

Conversely, if your users expectations are so low that they never use the bulk of functionality available to them, then again, you have failed.

While for many people this sort of work seems secondary to the actual technical delivery, there’s no point putting all your effort into designing and delivering your service, if you fail to manage expectations along the way.  Managing expectations is not just a PR exercise, it’s not “just spin”, it’s educating your users so that they are better able to engage with the services you’re providing to them.

So how do you approach expectations management?

1. Capture

The first thing to realise is that you will rarely approach a situation where there are NO expectations.  People are so heavily exposed  to technology throughout their lives that there will always be some level of expectation before you start. Therefore, whether you initiating a new project, reviewing an existing service, or just addressing a particular problem, first capture your users expectations. You could do this while capturing requirements, measuring their level of satisfaction with a service, or when getting the description of the problem (as all technical people know, problems often turn out to be poorly set expectations, rather than a fault with the system).

2. Set

Having captured the existing expectations you should look to try and set expectations, the obvious time being when you specify the solution to meet the requirements you’ve already captured.  A dry recital of the technical specifications is not enough here, you need to paint a picture of how the solution will support their business process, relating each feature to a step in the process so they can relate to it and fully understand.  Too often technical IT staff send a specification to people who cannot understand it, then complain later on when they are asked to deviate from it. It’s because the users never understood the specification, because it was effectively written in a different language.

3. Monitor

Throughout an implementation project, at completion and afterwards, you should continue to monitor your users expectations.  Are they being met, or even better exceeded?  Have they changed since the beginning of the project or review?  Any time you see a downturn which you didn’t initiate, or a mismatch developing between their expectations and the solution, as quickly as possible address that.  You can use existing mechanisms to do this, rather than increase your project overheads.  Project review meetings, feedback questionnaires, or responding to specific issues as they arise are all ideal opportunities to set (or reset) expectations.

It’s not easy, which is why it’s important

If you can manage user expectations, from the beginning of any engagement with the users (project, problem or review), you greatly increase your chances of success.  It won’t be easy, it may well be the most difficult part of the project if expectations are way off reality, but that makes managing them even more important, so don’t neglect this process, or be prepared for failure!

Fish are just like buses

Nothing for ages, then two come along at once

I didn’t think today’s session would be up to much; my last two  sessions at Gingerbread Lake turned up just one 8-9lb pike (though it fought well, which bodes well for the start of the traditional Pike season in October), and at about 4:30 I’d run out of bait and time, and I was packing up.

Like many anglers, I always pack up everything in order, leaving the rods until last, but in this case I’d just phoned my wife to say I was packing up, when the left hand rod had a screaming take.  I struck and was into a very lively fish.  After about 5 minutes I was thinking about getting it into the net (the fish wasn’t thinking about that at all), when the right hand rod had a similar screaming take.  Rather than lose the new fish into a snag I turned the baitrunner off, the fish was hooked and I slackened off the clutch just a bit to give it some room to run.

I managed to bully fish number one into the net, and left it there in about 2′ of water while I wound down on the right hand rod to find the fish still there, then after about five minutes got that one into the same net.

When I pulled them onto the unhooking mat, both fish were around 10lb and in great condition.  They fought like fish twice their size, and fell to critically balanced particles on semi-fixed lead bolt rigs.

And it all goes to show, always pack up your kit, landing net second to last, and rods last!