Returning to Windows: Part I

This series will run through how I've moved from being a full-time Linux desktop user to using Windows full-time. First up, a little about me.

I've been in the IT industry professionally for about 15 years, but I've always had something to do with computers. I started at around 7 learning BASIC and writing a few programs on my Sony Hit-Bit MSX.

(MSX is a whole other topic for another post, so I won't go into detail here.) 

After that I worked in Technical Support for a UK computer manufacturer before moving into R&D and finally into Linux system administration. During my time in R&D I had a lot to do with Microsoft and Windows in particular, developing PC builds and configurations around Windows from ME to Media Centre. I got to know them very well and didn't like the way either Microsoft, or Windows, worked.

So I changed my home desktop computers to Linux. Mandrake Linux at first as it was extremely user-friendly and attractive. It may look a little dated now, but against Windows 98 it was amazing.

I then moved onto Ubuntu as of Breezy Badger (5.10, released April 2005) and continued to make my protest against the Microsoft wheel corruption racket that I'd experienced when dealing with them.

So fast forward until now. Ubuntu was fantastic at the start, it promised so much, but as of Natty Narwhal (11.04) it's delivered so little. When I first started I always needed a decent video editor for my family videos. KDEnlive, Cinellera and later PiTiVi were video editors which were always halfway there, threatening to become the all-purpose easy editing suite that Windows Movie Maker had become. But 5 years later, it hasn't happened for one reason or another. A few weeks ago I simply couldn't hold out any longer, I installed Windows 7 on my main computer, stopped being a martyr and took the easy life again. My experiences since then have been mixed, but now I'm in the position where I can provide the fairly rare insight of an experienced Linux user discovering the pitfalls of being a newbie Windows user. In all honesty, I know what to expect, it's not that different to where I left it, but I still have a fresh view on most of it.

The next part of this series will be how the installation differed to what I'm used to. How easy is Windows 7 to set up and get ready to use compared to Ubuntu?

On the Origin of Cars and Human Beings

From my moderately recent uptake of running to attempt a Marathon in October it's struck me just how incredibly close the mechanics of motor vehicle engines are to the mechanics of Humans. A Heart Surgeon recently described to me that opening up the chest cavity and looking inside as "it's just mechanics". That got me thinking about just how right he is. Perhaps it's no accident that the two are so close in the way they work either; we've now had just over 100 years' of enhancement and refinement of car engines, so the natural process should follow the best formula for working with physics, proving that evolution is the ultimately the best judge. I'll try and explain how I think the two things match each other, and how they differ. By comparing them we can actually have a good guess about how cars will 'evolve' in the future, by looking at how the Human Body is better than an internal combustion engine.

Combustion
When exercising, you draw air into your lungs to react with food in your stomach, to be carried around the blood stream to your muscles. To help this process work, you need water. Without water you effectively dry up and grind to a halt. It's very easy to perceive this happening when you're dehydrating while exercising; you feel muscles tighten, your blood thickens and saliva turns very thick. This is very similar to how an engine will draw air in to the combustion chambers (lungs), combine the air with fuel (food) and use the chemical reaction to create energy. Also very similar is how lubrication is required. The engine requires oil to lubricate the workings otherwise it will dry up in the same way as your muscles and joints will dry up without water.

So the way the car engine produces energy is very similar to how we produce energy. What else is similar?

Tuning
When trying to get more performance out of your engine, it's a simple principle. More air and fuel in equals more energy generated. It's also remarkably similar with the human body. You exercise to increase your aerobic and anaerobic threshold by increasing the size of the energy pathways to your muscles. You exercise to make your heart and lungs more efficient to be able to make better use of the oxygen you can draw in and more efficient at using the energy. This is the equivalent of boring out your engine by making the cylinders bigger and holding more air, and increasing your engine's volumetric efficiency by allowing as much air to be drawn in on every stroke (breath) as possible by porting, polishing and general head work.
You can increase the fuel pump size, the fuel line size, the fuel regulator and the injectors / carburettor. This is the same principle as the size of your veins increasing to allow more fuel for your muscles to be carried. Most tuners will be familiar with the term 'Italian Tune-up' where giving an engine a blast will remove old deposits and scale and effectively allow your engine to perform better. The exact same thing happens with your body.

The Future
I firmly believe that any advances in the technology of the internal combustion engine have so closely imitated the mechanics of the human body that it's a fairly easy conclusion to come to that future advances will go down the same route. Where can we expect to see engine technology go next, if it does mimic human biology?

Well, the body's ability to self-heal and adapt to load is a great advantage. It makes it able to last, literally, a lifetime. As you ask your body to do more and more demanding things, your physique and attributes change to allow greater abilities at these tasks. Some modern cars have the ability to change the air and fuelling pattern depending on how it learns your driving style, so we can see some of that already. Even some technologies such as variable valve timing are also evidence that this is starting to happen, so I think we'll see engines able to adapt better to different driving conditions, demands and styles in future. We may also see more advancements in the way that engines are able to heal themselves by making use of modern materials.

It's no surprise that engines and the human body closely model each other, as both are intended to do the same thing; convert chemical energy into kinetic energy and deal with any associated wear and tear in the course of things. The human body has had quite a head start, but we're pushing transportation devices forward faster than evolution could manage, to the point where we could very well merge.

Can you see this trend continuing? Where will it go?

Making BASH Scripts HA Compatible - Daemonising BASH

Quite a few times on our clusters we've needed to make a cron job, or a shell script HA compatible. We'd like the cluster to be able to start and stop it, so it can failover with other resources if required.


It's actually a lot easier than it seems. The easiest way is making a while true loop with a sleep in the middle, then in each iteration check the current time against the run time of the script. It's kind of replacing cron, but needs must.


This is how I did it.

Stage 1 - Make a standard LSB compatible init script carcass


#!/bin/sh                                                
# description: Start or stop your res name
#                                                        
### BEGIN INIT INFO                                      
# Provides: your_res_name                     
# Required-Start: $network $syslog                       
# Required-Stop: $network                                
# Default-Start: 3                                       
# Default-Stop: 0                                        
# Description: Start or stop your res name      
### END INIT INFO    

RUNFILE="/var/run/your_res_name"


NAME="YourResName"
case "$1" in
'start')
        CHECKSTATUS
        [ "$RUNNING" ] && echo "$0 is already running" && exit 0
        echo $"Starting $0"
        touch $RUNFILE
        MAINLOOP &
        ;;
'stop')
        [ -f "$RUNFILE" ] && rm $RUNFILE
        pkill -f "$NAME "
        echo "$NAME"
        ;;
'restart')
        $0 stop
        sleep 5
        $0 start
        ;;
'status')
        CHECKSTATUS
        [ "$RUNNING" ] && echo "$NAME is running" && exit 0 || echo "$NAME is stopped" && exit 3;;
*)
        echo
        echo $"Usage: $0 {start|stop}"
        echo
        exit 1;;
esac

There's no CHECKSTATUS or MAINLOOP functions yet, we need to add those next.

MAINLOOP


You need a while true ; do ; done loop to sit there running through the stuff you want to check and then to do stuff at the appropriate times.


RUNTIME="000300" # 00:30:00
MAINLOOP() {


LOG="/var/log/$NAME.log"


while true
do
# Check for permission to run.
[ ! -f "$RUNFILE" ] && exit 0


# Check if we've already run today
if [ ! -f "$OUTPUTFILE" ]
then
# Or if we're still running
NUMPROCS=`pgrep -f "$NAME " | wc -l`
                if [ $NUMPROCS -lt 1 ]
                then
THETIME=`date +%H%M%S` # Get a numerically comparable time. 
if [ $THETIME -gt $RUNTIME ]
then
echo -e "\nApparently $THETIME is greater than $RUNTIME so it's time to do our thang" >> $LOG
echo -e "------------------------" >> $LOG
echo -e "\n*** Starting process.***\nThe time : $THETIME" >> $LOG
echo -e "\nNumber of existing processes : $NUMPROCS" >> $LOG
echo -e "\nLet's GO!\n" >> $LOG
RUN_OUTPUT
fi
fi
fi
sleep 1m
done
}


CHECKSTATUS


You can just use something simple like check for the run file and the background process running on this



CHECKSTATUS () {


if [ -f "$RUNFILE" ]
then
if [ `pgrep -f "mi_data_extract start" | wc -l` -gt 0 ]
then
RUNNING="yes"
else
unset $RUNNING
fi
fi


}


That's pretty much it. Check each operation of the init script with echoing out the return code from every state. e.g.


/etc/init.d/your_res start ; echo $? # from stopped, should be 0
/etc/init.d/your_res start ; echo $? # from started, should be 0
/etc/init.d/your_res stop ; echo $? # from started, should be 0
/etc/init.d/your_res stop ; echo $? # from stopped, should be 0
/etc/init.d/your_res status ; echo $? # from started, should be 0
/etc/init.d/your_res status ; echo $? # from stopped, should be 3


Once this is done, you can just add it to the cluster as a primitive LSB resource, add a monitor on it and let the cluster take care of your script.

Time For Ubuntu to Fork Evolution

No one can deny the current face of Linux to the masses is Ubuntu. It’s massively more popular than any other distro which makes it the flagship for breaking existing market strangleholds.

Take the Enterprise server OS market for instance, a traditionally strong area for Linux anyway, Canonical (the controlling company of Ubuntu) have rightly seen where they need to position themselves to gain the advantage with Server OS’s and have gone down the Cloud route with Ubuntu Enterprise Cloud. Also - beefing up the support options and the packaging to at least align themselves with the normal market leaders Suse and Red Hat helps to gain further server adoption by to using the momentum of all the other Ubuntu areas and user allegiances.

The personal desktop / netbook area is the next to be tackled. Obviously Ubuntu has been trundling along as the best choice for the tiny personal Linux desktop market for a while but it has needed to really stand-out to do battle with Windows and the latest player (rising on the back of the i[Pod|Phone] wave) Mac OS. Again Canonical have pulled the rabbit out of the hat and pointed Ubuntu desktop in exactly the right direction – Social Networking. With Ubuntu Lucid having fully integrated Social Networking and chat they’ve shown they know how people actually use their computers. 9 times out of 10 someone is turning their computer on to participate in Facebook or make Tweets on Twitter, or for the Old-Skoolers chat on MSN. To make the desktop OS actually part of this is exactly the best way to position it and ensures it’s already ahead of the opposition when they realise they need to do the same thing.

Finally, there’s an area that Ubuntu is very weak on and it’s where efforts need to be concentrated next - The Enterprise Desktop.
Novell have previously tried to leverage that market but did it all wrong. They didn’t understand that there is just one killer feature (just as with integrated desktop social networking) that needs to be in there which is Exchange support. Outlook and to a lesser extent Office keeps Windows XP / 7 firmly planted on the Enterprise desktop purely because of its ability to work perfectly with Exchange. Businesses now (rightly or wrongly) revolve around shared mail, contacts, calendaring and scheduling, and Exchange shows no signs of being supplanted yet as the default choice for this functionaility.

With all this in mind I present my recommendation for Ubuntu: Fork Evolution.

Evolution has some good MAPI functionality but for every step forward, it takes 2 steps back. The functionality is very buggy but at the same time is almost there and some real concerted structured development would see it work very nicely and be a drop in replacement for Outlook.
Also, Evolution needs all of this Social Networking goodness that is present in the me menu in Lucid to be integrated into Evolution too. One place for all messaging/contacts/calendaring with Social Networking in there too and we’re getting very close to a framework that supports the multitude of communication mediums we use today.

So in my mind, Ubuntu and Canonical can move one major step forward by leveraging Evolution. Make it fit the new Ubuntu desktop ethos better and make it work properly with Exchange. Once you do that, world domination for FOSS will follow.

The Future of Web Services / The Plight of the Infrastructure Techie

The landscape for Internet Infrastructure is changing, and it may be quite scary for techies..

Currently, we have quite a supply-chain in web service hosting. We do everything ourselves and that means lots of skill-sets in various places such as:

Linux/Windows SysAdmin
Networking SysAdmin
HA/Load-Balancing Specialists
DBAs
Developers
Management

Typically, management instruct developers to do something and this something needs to be consulted on from conception to delivery. The developer needs to make it in such a way that the DBA and SysAdmins are happy that it will scale and perform. If it’s business critical the HA people get involved and ensure development is geared around being resilient too. The whole thing creates an infrastructure ecosystem of staff because it’s all very DIY so lots of diverse skills are required.

What happens when all of that is someone elses problem, without the disadvantage of outsourcing costs?

Using Cloud Computing you can build web applications to be self-aware. All of the nasty stuff that you used to have to worry about is gone:

Scaling – Amazon Elastic Compute lets you boot loads of instances of your pre-configured application OS images (read-only) manually or automatically. Because the booting of extra instances is web service controlled your app can decide if it needs more power or not.

Load-balancing – Single IP endpoints now control this. No more multiple servers with non-ARPing interfaces and virtual IPs. Amazon Elastic Load Balancing distributes load depending on instance issues, load metrics etc. OR you can plug it into Amazon Cloudwatch which monitors certain metrics and load-balance depending on that.

Hardware – It’s all virtual innit? It’s no longer our problem.

Database Scaling and Capacity – Apps just abstract DB API’s these days anyway. All apps ever do is ask the connector to pull data out into an object or insert data in, or do stuff with the data that’s there. If all you ever see is an interface into a limitless space of data then all you have to worry about is what you do with the data. Amazon SimpleDB does this for you.

Shared data storage – Amazon Elastic Block Storage behave like a SAN where you can have data sat on a filesystem to be accessed by apps on multiple instances. Again, any problems with this are someone else’s. All I care about is accessing my data.

Data mining – If you have lots of data to interrogate you can use Amazon MapReduce to process it.

There are lots of other ways you can use elastic cloud computing to remove a LOT of current infrastructure costs and concerns, eventually leading you to just have to worry about the application. Once you get to that stage you can just concentrate on making it work and doing it right. For a business, it’s a non-decision – or at least it should be. Without all the associated costs of infrastructure and infrastructure people a business can save obscene amounts of money and be a lot more efficient to boot. So why aren’t they?

Well, businesses are just scared of anything new. Elastic computing has to continue to innovate and provide something all-in-one and pre-packaged that it makes a lot more sense (not just a bit more sense) to migrate. Also business don’t like the idea of their data being somewhere they can’t control. That mindset just has to change, and it will as more and more large enterprises get on the cloud application bandwagon and start using tools such as Google Calendar / Mail / Contacts for company business. SLA’s give some peace of mind to directors but not quite yet enough.

But once the scales start tipping, the only people safe are developers.

Printing, And How It's Like Cars

Printers are like the car industry. Stay with me. I'll explain.

Back in the day, printers were tough as old boots. They supported either Postscript or PCL. That was fine, there were 2 standards and most printers supported either.

Then, along came Microsoft with WDM and the ability to use drivers to offload the printers processing into software. Suddenly people like Epson could build cheap mass-produced shit like the Stylus C20 USB printers etc. that didn't have a hope of supporting a standard like PCL or PS. 1-0 to Microsoft, as with their WDM drivers for modems, they played their market advantage to ensure that only people running Windows could use software printers (winprinters).

HP were caught a little on the hop. They suddenly had to play catchup and started doing this winprinter rubbish too. They did, but at the expense of their quality and their reputation in the marketplace. Eventually HP came back around to supporting PCL and Postscript in almost all of their printers and now they're back on top.

Compare to the car industry in the UK - Ford and Vauxhall have always been the mainstream cheap brands, they got good at shaving costs without compromising too much in reliability. Suddenly people like VAG, BMW etc. had to lower prices to compete but they weren't used to doing things on the cheap so the stuff they produced was often poor quality in the late 90's / early 2000's and the Ford and Vauxhall stuff was actually more reliable. It's only today that the higher marques have been able to compete on price and also quality with the cheaper stuff.

'til next time.

Apple are scared of Linux

It's the only explanation I can think of when trying to work out why, after many years of Linux gaining ordinary user popularity, Apple continue to refuse to allow Linux users to use iPods.

I'm a Linux user. I can't afford a Mac (and TBH don't really see the need, it's Linux with all the disadvantages but none of the advantages) and I don't want Windows. I use Kubuntu on my work laptop, Kubuntu on my home desktop and Ubuntu on my old home laptop. If I got given an iPod, or I had an iPhone I couldn't register it. Sure I could take it somewhere with a Windows PC and plug it in to register it but that's not the easiest task for me these days. I don't really have access to any Windows PCs where I can use the Internet and plug random things into them.

After that how do I sync my music? You have to use iTunes and there's no Linux version, even though Mac OS X is based on Unix and Apple obviously had to spend a lot of time and money on making a Windows version. Surely it wouldn't be that difficult to make a Linux version of iTunes?

The answer to all this is simple - Apple see Linux and probably more specifically Ubuntu as a direct competitor to the traction Mac OS X is getting against Windows. Windows is dropping market share and currently Apple is using all the marketing it can to ensure it picks up the slack. With Ubuntu available for free on standard PC hardware and with arguably more power Apple should be worried. But while Apple continue to take this stance against Linux I'm afraid I can't use any of their products. They exclude me. Android doesn't...