Tuesday, 9 March 2010

The Future of Web Services / The Plight of the Infrastructure Techie

The landscape for Internet Infrastructure is changing, and it may be quite scary for techies..

Currently, we have quite a supply-chain in web service hosting. We do everything ourselves and that means lots of skill-sets in various places such as:

Linux/Windows SysAdmin
Networking SysAdmin
HA/Load-Balancing Specialists
DBAs
Developers
Management

Typically, management instruct developers to do something and this something needs to be consulted on from conception to delivery. The developer needs to make it in such a way that the DBA and SysAdmins are happy that it will scale and perform. If it’s business critical the HA people get involved and ensure development is geared around being resilient too. The whole thing creates an infrastructure ecosystem of staff because it’s all very DIY so lots of diverse skills are required.

What happens when all of that is someone elses problem, without the disadvantage of outsourcing costs?

Using Cloud Computing you can build web applications to be self-aware. All of the nasty stuff that you used to have to worry about is gone:

Scaling – Amazon Elastic Compute lets you boot loads of instances of your pre-configured application OS images (read-only) manually or automatically. Because the booting of extra instances is web service controlled your app can decide if it needs more power or not.

Load-balancing – Single IP endpoints now control this. No more multiple servers with non-ARPing interfaces and virtual IPs. Amazon Elastic Load Balancing distributes load depending on instance issues, load metrics etc. OR you can plug it into Amazon Cloudwatch which monitors certain metrics and load-balance depending on that.

Hardware – It’s all virtual innit? It’s no longer our problem.

Database Scaling and Capacity – Apps just abstract DB API’s these days anyway. All apps ever do is ask the connector to pull data out into an object or insert data in, or do stuff with the data that’s there. If all you ever see is an interface into a limitless space of data then all you have to worry about is what you do with the data. Amazon SimpleDB does this for you.

Shared data storage – Amazon Elastic Block Storage behave like a SAN where you can have data sat on a filesystem to be accessed by apps on multiple instances. Again, any problems with this are someone else’s. All I care about is accessing my data.

Data mining – If you have lots of data to interrogate you can use Amazon MapReduce to process it.

There are lots of other ways you can use elastic cloud computing to remove a LOT of current infrastructure costs and concerns, eventually leading you to just have to worry about the application. Once you get to that stage you can just concentrate on making it work and doing it right. For a business, it’s a non-decision – or at least it should be. Without all the associated costs of infrastructure and infrastructure people a business can save obscene amounts of money and be a lot more efficient to boot. So why aren’t they?

Well, businesses are just scared of anything new. Elastic computing has to continue to innovate and provide something all-in-one and pre-packaged that it makes a lot more sense (not just a bit more sense) to migrate. Also business don’t like the idea of their data being somewhere they can’t control. That mindset just has to change, and it will as more and more large enterprises get on the cloud application bandwagon and start using tools such as Google Calendar / Mail / Contacts for company business. SLA’s give some peace of mind to directors but not quite yet enough.

But once the scales start tipping, the only people safe are developers.