Wednesday 30 January 2008

SLAP - LOCATE Message

The LOCATE message is used by service clients to discover the existence and state of services on the network. Sent as a UDP broadcast it requests information about a specific service.

Upon receipt of a LOCATE, the relevant service is expected to respond immediately by broadcasting an ANNOUNCE message to enable the client to understand the current state of the service.

LOCATE [service name]

Where

service nameA name for the service, well known within the system domain

A typical LOCATE message would look like this.

LOCATE SMSCProxy 

Use of the LOCATE message is optional by the client if the intervals between ANNOUNCE broadcasts by the service are sufficiently small. In many situations though, this delay is unacceptable and the client will initiate the ANNOUNCE broadcast by sending a LOCATE message.

Having both LOCATE and ANNOUNCE messages also has the side benefit of allowing the protocol to have a comedy name.

Tuesday 29 January 2008

The champagne's a bit flat.

In need of somewhere to discuss a bit of strategy after a day with our board, Julian and I thought we'd grab a quick glass of champagne at St Pancras's new, longest in Europe, champagne bar. What a dissappointment.

Far from being the happening destination from which adventures start, it was a cold, stale affair with disinterested staff who directed us the wrong way for a seat.

I've revelled in the transformation of St Pancras into an international hub. It's a shame the social centrepiece hasn't embrace the vision.



Mobile post from the Esendex BlogIt service

Finally we have a sign at our London office

Monday 28 January 2008

Web 2.0 Expo, swapping Berlin for the Bay Area

I've just been booking travel and accomodation for Web 2.0 Expo in San Francisco. I went to the European version last year in Berlin and it was a bit of an eye opener for me. The venue was pretty poor, both location and internal, but the subjects covered struck many chords with me and work I'm engaged in.

I've spent the last few years focussing on the mobile sector, eschewing Internet and developer conferences in favour of mobile specific affairs like Mobile World Congress and Global Messaging Congress. While these conferences have been fascinating and tedious to various degrees they have always been successful in giving me time to reflect and cogitate on our services, how they fit into the mobile ecosystem and what they should become in future.

Web 2.0 Expo in Berlin made me realise that there is a lot of amazing work being carried by the darlings of the Web 2.0 world that have many parallels with the challenges we face at Esendex. By immersing myself back in to the web world as well, there was a lot that I could learn.

As we continue to grow at the phenomenal rate we have been and evolve our services we come across all sorts of scaling and performance challenges. These definitely rank in the 'nice to have' category of problem but they're no less challenging. Hearing about lessons learned by the likes of Flickr et al. can only help us decide how we continue to stay a few steps ahead of demand in the future.

These new services have also upped the ante when it comes to what an Internet service should be able to provide. Their use of AJAX and REST architectures have provided a rich user experience far beyond where, to be brutally honest, we are right at the moment.

That is changing, we have some exciting new services coming to our customers over the first half of this year that bring us bang up to date. We've spent so much time focusing on making sure our messages get through in a timely fashion we've probably taken our eye off the ball on the front end usability. So there has been a big focus in the development team on making our services richer and easier to use. Obviously I'll fill you in as soon as I can.

I suspect that the Berlin conference did suffer from being smaller than it's American parent, so I'm looking forward to seeing how it works on a bigger scale. There are also worse places to go than San Francisco, not least to get my fix of unbridled, Bay-Side, can-be-done-ness.

Sunday 27 January 2008

I am having an iPhone love in moment.

I am having an iPhone love in moment. Watching a DVD at home, like the look of a couple of the trailers, browsed to lovefilm.com and queued them up. I'd never have bothered firing up my laptop. Immediate Internet in my hand, marvellous.



Mobile post from the Esendex BlogIt service

Friday 25 January 2008

Is mobile blogging the answer to blog torpor?

As regular readers will know, I had a bit of a blogging hiatus through December. Truth is I just got out of the habit. I got busy and my blog was the balancing item.

I suspect that this is the case for the vast majority of blogs out there. We start with good intentions but soon the next interesting thing comes along and today's brilliant, can't live without, new toy becomes yesterday's forgotten plaything.

Services like Twitter are designed to disseminate transitory information. I have signed up for an account but I really can't imagine anyone would find my random thoughts of interest, and anyway I can't really be bothered. The good thing about Twitter is if I say something that I subsequently wish I hadn't it doesn't hang around. It's forgotten in the sea of titbits about everyone's lives.

Blogs however are a different story. With their archives and syndication, once you've hit publish, that's it. This certainly makes me consider everything I'm posting and perhaps that's constraining the content and timeliness.

One of the problems with timeliness is access to a computer to make posting possible. By the time you get home, there are umpteen other things to catch up on before writing up the blog post you thought about while on the train, in the car, out at lunch.

In the past I've used my BlackBerry for mobile blogging, sending an email to this blog for editing later. When I remember to do it, it works pretty well, though I end up having to reformat and strip off the rather long (thanks to legislation) company email signature. We've recently launched BlogIt here at Esendex so people without email devices can do the same using just SMS.

While these services help to enable the physical act of blogging as and when the mood strikes I also think they require a bit of an attitude shift. I have to remember that every blog post doesn't have to be a long considered essay, it's perfectly legitimate to post something up as when the feeling takes me.

I probably should be championing mobile blogging via SMS as the answer with us running the BlogIt service, but in reality I'll use a mix of both. It'll depend on the situation at the time, what device I have with me, how long the post is, and whether I want to give it a bit of thought before posting.

The key point is that mobile blogging, be it by SMS or mobile email, gives me that choice. For something to become a habit, it needs to be easily assimilated into your everyday life. I spend a lot of time emailing and texting while on the move so now I can blog just as easily.

Expect to hear more from me.

iPhone is a bit player

Very good article by Ewan at SMSTextNews on why he thinks the iPhone is revolutionary but it hasn't really changed anything and won't do until there's a strategy change from Jobs & Co.

The Apple iPhone will only ever be a bit player. What’s next?

I'm inclined to agree with him. I love the phone, but I'm definitely in the ultra-geek category. 3G support and a lower price point should see them fly of the shelves though.

SLAP - ANNOUNCE Message

The ANNOUNCE message is the core message in SLAP. Sent as a UDP broadcast, it announces information about a service to the network. It's purpose is to inform interested clients about the current state of the service.

ANNOUNCE [service name] [state (Green|Amber|Red|Blue)] [service uri]
<[optional attributes (name=value)]>

Where

service nameA name for the service, well known within the system domain
stateThe current state of the service as represented by one of 4 values Green, Amber, Red or Blue, see below
service uriUnique network address for the service. Can be any format that the client would recognise.
optional attributesAn optional series of attribute value pairs seperated with a new line.

Service States

Service states are used in SLAP to give a simple indication as to the service state.

StateDescription
GreenService is fully operational and available
AmberService is operational but under load
RedService is not operational
BlueService state is unknown

If more information is required, then the service would include this with the optional attirbutes of the message. The content of these atttributes is not defined and is open to individual system implementations.

For example, information on current service performance (transations per second, queue size, memory utilisation, etc) could be included to allow the client to make utilisation decisions.

A typical ANNOUNCE message would look like this.

ANNOUNCE SMSCProxy Green node1.company.com:1234
Route=Operator1
Binds=3

Generally these messages would be broadcast at regular intervals to keep clients updated with service information. There are times when a client needs to actively interrogate the state of a service, for instance on startup, for which the LOCATE message is defined. More of that in the next post on SLAP.

Monday 21 January 2008

SLAP, an introduction

Part of the reason I've been so quite on the blogging front of late is that I've got carried away with a bit of development spiking. Nothing that goes straight into production I hasten to add. I'm too far away from our development processes to be trusted with access to the live source tree, and anyway Nicholas (Development Manager) probably wouldn't let me.

I am in the enviable position of being able to create proptotypes, test things out and hand them over to the professionals to make them a reality. It's one of these projects that became SLAP (Service Location and Announcement Protocol).

One of the key technical challenges facing an organisation like ours is scaling our applications out to meet demand while at the same time keeping everything highly available.

Our architecure comprises a series of software agents that pass messages between each other routing and manipulating traffic based on a series of rules. For example

  • A outbound bulk SMS will head straight to an SMS Router for it to decide based on the state of a set of Mobile Carrier connections which is the best current route.
  • A premimum SMS message will be passed to a Subscription checking agent to confirm that the subscriber is valid, and then possibly onto a network lookup agent to resolve the destination network before heading onto the SMS Router for onward submission via the correct network for billing purposes.

As I'm sure you can imagine running multiple instances of these agents across multiple servers and have them know the state of the agents around them so messages get passed around reliably is no mean feat. Should an Mobile Operator connection become unavailable or we lose a server, the agents should be able to adapt to that.

One option is to use hardware load balancers, as we do in front of our web servers, but this quickly becomes very expensive and very costly to maintain as you end up with multiple layers of load balancing. Also it's very hard to dynamically configure the network of agents to cope with peak periods of traffic.

After reading Scalable Internet Architectures I was inspired to come up with a simpler, and cheaper, solution. SLAP was born which is an alternative, software approach that is deliverying huge benefits to us already.

Nodes broadcast one of 2 message types: ANNOUNCE and LOCATE over UDP which are subscribed to by other nodes on the network. These nodes can be application monitoring agents or clients wishing to consume services offered by the node.

It's simplicity hides a power that has allowed us to architect our system to self configure and repair itself in the event of failures. In subsequent posts I'll describe the specifics of the protocol and our describe our experiences using it.

Friday 18 January 2008

Still loving my iPhone

I'm pretty notorious for having a very short attention span. I'll jump passionately in to a new service or toy with both feet, love it for a couple of weeks until something else takes my fancy. Julian, as you can see from his derisory comments on : iPhone - Smartphone for the Normob?, has been of the opinion that my iPhone purchase would go the same way.

Well I'm still loving it and the latest updates announced at a MacWorld have made it better. It really is a device that fits into my personal life.

Yes the camera's not great and the data coverage can be irritating but it's so well thought out and put together that I still keep coming back for more. I've even got used to the keyboard.

This year will see a slew of copies from Nokia, LG, Samsung, et al and it'll be really interesting to see how these manifest themselves. I think the challenge for them will be to not try make it backwardly compatible, interface wise, with the rest of their estate.

One of the key reasons the iPhone is such a success is that Apple have taken decisions about how you want to use the device on your behalf. Their team of UI designers have worked out the best way for it to work so you don't have to.

The temptation when developing software is to give user full control, lots of features that they can take advantage of. You never can tell what a user might want to do so make sure you've got your bases covered. The danger with this approach is that you end up with a sea of options that confuse the user rather than empower them.

I liken this to the multiple camera options you get (at least I assume you still do) with Sky Sports coverage. So you can watch Wayne Rooney scratch is rear or John Terry yelling at his team mates while the rest of the game is going on.

This kind of service doesn't appeal to me at all. Why should I make decisions about the best angle when a highly paid (assumption) highly experienced sports TV producer is there to make the decisions for me and make sure I don't miss the best bits.

The iPhone has been a liberating experience for me. Usually with a new device I'm desperate to dive in and configure, load things, try things out and I just end up getting frustrated and ultimately fall out of love with it. Apple's designers have prevented me from doing that while making some great decisions about how the iPhone should be used.

This time I'm using the device in the way it was designed and our relationship is stronger than ever.