Peter Williams

Sprinklers

I have been battling the sprinkler system in my front yard for a while now. I spend a good part of last weekend working on it only to have completion of the project ripped from my grasp by a burst pipe (and my lack of competence in sprinkler systems). Today, however, I am triumphant.

Here is picture of my new found enemies unconditional surrender.

(That is the replacement for the section of burst pipe that was such a problem. Notice how it is not leaking. It’s the small things.)

The back story

In 2006 we had a walk added to our front yard. It added some nice visual interest and has been great. But it ended up covering one of the sprinkler heads. No big deal, we just turned off the front sprinklers. We had – and still do have, for that matter – big plans for xerascaping our front yard. We live in a semi-arid climate and have no need for a lush carpet of grass in the front yard.

A year and a half later we have still not finished the front yard. The grass is looking really bad this spring, so we decided to fix the sprinklers. I like DYI projects – they are a hobby of mine – so last weekend I jumped in with gusto.

Everything was going great… I found the main line I need to cap to disable the heads under the walk. I installed a new head for some new small sprinklers in a flower bed. I removed another head whose placement was just dumb. Then I turn the sprinkler on and it worked. Well, except for the water spraying out of the ground in the middle of the yard.

It seems that in the intervening two winters the sprinkler line burst. So I dug the appropriate hole, cut out the busted pipe, and attempted to install a new section. Several times. And then I had to give up for the day because it was getting dark. Today, I started again, and as they say, the rest is history.

My Brother Has a Blog (Finally)

Cat and I have been gently suggesting that my brother should get a blog for years. Now he has finally done it. I am not sure what the personality¹ of his blog will turn out to be, but I am looking forward to finding out. He is smart – and a better photographer than I knew – so it should be good.

Want a Job?

As you have probably noticed, I recently started a new job. Which means that I also recently left a job.

The job I left was as at Absolute Performance, and it was a pretty good gig. The good news is that my leaving means that there is a spot for you. If you are interested in working on some cool Ruby, Java and C++ code with a really great team you should send them a resume. Oh, and don’t forget to tell them I sent you, maybe they will buy me a lunch or something.

Java and Scalability

Every time I hear someone say that Java is “scalable” my initial reaction is to kick the person who said it in the shin.

I have been talking to a lot of people lately about the tools we are using at Gnip. Every time I tell someone that major parts of our system are written in Java the response seems to be, “Oh, for it’s scaling capability?” While I was safely ensconced in the Ruby world I had hope that this malformed meme was dead. It seems that in the wider world it’s not quite dead.

I never actually kick the person, by the way. Instead I just sigh and explain that, no that is not the reason. Scalability cannot be the reason we use Java, because Java does not scale any better, or worse, than any other general purpose language.

There are a variety of different sorts of scalability. The most interesting type of scaling in the context of web applications, like Gnip, is how easily can you increase the number of requests/sec the system can handle. This sort of scalability, or lack thereof, derives pretty much entirely from the architecture of the system. No language will magically make your system be able, or unable, to handle an order of magnitude increase in the number of requests.

The culture¹ of Java actually encourages the development of mediumly, rather than highly, scalable systems. It does this by favoring the use of multi-threading, shared state, vertical scaling and large monolithic components. These techniques do not scale infinitely. Fortunately, Java is fast enough that they can scale to quite significant levels. Even though the culture of Java encourages these less than perfectly scalable techniques you can build highly scalable systems with Java quite readily. You just have to be willing to buck the culture when it is appropriate.

Performance, on the other hand, does derived, to a significant degree, from your language,² and that is why we use Java.

HTTP and XMPP

My new boss is contemplating whether or not HTTP will remain the protocol of choice in the future. He seems to have reached the conclusion that XMPP is a better protocol than HTTP for the network infrastructure we have today.

With today’s connection characteristics, I wonder if HTTP would have been the weapon of choice 15-20 years ago? I doubt it.

Based on this conclusion Jud appears to believe that XMPP will replace HTTP in the future as the protocol of choice. I disagree with Jud on both points.

The Internet is much more reliable today than it has ever been before. It is so good, in fact, that there are many situations where you can trust the network these days. This is particularly true high levels of reliability are not required. However, the network is still not perfect, nor is it likely to ever be. Worse yet, the software that uses the network is still depressingly flaky.

More importantly, I don’t think HTTP “won” because of problems with the network. HTTP is ubiquitous today because it facilitates a programming model that can the use to solve some really hard problems reasonably easily. That programming model an implementation of the REST architectural style. The constraints of REST allow building highly scalable and reliable systems far more easily than any other approach available today.

In his post Jud speaks of HTTP as if it where transport protocol. As a transport HTTP is not terribly compelling. It has fairly high overhead for individual requests. It’s connection model usually ends up creating more TCP connections absolutely necessary. It allows a lot of variability in the capabilities of clients. And so on.

However, HTTP is decidedly not a transport protocol. It is an application protocol. HTTP provides a sophisticated set of semantics specifically designed to facilitate the implementation, and optimization, of REST style applications.

XMPP is, on the other hand, is a transport protocol (unlike HTTP). To be precise it is a (near) real-time message transport protocol. If you need that XMPP is an excellent choice. Particularly if the messages you a dealing with have a limited duration of meaningfulness. For example, if your application loses it’s connectivity to the message sender for any significant period of time it is likely that the application will not receive at least some of the messages sent via XMPP during that time. The server may spool some messages but completely unreasonable to expect APIs to keep track of an arbitrary number of undelivered messages for an arbitrary number of clients. The cost of doing that are just too high for a high volume producer to be able to implement.

I think XMPP will continue to get more penetration and mind share. It is a good protocol. It is not a competitor to HTTP, though. The two protocols serve very different purposes. I expect that many systems will utilized both. If you have a need for real-time messaging and you have relatively weak reliability requirements, or you are willing and able to invest significant effort implementing reliability in your application, use XMPP. But real-time messaging do not an application make.

HTTP is not a “dinosaur”, as Jud puts it, it is a shocking advanced piece of alien technology we have only recently discovered (as an industry) how to fully utilize. Actually, I am pretty sure we have not yet figured out how to fully utilized it. We will continue to see more and more applications and data service APIs implemented using the wicked cool semantics of REST/HTTP.