Elements or Lower

Mon, 12 Dec 2005

Jon Stewart Live

The missus and I had the immense pleasure of being able to attend the 4PM performance of Jon Stewart (and two writers from The Daily Show) giving selected readings from America: A Citizen’s Guide to Democracy Inaction.

“The theatre would like me to announce that the role of Mary Poppins in tonight’s performance will be played by three Jews reading from a book.”

The experience taught me a number of things:

  1. There are an awful lot of Americans living in London — and on Sunday afternoon, they were all in a theatre in Soho.

  2. Unlike many of his fans, Jon Stewart really doesn’t see his appearance on Crossfire as his finest moment.

  3. Audience members who, in a question-and-answer session, attempt to be funny and fail, deserve everything they get.

  4. Nothing says “Christmas treat” like Maltesers from a box.

  5. The Daily Show is motivated, not out of disdain for America and American values, but out of love for them. These guys are genuinely disappointed that it’s all gone so horribly wrong.

Incidentally, isn’t it always the way that when you encounter someone whose work you admire, they’re in the gents taking a leak, effectively precluding any hand-shaking opportunity for fear of (a) cooties, and (b) it being misconstrued as an attempt at a hand-shandy opportunity?

Mon, 05 Dec 2005

The PageCache

I’ve previously noted that the CMS I’ve put together uses a fried rather than baked model for its presentation layer. For a few months now, this has only partly been true.

The presentation layer now implements a cache for the final HTML of pages, and serves from that when there’s a copy of the requested page there. The advantage of this is an acceleration in the delivery of cached pages, and a reduction in the amount of redundant work the CMS has to do, especially for popular pages.

The cache is a regular database table, containing a resource ID, the “framework” (viewing context), and the actual generated HTML. When a request is made for a page, once the CMS has analysed the URL to establish the resource ID and framework in question, it checks to see if there’s matching content in the cache. If there is, it serves it; if not, it proceeds to generate the content as normal.

This is different from a normal baked CMS, only in that the pre-generated content is effectively served from a database rather than as static files on the server, and that the CMS continues to fry-up content if there isn’t anything already baked. By doing this, and by binding the cache to the CMS at a fairly fundamental level, we can achieve quite a lot of flexibility.

For a typical page, if there isn’t any pre-generated content, the CMS will generate the page as normal, and then try to store the final HTML in the database for the next request. The CMS won’t actually permit the storage of pages, however, in the following circumstances:

  1. The page is a PDF file.

  2. The page is being viewed in the test environment (the CMS can have separate test and live content for any page).

  3. The request contained a query string or POST content.

  4. The content is marked as do-not-cache.

All that happens here is that the generated content is never stored, and so the CMS will be forced to re-generate it for each identical request. The initial check for pre-generated content, therefore, trusts the database completely. If there’s pre-generated content available, it’s always served. This keeps the processing overhead to serving that content to an absolute minimum.

The database, however, is deliberately very fragile. The CMS wipes the entire cache overnight, to avoid any pages becoming stale, and the administration layer wipes selected portions of the cache when amendments are made. Generally, changing the textual content of a page means that the CMS only need wipe the cache for that page alone, whereas changing the title or metadata of a page prompts wiping the cache for the page, its descendants in the site hierarchy and its siblings in the site hierarchy. Moving a page within the site prompts the CMS to wipe the entire cache, and so on. The CMS tries to be as cautious as it can be here — it’s better to wipe too much of the cache, than not enough.

Even then, given the range of sources from which the CMS can acquire content, sometimes the cache still can occasionally contain data that’s not perfectly fresh, and so there are options within the “Administration Shell” to wipe the entire cache, or to wipe the cache only for a specific resource.

Because the preparation of a finished page of HTML can involve a lot of work for the CMS, including a number of XML transformations, introducing the cache has helped response times — particularly for popular pages — by a surprising amount. It’s far, far easier for the CMS to shunt data straight from the database than to go through the normal page generation process, and because the system runs under mod_perl, the CMS code is itself compiled into the Apache process, and the database connection is also cached and reused.

The CMS request log keeps a high-resolution timer of the processing time taken from parsing the incoming HTTP headers to initiating the log record. Taking the homepage as an example, a sample request without the PageCache took 0.906 seconds to complete. With the output of that request cached, the second request took only 0.007 seconds. Most pages don’t take quite that long to generate the content (typically around 0.3-0.6 seconds) — but even here, the difference between half a second and less than a tenth of a second is palpable.

Wed, 30 Nov 2005

Access Versions, Redacted

Last year, I picked a beef with Mike Davies and Matt May over their arguments concerning the counter-productivity of having a separate “Easy Access” version of a website. Matt was kind enough to post a response to this in the comments here, which of course was lost when I turned comments off from the site.

Thankfully, Matt crossposted his comments to his own site. As (typically) belated as this is, I’d like to observe how calm, rational and unargumentative Matt was about this. A lesser man might have called me out for being overly defensive and, frankly, jumped-up.

Whilst I still feel that the apartheid analogy is, as the Missus would say, “a bit extra”, in hindsight it’s pretty obvious that my argument here was really that:

  1. Woking had, at that time, an “Easy Access” version.
  2. I thought I was doing a good thing by implementing it.

Deep down, I knew that making the regular site accessible beyond WAI-A was really the right thing to do, but I was too enamoured with the fact that the “Easy Access” version of the site showcased how the CMS could re-contextualise the whole site at the drop of a hat.

We have other uses for that technology now (more on which another time), so I don’t feel it’s wasted now that we’ve fixed up the normal site and dropped the “Easy Access” version. And, honestly, it really is so much better this way.

Both Honoured and Shamed

Observe, if you will, item #40 in this list of links from about a year ago.

Now, on the one hand, I’m massively honoured that Joe Clark, renowned accessibility specialist and author of what has affectionately come be known in some circles as the Goatse book, has actually seen my blog, let alone seen fit to link to it and describe it as both a “radically unusual weblog format” and “nice to look at in Safari”.

On the other, I’m shamed that the same Joe Clark, renowned accessibility specialist, observed that it was guilty of “violating all sorts of WCAG requirements”.

I never actually realised that the old design here had serious accessibility problems — but as someone that likes to consider themselves both a web professional, and accessibility-considerate, it bothers me tremendously that it might have, and even more that I didn’t realise. Having said that, I know there were validation issues with the old design that I never bothered to address fully, and there are a couple of (WAI-AA and above) guidelines that catch me out every time, so hopefully the problems were related to those and nothing that I’m likely to unwittingly perpetuate as a matter of idiom.

I’ve checked out the new design with validity and accessibility in mind, and it seems to have a more-or-less clean bill of health. Guideline 10.5 bites me here even now, though, which is a pity.

Mon, 28 Nov 2005

Woking gets a makeover

Way back in May 2004, I set myself the mission of reworking the design of the Woking Borough Council web site in CSS.

I managed to let it take me until September this year to actually get this done, and even now there are parts of the site where old, tables-based markup, riddled with presentational attributes and spacer GIFs, is nested inside the new CSS-driven templates. This is largely a product of the different ways the CMS can acquire content — parts of the site are derived from legacy CGI scripts massaged into the newer CMS processes. But at least the templates are done.

Of course, the Council themselves weren’t motivated to change the site templates for the same reasons I was. I felt it was high time I made the transition to rigorously separate presentation from semantics, and to put into practice the CSS techniques in which I’d only recently begun to dabble. The new implementation would have some key, measurable benefits for the site, however, and these prompted their enthusiasm for the project.

Notwithstanding the many and varied concerns over their accessibility testing, the monthly SiteMorse league tables furnished us with a number of key target areas for improving the performance of the site as a whole. We’d done well with metadata, improved our error count, and remained focussed on accessibility — but our download speeds were consistently poor.

As part of my proposal document for the project, I ran a small experiment here. The breadcrumb trail at the top of each page had been implemented as a table, but my thoughts on the CSS version led me to believe it should be an ordered list. Taking the breadcrumb trail from one example page of the site and re-coding it in this way resulted in the following delightful realisiation:

The two different approaches result in an identical visual appearance. The first approach, however, uses 846 bytes of code. The second — taking the HTML and CSS together — uses only 575 bytes. This saving is compounded by the fact that the stylesheet would only need to be loaded once, and would then be applied to any page, whereas the HTML approach is included in full for every single page on the site. The HTML alone in the second approach is only 279 bytes.

In itself, this is a small saving — but the templates contained many, many similar examples. Moreover, each page on the site contained dozens of small graphics to drive the presentation. Each primary navigation button contained the text of the button as part of the graphic, and had a rollover state — for our (at the time) 10 different navigation buttons, we therefore had to load 20 different graphics onto the page. Plus background images (specified as attributes to individual table cells, of course), spacer GIFs, drop shadows, rounded corners, and (on the homepage) a further 10 buttons and text-in-graphic headers to different blocks on the page.

We’d recently introduced a series of links on each page to run the page through Google Translate — which worked well enough for the actual text of the page, but of course didn’t touch all the text rendered as graphics, leading to a page mostly in one language, but with navigation in another.

The CSS project therefore had three key goals, other than quelling my shame:

  1. Help our download times by trimming the code itself as much as possible.
  2. Further help this by reducing the size and number of graphics used as part of the templates.
  3. Remove instances of text rendered as graphics, whilst staying pretty.

Additionally, we hoped to be able to achieve two more:

  1. Improve our accessibility through semantic code.
  2. Improve our error quotient through simplifying the HTML.

Finally, it was important that the finished version resembled the existing design as closely as possible. It was clear that the two versions wouldn’t match exactly — if nothing else, rendering text exclusively as text would guarantee that — but this wasn’t so much a redesign as a re-implementation of the existing design.

The work was completed in two main stages. The first was to take an existing page from the site and recode it; the second was to then implement this new design in the CMS’s XSLT templates.

In recoding the HTML (essentially a case of throwing away all the presentational cruft and getting down and dirty with the semantics), I had a notion that we would be able to make life easier for JAWS users (et al) by placing the primary site navigation towards the end of the HTML source rather than at the beginning, and then using a touch of creative CSS to keep them in the left-hand “ribbon” column. The principal blocks we ended up with for most pages, then, are as follows:

  1. The borough logo
  2. The breadcrumb trail
  3. The page header
  4. The “This Section” navigation
  5. The main page content
  6. The search box, primary site navigation and translation tools
  7. The page footer

The next stage was to create a range of background graphics in Fireworks, with the intent of excessive use of the CSS Sprites technique to minimise the number of graphics which needed to be downloaded, without sacrificing our multicoloured rollover effects, or the range of button imagery on our homepage.

I had been worrying about the rounded corners I’d implemented on many of the panels we had in the previous design — I didn’t want to lose them altogether, but equally, I didn’t really want to have to include separate corner graphics and the extra DIVs necessary to embed them. About this time, I learned of Alessandro Fulciniti’s Nifty Corners — some presentational Javascript to round off the corners of specified blocks without needing images. Like sIFR, Nifty Corners is resolutely unobtrusive — if you don’t have Javascript enabled, your corners merely remain sharp. No big deal. Lovely.

In order to get sign-off on the changes, I then (after spending a short time on other projects) incorporated the new design into a fresh set of XSLT templates using a system built into the CMS whereby the presentation layer can switch to a different context based on a small change to the URL of any given page. This enabled us to compare the old and new implementations of the design for any page on the site very easily, and we used this to obtain some measure of the success of the project:

[We ran a] speed check on two pages with their closest counterparts on the real site. The existing homepage was estimated at a 54 second download on a 56K modem. The new implementation weighed in at 19 seconds. The comparison of the inner page is less dramatic, with the old design at 16 seconds and the new at 10 seconds. The old inner page needed a total of 71 individual HTTP requests (mostly images), whilst the new needs 27.

Once the implementation was signed-off, all I had to do was make the new XSLT templates the default ones, and the work was largely complete.

Because the new implementation of the design addressed everything that our old “Easy Access” version was intended to do, we also switched off that feature from the site. The new site has a print stylesheet that removes much of the navigation from a printed page, and the implementation is frankly more accessible than the Easy Access version itself actually managed. With a few recent tweaks, many pages on the site (including the homepage) now seem to be WAI-AAA accessible — although of course it’s really not possible to be sure of that without further real-world, non-automated testing.

While all this was being worked on, the format of the SiteMorse benchmarks changed somewhat. The monthly league tables no longer specify a score for HTML errors (although the new implementation validates consistently), and the automated WAI-A and AA scores are now given as a percentage of pages assessed that contained at least one accessibility error. For October, we had 13.6% of pages with at least one WAI-A error, and 38.4% of pages with at least one WAI-AA error. Unfortunately, of course, you have to buy the full report to find out just what your errors are perceived to be. We might just go ahead and do that, and I’ll be sure to follow up here with any conclusions that may be drawn from this.

On the positive side, whilst SiteMorse consider our actual download times to still be unacceptable on a 56K connection (though they’re now a “Pass” for ADSL), our server has furnished us with the fastest actual response time of all 463 assessed local government sites for the past two months running. This has much less to do with the CSS makeover than the beefy hardware and the CMS’s own caching strategy (more on which another time), but I’m awfully pleased about it anyway.

Fri, 25 Nov 2005

Back from the dead

We’ll see how long this lasts, but at least there’s a fresh design to sweep away the cobwebs, if not the inane burbling about Doctor Who that’s been riding high here for, oh, about seven months now.

This year’s been fantastic, and possibly the busiest on record, both personally and professionally. That’s a terrible excuse for hopelessly abandoning this site, but at least it means that I now have a pleasing backlog of things I want to write about.

First up: reworking the Woking site in CSS.

Fri, 20 May 2005

Say it ain’t so…

By and large, the new Doctor Who is a joy. Any inadequacies thus far have been minor, and I’m an eight-year-old spellbound on a Saturday teatime all over again. I can even pretend the farting aliens never really happened.

But, dear lord in heaven, has Russell T. Davies gone completely out of his mind?

Mon, 16 May 2005

Chisels

Another dream: the world is being taken over, in the style of Invasion of the Body Snatchers, by beings who possess the bodies of ordinary people. By cutting the spinal cord at the base of the back with a chisel, the possessed can be brought back — although they’re, naturally, paralysed from the waist down.

Somewhere in me, there’s a shockingly poor science fiction writer being mercifully withheld from the rest of the world.

Wed, 11 May 2005

A Moment

Me: You know, I really feel that Doctor Who has cheated me out of really feeling the hype for the new Star Wars movie…

The Missus: Okay, I think we’ve now reached a crescendo of nerding and I’m off to bed. You are Louise Nurding, and I claim my five pounds.

Fri, 29 Apr 2005

Excellent…

Okay, I’ve got a whole bunch of proper stuff I want to write about, but in the meantime just let me say that not only is this page part of a, frankly, awe-inspiring review of Tiger, and that I’ve come to agree with nearly everything John Siracusa says, but also that the movie on that particular page is sheer genius in every way. Thanks.

Fri, 15 Apr 2005

Boobs!

Whilst conducting a spot of training in the CMS yesterday, one of the participants observed that when adding search keywords to a resource, one could boost one’s search rankings by adding “popular keywords, you know, like sex or boobs”.

I can’t help feeling there’s something quite sweet about the idea of searching Google for boobs. It’s so very much the first rung on the ladder of depravity that ends with donkeys and midgets

…and now it’s April

My dedication to this blogging lark is nothing less than extraordinary. The moment a spot of fairly intense work comes along, I abandon the whole thing in favour of scrabbling to meet deadlines. Although things have barely calmed down now, I need to either resume posting or close the blog down altogether. Here’s plumping for another go at it, at least for a further couple of months.

In the meantime, the comment spam has reached frankly alarming proportions. I’d occasionally run through the “writeback” files kept by my blogging system, Blosxom, to clean out the comment spam, but this is a tiresome, manual process — and I just can’t keep up. Coupled with the fact that the comment system on this blog has never worked completely to my liking, I’ve decided to remove the comment facility altogether. I’d like to thank the people responsible for the handful of genuine comments that have been posted, and roast the people responsible for the rest on an inadequately-buttered spit.

Tue, 18 Jan 2005

A Birthday Compliment

The missus’ 30th birthday party had everything — not least a minor spot of trouble with a pair of gatecrashers that led to my future brother-in-law paying me perhaps the greatest compliment of my adult life:

You can be quite threatening, in a mincing sort of way.

Well, if not the greatest compliment, it’s certainly the most inventive.

Tue, 11 Jan 2005

All I’m Going to Say is This

Footnote 2 on the US site is the best thing I’ve read in months, and it’s almost as funny on the UK site.

Thu, 06 Jan 2005

It looks like I might need a “BBC” category soon

This is why Manic is the froth on my morning cappuccino.

Wed, 05 Jan 2005

Missing the BBC

The missus and I took an excellent and much-anticipated holiday to Las Vegas between Christmas and New Year. News of the Asian Tsunami was just beginning to emerge as we left, although at the time no-one in the West yet had any idea of the scale of the disaster. When we arrived in the States, we kept an eye on the TV news as best we could, Vegas existing in a weird little bubble of entertainment and gambling.

It’s little wonder that Americans have a reputation for being ill-informed about events beyond their shores, as the news coverage they get is simply dire, even on CNN. Reports focussed intently on the stories of individual American families being reunited with people they’d worried had been lost, and opinion pieces on “could it happen here” — further confirmation of the culture of fear choking the nation.

When we returned to the UK, it took ten minutes with News 24 for us to get properly acquainted with the extent of the situation, the mechanics of the tsunami, the differing cultures of the regions involved, the Buddhist attitude towards suffering and the religious context for other groups, and the sincerity of people’s reactions to it all. There’s a great deal I love about the US, and I adore holidays there (US-VISIT notwithstanding) — but I swear that TV here is just better in all respects. Even TV of US origin is better viewed over here, by dint of a substantially less infuriating programme/advertising ratio.

Incidentally, at least while we were there, the Tsunami Asian Bar and Grill at The Venetian, had precisely no visible acknowledgement of the sudden tastelessness of their name. It’s possible, since we didn’t dine there, that they were quietly donating 10% of their proceeds to an aid fund without an extravagant announcement to the effect. But somehow, Vegas being Vegas, I rather doubt it.