In the olden days, finding information on the web was hard, and good link directories were worth their bytes in gold, and good content was worth even more. Nowadays we have good search engines, thanks to Google, and lots of good content (and lots of bad content too, but that’s not really a problem).

Wikipedia is one source of good content on the web. But it’s not the only one – it’s just the most obvious one in many cases. My problem with Wikipedia is mainly how people link to it. Here’s an example:

　×　I like to feed the pigeons. I sometimes feed the sparrows too.

This I’d like to claim is by far the most common way of referring to Wikipedia. What’s wrong with it?

1. If your reader doesn’t know what a pigeon or sparrow is, it’s 99.9% certain that’s because the reader isn’t very good at English. In this case, if anything, every word ought to be linked to a dictionary, but that wouldn’t be very useful except for 0.1% of your readers. They should know how to look up words themselves if they’re not proficient in the language of the text they’re reading, don’t you think?

2. If your reader suddenly becomes very interested in pigeons or sparrows by reading your text, I’m sure one of the first places your reader would look for informations on said birds is Wikipedia. There is no need to point people to Wikipedia, because everyone knows how to find Wikipedia articles anyway. In Firefox, just type “wikipedia pigeon” in the address bar, and you’ll get redirected to the page. In other browsers, go to google.com and type it in and press “I’m feeling lucky”. It’s real easy.
3. The reader might think that you’re linking to some really interesting tidbit about pigeons or sparrows, or maybe a funny Youtube video. But to find out, the reader has to mouse over the links and check the status bar where they point. Only to find out they just point to the Wikipedia entries, which the reader could easily have found without your assistance. This makes reading cumbersome.
4. It degrades the quality of interlinking on the web. All you’re doing is helping Wikipedia get a higher PageRank – and their PageRank is already as high as it can get. You’re not helping the guy who has spent serious time documenting pigeons and sparrows and runs a really interesting web site on the subject that your readers may actually enjoy if you had only taken the time to find and link to it – like in the old days.

In the example above, it’d be easy to argue that “pigeons” and “sparrows” don’t need to be linked at all. But this is how thoughtlessly people use hyperlinks. So let’s change the example to something similar yet where hyperlinking a word might be more appropriate:

　×　But the pronunciation doesn’t change since the word is a dvandva.

I bet you don’t know what a dvandva is. It doesn’t matter here anyway since I’m just using it as an example but I’m sure you’ve already went and read the Wikipedia article. The Wikipedia entry comes out first on Google for a search on “dvandva”. Why? Probably because of hyperlinks like the above example. But the article sucks. Here’s an example of good use of hyperlinking:

　○　But the pronunciation doesn’t change since the word is a dvandva.

The target of this link is a paper written at a university regarding dvandvas in Japanese. Of course, if the context isn’t about Japanese then it might not be the best link target, but I’m sure there are more good articles about dvandvas. I thought that paper was da proverbial bomb. Really good reading. That’s why I link to it – to encourage my readers to read it, and to promote it in the search rankings. (Of course this is hypothetical since I’m actually writing about something else right now but if I were writing about dvandvas…)

It takes time to find good link targets – but please take the time! For your own, your readers’, and the authors’ of those link targets sake. And for the future of the Internet.

Note that I am not opposed to linking to Wikipedia completely. If the Wikipedia article on a subject really is the best piece of information on it on the whole web, and the subject demands a hyperlink in order to be understood by most people, then indeed it’s the corresponding Wikipedia article you should link to.

Also of course when discussing Wikipedia itself it is highly appropriate to link to Wikipedia sources. But even in this case, I see it go wrong, for instance like this:

　×　Wikipedia recently started adding the “” attribute to outgoing links.

That “nofollow” link to the Wikipedia article on the “nofollow” attribute violates the point outlined above in the same way “pigeons” did. Here’s a better way of linking it:

　○　Wikipedia recently started adding the “nofollow” attribute to outgoing links.

which links to Wikipedia’s meta wiki describing the policy. That’s a good way of linking to Wikipedia. Here’s an even better way of doing it:

　☆　Wikipedia recently started adding the “nofollow” attribute to outgoing links.

That links to the most interesting text on the subject that I could find in a couple of minutes. I’ll gladly share that good piece of writing on this subject with you – that’s why I link to it.

Lastly, I’d like to mention that I think said Wikipedia policy of adding the “nofollow” attribute to outgoing links on Wikipedia is stupid and bad. I think that if you read my above argumentation, you’ll see why I think that. People are linking to Wikipedia en masse for no good reason, bloating its PageRank and diminishing the chance of other, better, sources of information to get found. If at least being cited in a Wikipedia entry boosted the PageRank of the source, then the chance of someone finding it would improve just a little. Not to mention it would be fair. The paper on dvandva above and Ed Felten’s blog entry deserve that.

[No interwebs were hurt in the writing of this blog post; all links to Wikipedia have the "nofollow" attribute set.]

The page consists of basically three main parts: Essential Japanese programming vocabulary, Expressions for communicating technical issues in Japanese (with a Part II), and A look at a Japanese software specification. They’re based on material I’ve gathered during my years working in Japan, mostly in completely disorganized, scattered text files, so collecting and choosing the useful bits of it for this page was more work than it should have been…

Actually, I first got the idea of writing a “Japanese for Programmers” book some time ago when talking with a former colleague. The idea was that there are so many especially Indian software developers working in Japan, but there are as yet no books aimed specifically at this segment, and there should be a huge market.

Anyway, I’m an avid reader of Seth Godin’s blog, and Seth is the guy behind Squidoo as well, so I naturally stumbled upon it. I gave it a go, and it’s actually a really fun and easy way to create modern, stylish, SEO’d web pages without having to bother with the technical issues.

And then in last month’s search queries used to find my site I saw “japanese speaking programming“, and that finally got me going to do something with my Japanese for Programmers idea – I mean people are searching for it, and they’re already finding my site from it (there’s no “Japanese for Programmers” site on the net anyhow!), despite me not specifically covering that topic, so it seems everything falls into place. And thus the Japanese for Programmers Squidoo “lens” was born!

So I hope it’ll get lots of readers and comments. I still have loads more material, so if it gets popular I might do a follow up. Here’s the URL again:

http://www.squidoo.com/japanese-for-programmers

If you haven’t heard of Mixi that means you aren’t Japanese or Japanophile. To put it generalized and bluntly: Mixi is the only social networking site in Japan. Japan is the second largest economy in the world (★pause for reflection★). The reason it’s so popular is basically the same as why Microsoft products are: they were there first, and everyone else uses them, and the basic functionality is actually good.

Mixi, technically, is stone age. Although recently they’ve introduced video upload etc that we have become accustomed with on the modern web, the basic technology is just server-side perl scripts outputting broken html with a table-based design. In other words: it’s web 1.0, although they have a pastel color, but it’s the wrong hue, and pastel color alone doesn’t make web 2.0 – you need rounded corners and rss too.

But as a consumer-oriented product, Mixi is really state of the art. It’s actually statier than the statiest art. I started using the predecessors to nowadays’ social networking sites in junior high school, back in Sweden. That was like 10 years ago now I guess. (Heh, when I think back, that was about the time I got my first mobile phone. Was that only ten years ago?!) . Even though they used about the same technology then as Mixi does now, the culture and usage patterns are completely different. They were about kids doing their best to make their pages look as hideous as possible (like today’s Myspace) and presenting themselves as generally emo and cool. And guys (both young and very old) trying to pick up young girls, of course. But Mixi is not like that.

Oh well, there’s that too. But Mixi is much more woven into the fabric of Japanese society. It’s like an ad-sponsored public service page (fortunately, and strangely, the mobile version doesn’t have ads). And fortunately, you can’t design your own page, and there are no widgets etc, so it’s actually possible to browse around people’s profiles and community pages. Really nice, although I bet it’s more because the Mixi people haven’t figured out how to implement it technically than a conscious decision.

I joined Mixi when I realized my Japanese language skillz had gotten good enough for me to actually understand pretty much all of the communication taking place there. And the reason I keep using it is still mostly to practice reading Japanese; every day on the train I read some new, interesting tidbits from the parts of Japanese society that concern me. Like what’s happening in my town, what’s happening along the train lines I use, what events are going on at my favorite bars and clubs, or if there’s a Swedish-speaking off-kai soon (off-kai: オフ会, people who talk online meet up in real life), etc. I give it three thumbs up!

Anyway, now for the real anecdote here, and the reason I figured I’d write this blog post at all: In their reception they had this wall with all kinds of catchy words and phrases written on it in the style of a tag cloud. Very, very web 2.0 hip I must say… If anything proves that you’re falling behind current developments in the world of the web, it’s that you’re trying to mimic a Google office, I’d say. (I’d like my office to look classical and sophisticated, and there’s always music in the air.)

Now, you can notice that, just beside “web 3.0“, they’ve included the word “self-fertilization“. I don’t suppose I’m the only one who kinda gets a bit suspicious because of that. And I find the graphical proximity to “web 3.0″ especially intriguing. I don’t suppose it’s a statement of theirs? Nah, it’s probably one of those engrish.com kinda moments, you know, when Japanese people confuse R and L, or use Google Translate to translate business emails. Anyways, it’s funny.

“Web 2.0 is a term that corresponds to Ajax.”
My standard answer to “what is web 2.0?” would be something along the lines of “it’s about control of users and data“, i.e. you build a big web site where users can generate the content, market it, and pray you’ll be among the 1% that are somewhat successful. Then you capitalize on providing pieces of that data. Google if anyone should know that. Ajax (or whatever you choose to call it) is a technology that is certainly a part of the modern web, but really, aren’t pastel colors and rounded corners more important for a web 2.0 site? even if you build it with old-fashioned server-side scripts only. Ajax as a technology is an enabler, not a necessity.

“[Web 3.0] is a different way of building applications.”
Yes, maybe that too. The web by the time 3.0 comes around is bound to have some new technology – or rather some new uses of the technologies we have. And this will allow us to make applications in a different way. So the statement is trivially true, but it does not provide a definition or even a speculation of what web 3.0 will be like. Let’s move on…

“Web 3.0 will be applications that are pieced together.”
This is more interesting. Like mashups? I’m sure all of us computer software users would like to have that. But I’ll believe it when I see it: applications from different vendors cooperating. So far unix system tools are the only ones to come close to this ideal. Unfortunately, I don’t think this is realistic for web apps.

“The applications are relatively small”
Now this is one of my favorite pet peeves. I nagged a bit about this on my CSS Nite presentation as well: Right now, many people, and especially Apple, are thinking of web apps/widgets/gadgets as small, more-or-less useless applications for trivial tasks such as analog clocks or calculators, or something that in the very least is separated from the tasks of “real” applications. I don’t agree with this at all…

Consider Mac OS with its Dashboard (even though it is a great reification of the ideal of web tech-based apps): you have a “normal” mode; your normal apps, running along in their windows as usual, and a “widgets” mode; your normal apps—no wait, these aren’t normal apps – these are widgets, special kind of apps made in a special, even naive, kind of way, to be run in a sandbox isolated from your normal desktop working environment. Even Opera has copied this thinking.

This is definitely how it is right now, so I can understand Apple’s decision. But I don’t think it’s how it will be, in the future, boys and girls. Web technology-based application will be just as common as other applications, I think. And in a few years they will be just as big as well. Many have tried to achieve this before – Java and Dotnet come to mind – but web technology has already won. I thought a lot about this, and I’ll write about it some time.

“The data is in the cloud”
I want some of what you’re smoking too dude, but there is no cloud. Data belongs to somebody, access is restricted, bandwidth is limited. The Internet is not a cloud (it’s a series of tubes).

There are a few more goodies in the video. Anyway, what will the web 3.0 be like? I mean, if you can figure that out now, you’ll be a billionaire. I think I have pretty decent idea, and I’ll continue writing about web 3.0 here in this blog. I think and hope that Google won’t be playing a big part in it. (They might be absorbed into something bigger, though).

This trick is so simple and incredibly easy to pull off that you have no reason not to do it. There are essentially three things involved:

1. Make apache serve .gz.js files with gzip transfer encoding
2. Bundle up all your JavaScript files into one (or a few) big file(s)
3. Gzip your One Big File(s)

Let’s go through these step by step…

1. Make apache serve .gz.js files with gzip transfer encoding
Gzip transfer encoding is great! The only bad thing about it is that on dynamic content, the web server will have to use up some processing power to compress the files on the fly, but in this case, we’re serving static JavaScript source files, so that’s not even an argument! (As for the client, if it’s got enough CPU to run your web app, it’s not gonna have any trouble doing a little unzipping…)

The easiest way to do this is to edit the .htaccess config file in the directory where you put your JavaScript files, or any directory above that one to make it more global. The only thing you have to do is add the following line:
AddEncoding x-gzip gz

This adds the gzip transfer encoding to apache for files with a “gz” file extension.

Notice the title says “.gz.js” though: by adding “.js” at the end, the files will be served correctly with the text/javascript mime type. Otherwise the browser might go berserk. That’s also why I wouldn’t recommend using “.js.gz”; it’ll get served with gzip transfer encoding alright, but not with the JavaScript mimetype.

This solution works even with clients that don’t support the gzip transfer encoding, because web browsers send an “accept-encoding” header that tells the server which encodings the client supports, so apache won’t use gzip transfer encoding if it’s not supported by the client.

2. Bundle up your JavaScript source file into one (or a few) big file(s)
This means simply appending all your source files to one file. Or a few files. I usually bundle up the libraries/utility code etc, i.e. code that you don’t change very often, into one file and the rest of the app into one file that gets updated more often. You can do this by copy-pasting in your text editor if you like, or make a script that does it. I usually have the whole deployment process automated on the server, which I’ll write about some other time perhaps.

Another good way to do it is using Dojo ShrinkSafe. ShrinkSafe will not only append all your source files into one big file, but also reduce the size even further by optimizing the JavaScript code. However, I have had problems with scripts not running correctly (in all browsers) after running them through ShrinkSafe, despite their claims of it being “safe”. I would especially recommend not using the “strip newline chars?” option. Anyway, the file size reduction the results from using ShrinkSafe is small compared to just gzipping the files (I’ll show you some numbers later) so if you’re afraid of regressions, don’t use it.

3. Gzip your One Big File(s)
Now that you’ve got one or two or so JavaScript files, you need to gzip them into a gzip file. You can use the command-line gzip tool on the server to do this, like:
gzip -n9c scriptfile.js > scriptfile.gz.js

The -n option means to not store the the original file name in the gzip file. No big deal if you do, but there’s no reason to do it either. -9 means maximum compression. I like doing things to the extreme! And -c means to write to standard output instead of modifying the source file, so we need to pipe it “>” to a .gz.js file at the end. Or you can use a UI tool such as 7-Zip that supports gzip compression. That’s just damn easy.

Then upload the .gz.js script file to your server, and change all your old <script> tags for every file you had to one pointing to it. I.e. like:
<script type=”text/javascript” src=”scriptfile.gz.js”></script>
That’s all there is to it!

Show Me The Numbers!
Let’s use my Minesweeper game for comparison. The code is served in two files: libs.gz.js containing Prototype and Scriptaculous core and Effects libraries, and minesweeper.gz.js containing the actual game. The breakdown of the files in these libraries before combining them is as follows (at the time of writing):

Original
libs.js: 95 kB + 3 kB + 38 kB = 135 kB
minesweeper.js: 3 + 9 + 9 + 4 + 2 = 25 kB

Shrunk
libs.js: 92 kB (32% smaller)
minesweeper.js: 13 kB (48% smaller)

Gzipped
libs.js: 30 kB (78% smaller)
minesweeper.js: 7 kB (72% smaller)

Shrunk + gzipped
libs.js: 27 kB (80% smaller)
minesweeper.js: 4 kB (84% smaller)

Now, add to that the fact that every HTTP request for a file incurs a 1 kB overhead in HTTP headers, and you get an original total amount transferred of 135 + 25 + 8 = 168 kB compared to 27 + 4 + 2 = 33 kB, i.e. a total saving of 135 kB, i.e. 80% less bandwidth used, not to mention that making two http requests is much faster than making eight http requests!

The next step would be to bundle up html, css, scripts, and everything you possibly can into one request… (In fact, I have scripts doing that automatically at runtime on some sites…) But you have to draw the line somewhere. There’s a definite trade off in maintainability, cachability, and compatibility of the site. So I’ll stop here.

So I was surprised by how hard it is to turn off this aggressive non-caching policy. I googled for a few minutes and browsed the php documentation without finding an easy way of doing it. Ok, so you can use the following code snippet to enable caching in php (the argument to the function is number of seconds the page is valid):

function send_cache_headers($expire) {
  header("Cache-Control: max-age=$expire");
  header("Pragma: cache");
  header("Expires: " . gmdate('D, d M Y H:i:s \G\M\T', time() + $expire));
}


I combined the two mentioned matters and did a presentation and demo developing a chat widget using the freshly-released Ajax Messaging Library. Well, it’s in early beta quality with many features planned but not yet implemented, but it works! Even on stage (except for a small bug, hehe). Anyway, it’s out there now! I’ll write more about it later.

P.S. I did the presentation in my socks; my shoes were drenched this morning by Typhoon no. 4.

I’ll also have a presentation on The Days of Web Standards 2007 on 14~15 of July. My presentation is on Sunday the 14th at 14:30. It’ll be a bit longer and also on widgets (although this time too I’ll try to sneak in as much talk about web application development as possible without Marketing noticing). I’ve got some pretty nifty ideas for a live coding session and demo… I hope someone will attend. Anyway, there are many very interesting speakers scheduled…