Localization support for language identifier

Something’s wrong when a language identifier doesn’t have localization support. So I cooked up a little localization code for What Language Is This?, which proved to be not as easy as one might guess. That’s because some of the textual content of the web app is in HTML, other is generated by PHP, and yet other is generated in JavaScript. I wanted to have one single source of localized strings for all three output paths to simplify overviewing, translating, changing, and adding strings to the web app.

I’m not sure if there’s any good solution for this out there, but I cooked up my own. Each language translation has its strings in a text file formatted like an ini file with id keys and localized strings separated by an equals sign. You can view the English and Japanese raw text files if you like. These are read into a PHP array (i.e. dictionary), after first looking at what language is specified by the URL (/en for English, /ja for Japanese or any other code), and if that is not specified then looking at what languages the browser is set to prefer via the Accept-Language HTTP header. If the requested language is not available then default to English.

To get the html output localized, the php script that reads through and configures the app (the plain html file itself is set up to run offline for debugging purposes only) looks for string ids enclosed in percent signs, i.e. like %strings id%. These are then replaced with the localized strings from the dictionary. The php-generated content is trivially changed to look up strings from the dictionary. On the JavaScript side, I wanted access to the same string dictionary that I had on the php side, so this is inserted into a <script> block of the generated html output as a JavaScript object (i.e. dictionary). String id lookups can then be done on this object from the JavaScript code just like on the php side. In other words, the php string dictionary is converted into JSON, which is used from the JavaScript side.

あれ何語? What Language Is This? in 日本語

あれ何語? What Language Is This? in 日本語

It all works pretty well and meets my goals. The only downside is that it relies on the server to do some processing, so when I develop on the offline version the strings aren’t available, instead I get to see the raw string ids, which can be useful too, but you have to rely on imagination to envision the end result. Isn’t programming always like that anyway, though?

The first translated version of What Language Is This? is of course Japanese, done by myself and my wife (初めての共同作業? lol), not just because it’s easy for me to do, but also because when looking at the AddThis stats, Japan is the top ranking country, and also as you know the average English skills in Japan are pretty bad, so I suspect there is a demand for a Japanese translation. Looking at the access stats, and discounting those with good English skills (India, Netherlands, Scandinavia, for example), next in line would most likely be Spanish, French, and German, in that order. Anyone feel like helping? Please drop me a comment in that case. I can offer proper credit and a link back from the site in return.


New Feedback Feature for What Language Is This?

I got around to implementing a feature I’ve been planning for What Language Is This? today: feedback. Not the comments – that’s been there from the start – but a way of sending immediate feedback on specific results. So that if you disagree with the result, or you know the correct language but it’s not yet supported, just click on “send feedback” that appears with each result, and a simple form pops up that where you can indicate what the problem with that result is.


The entered text can also be sent with the feedback, allowing me to gather more sample texts to use as material for the statistical analysis used as a basis when identifying the language, and for testing (there’s an automatic test feature built in to What Language Is This?, just run selftest() from a JavaScript console on the page and it’ll test all supported languages to check for regressions – very handy when updating the database, since it’s easy to accidentally break some of the fine tuning).

Anyway, I think it’ll be useful, and I hope everyone will use it a lot since it’ll help me improve the site. I’m already getting a lot of useful and encouraging comments so it’s really fun to keep on developing it. For the next update I’ll probably add more languages.


What Language Is This? Dot Com!

http://whatlanguageisthis.com/

Since the language analyzer is becoming one of the most used web services that I run, the other day I was thinking that it would be cool get it its own domain (and a .com domain costs just 50 SEK (around 850 yen in normal times) anyway). So I was thinking about what domain name to get – that isn’t already taken – and well, one of the most common search phrases people use to find the language analyzer is “what language is this webpage/blog/text/whatever” and luckily whatlanguageisthis.com was available, so there it is! I think it’s quite easy to remember and very easy to tell people. 4 stars out of 5, perhaps? Pretty good.


Setting up the new site was pretty easy; it’s essentially just a php script that chdirs into the language analyzer directory and continues from there as before.

I also did another nice update: the data file that the app uses to identify the language is now downloaded after the page and all the application javascript files have loaded. That means the page should load much faster, and the user can start reading the instructions or entering text while the data is being downloaded in the background. If the user clicks “Go” before the data file is downloaded, it will stop and wait, while displaying a typical web 2.0-ish loading indicator.

I’m planning to add support for more languages soon, and improve identification of similar-looking languages even further. Anyway, here’s the url for the new site again:
http://whatlanguageisthis.com/