June 16th, 2009
Lately, What Language Is This?, the web-based language identification tool I’m running, has been getting many hits from Tamil-language sources, probably as a result of being covered in two seemingly popular blogs, techintamil.blogspot.com, and tamilnenjam.com. As another blogger pointed out,
Also this service is very good at identifying indic languages (where as many other services fail to understand).
Well, thanks. And yes, I have been making sure that the languages of the Indian subcontinent and its surrounding areas are thoroughly supported for identification.
But two notable languages have been missing, and I finally got around to adding them. Namely the two Dravidian languages Malayalam (not to be confused with Malay, to which it is unrelated) and Kannada (not to be confused with Canada, to which it is unrelated).
Together with the already supported Tamil and Telugu, this means that all four literary Dravidian languages are supported now! I hope this will be of use to many, and I’d like to thank the Dravidian-speaking bloggers for their support in the form of writing about the site.
Tags: improvements, language, language analyzer, projects
Posted in projects | No Comments »
February 1st, 2009
I got around to implementing a feature I’ve been planning for What Language Is This? today: feedback. Not the comments – that’s been there from the start – but a way of sending immediate feedback on specific results. So that if you disagree with the result, or you know the correct language but it’s not yet supported, just click on “send feedback” that appears with each result, and a simple form pops up that where you can indicate what the problem with that result is.
The entered text can also be sent with the feedback, allowing me to gather more sample texts to use as material for the statistical analysis used as a basis when identifying the language, and for testing (there’s an automatic test feature built in to What Language Is This?, just run selftest() from a JavaScript console on the page and it’ll test all supported languages to check for regressions – very handy when updating the database, since it’s easy to accidentally break some of the fine tuning).
Anyway, I think it’ll be useful, and I hope everyone will use it a lot since it’ll help me improve the site. I’m already getting a lot of useful and encouraging comments so it’s really fun to keep on developing it. For the next update I’ll probably add more languages.
Tags: improvements, language analyzer, projects, testing, web apps
Posted in projects | 2 Comments »
July 5th, 2008
http://whatlanguageisthis.com/
Since the language analyzer is becoming one of the most used web services that I run, the other day I was thinking that it would be cool get it its own domain (and a .com domain costs just 50 SEK (around 850 yen in normal times) anyway). So I was thinking about what domain name to get – that isn’t already taken – and well, one of the most common search phrases people use to find the language analyzer is “what language is this webpage/blog/text/whatever” and luckily whatlanguageisthis.com was available, so there it is! I think it’s quite easy to remember and very easy to tell people. 4 stars out of 5, perhaps? Pretty good.
Setting up the new site was pretty easy; it’s essentially just a php script that chdirs into the language analyzer directory and continues from there as before.
I also did another nice update: the data file that the app uses to identify the language is now downloaded after the page and all the application javascript files have loaded. That means the page should load much faster, and the user can start reading the instructions or entering text while the data is being downloaded in the background. If the user clicks “Go” before the data file is downloaded, it will stop and wait, while displaying a typical web 2.0-ish loading indicator.
I’m planning to add support for more languages soon, and improve identification of similar-looking languages even further. Anyway, here’s the url for the new site again:
http://whatlanguageisthis.com/
Tags: language analyzer, projects, web apps
Posted in projects | 42 Comments »
May 10th, 2008
Over time I’ve been making some smaller changes to the language analyzer (my language identification web app), like manually tuning it to better distinguish between hard-to-distinguish languages, like the Scandinavian languages, Serbian-Bosnian-Croatian-Slovenian, Afrikaans and Dutch, and Czech and Slovak.
But I’ve been wondering what languages people use it for, so yesterday evening, while drinking shochu (in spite of which I could only find one bug today! but I did write a processing and database-intensive function, n00b style, which I replaced with a single SQL query today…), I added logging of the results. Only when the language identification certainty is reasonably high is it logged, and only the result; the actual text inputted is not sent. This, of course, happens in the background. A language is only logged once per client, and results from clicking the “example” button (Tower of Babel extracts – I like that story) are not logged.
This morning I added the top ranking to the page. It’s generated on the server side in order for the search engines to see it. The top 5 languages for the past seven days are printed. At this time, i.e. about 15 hours after the result logging started, these are Spanish, Korean, Portuguese, and Thai.
You can see the currently most inputted languages live: http://henrikfalck.com/languageanalyzer/
Tags: language analyzer, projects, web apps
Posted in projects | No Comments »
January 27th, 2008
“Have you ever wondered what language a blog entry you glanced at might be in?” was the question I set out to work on more than two years ago, if memory serves me right. I always get curious when I see a blog post in an unknown language. I mean not just a language I don’t speak – a language I can’t identify.
I thought it would be a really hard problem to solve – writing a piece of software that could figure that out. It turned out not to be so hard though. Just hours of programming, and probably a lot of luck. Because my initial hunches on how to tune the algorithms proved to be pretty right, and I was, and still am, really startled at how good the software became.
I released it as the Wørd – Language Analyzer Opera widget. Unfortunately the target audience for Opera widgets is quite small, so I always thought of making it into a web page. I don’t know why it took so long, but here it is!
The web page version has some new, cool improvements. It will try to detect as you’re typing, for instance. It also has improved support for Swedish, Serbian, and Afrikaans. And the UI is in my opinion better than the widget version.
So please try it yourself and see how it works. It’s pretty fun to just copy-paste any piece of text your can find on the Internet into it, or just type something in a language you know yourself and see when it gets it right. Here’s the address again:
http://henrikfalck.com/languageanalyzer/
Tags: language analyzer, projects, web apps, widgets
Posted in projects | 7 Comments »
