When I saw the latest in the Lord of the Rings trilogy of movies a short while ago, I wondered how Tolkien had invented the artificial languages of Middle Earth. In my previous article, I told of my desire to discover which real language had been the biggest influence on Tolkien for his invented ones. As a software developer, I wanted to discover this information algorithmically. My idea was to use my own string similarity algorithm to compare each word from a list of Tolkien words to words from 14 other real languages. For each Tolkien word, I would find and record the language with the word that is (lexically) most similar. The set of most-similar words and the languages from which they came would provide new insights into the influences on Tolkien.
Lord Of The Strings Part 2 - Analyzing the Results (Page 5 of 7 )
With the first pass mechanism incorporated into the algorithm as described, I found that the program runs in quite acceptable time. On my 1.6 GHz PC with 768 MB Ram using Sun's Java J2SE 1.4.2 and MySQL 3.23.49, it runs to completion in about four minutes. The java program and database both run on the same machine, so there is no network latency in this configuration. The main reason for this relatively quick execution time is the first pass mechanism that returns a subset of the whole dictionary of words. Without this mechanism, I am certain that the program would take orders of magnitude longer to finish. (And if it took longer to finish, then my idea of storing results back into the database as they are found would gain significance.)
Anyway, let's have a look at the results obtained. First of all, let's see how many exact matches were found and for which languages:
> select count(*), lang from matches where similarity=1.0 group by lang; +----------+-----------+ | count(*) | lang | +----------+-----------+ | 2 | DANISH | | 7 | DUTCH | | 3 | ENGLISH | | 13 | FINNISH | | 1 | FRENCH | | 1 | GERMAN | | 1 | HUNGARIAN | | 7 | JAPANESE | | 2 | LATIN | | 2 | NORWEGIAN | | 2 | POLISH | | 4 | SPANISH | | 1 | SWEDISH | +----------+-----------+ 13 rows in set (0.00 sec)
The most exact matches were found for Finnish. By executing the following query, we can inspect the words of those exact matches:
select word from matches where similarity
=1 and lang='Finnish';
The exact matches were found for the Finnish words 'andor', 'aragorn', 'arwen', 'balin', 'hobbit', 'ilmarin', 'lindon', 'maia', 'nahar', 'ori', 'rohan', 'sylvan', and 'velar'; whilst English scored exact matches for the words 'goblin', 'hobgoblin' and 'thrush'.
The following query shows the frequency with which each language was chosen as being most similar, over the whole set of 470 Tolkien words.
> select lang, count(*) as hits from matches group by lang order by hits desc; +-----------+------+ | lang | hits | +-----------+------+ | FINNISH | 100 | | ENGLISH | 64 | | SPANISH | 62 | | JAPANESE | 52 | | DUTCH | 40 | | LATIN | 26 | | POLISH | 25 | | DANISH | 19 | | NORWEGIAN | 19 | | FRENCH | 18 | | HUNGARIAN | 18 | | GERMAN | 16 | | SWAHILI | 7 | | SWEDISH | 4 | +-----------+------+ 14 rows in set (0.00 sec)