Home arrow MySQL arrow Page 6 - Lord Of The Strings Part 2
MYSQL

Lord Of The Strings Part 2


When I saw the latest in the Lord of the Rings trilogy of movies a short while ago, I wondered how Tolkien had invented the artificial languages of Middle Earth. In my previous article, I told of my desire to discover which real language had been the biggest influence on Tolkien for his invented ones. As a software developer, I wanted to discover this information algorithmically. My idea was to use my own string similarity algorithmto compare each word from a list of Tolkien words to words from 14 other real languages. For each Tolkien word, I would find and record the language with the word that is (lexically) most similar. The set of most-similar words and the languages from which they came would provide new insights into the influences on Tolkien.

Author Info:
By: Simon White
Rating: 5 stars5 stars5 stars5 stars5 stars / 28
March 17, 2004
TABLE OF CONTENTS:
  1. · Lord Of The Strings Part 2
  2. · The Size of the Problem
  3. · Discovering String Similarities
  4. · A Class Called Compare
  5. · Analyzing the Results
  6. · The Finnish Line
  7. · Differences on the Table

print this article
SEARCH DEVARTICLES

Lord Of The Strings Part 2 - The Finnish Line
(Page 6 of 7 )

If a language is selected as the best match for a Tolkien word, I call it a 'hit'. As you can see from the results of the query, Finnish scores the highest by a clear margin, with 100 hits. The languages with the next-most hits are English, Spanish and Japanese, with 64, 62 and 52 hits, respectively. At first sight, this seems to support the argument that Finnish is the language that had the biggest influence on Tolkien (as suggested by the Finnish speaker, Harri Perälä).

However, if you recall the sizes of the word lists, analyzed at the end of the Part 1 article, you will remember that the Finnish word list is also the largest word list of all the languages, comprising 21.4% of the words in the database. So perhaps the reason why Finnish scores so many hits is simply because I had so many Finnish words in the database?

Clearly, to conclude anything from the results, we need to take account of the relative sizes of the word lists for each of the languages. Perhaps the best way of doing this is to compute an expected number of hits for each language (based on the size of its word list), and compare that number to the actual number of hits that we found. The following pair of queries computes the number of expected hits for each of the languages, under the assumption that any of the languages is equally likely to contain the most similar word to any given Tolkien word (a so-called 'null hypothesis').


mysqlselect @total:=count(*) from words where lang<>'tolkien';
+------------------+
| @total:=count(*) |
+------------------+
|          1342940 |
+------------------+
1 row in set 
(1.20 sec
mysqlselect langround(count(*)*470/@total,1) as expected_hits 
       from words where lang
<>'tolkien' 
       group by lang 
       order by expected_hits desc
;
+-----------+---------------+
lang      expected_hits |
+-----------+---------------+
FINNISH   |         100.5 |
DUTCH     |          62.4 |
GERMAN    |          56.0 |
FRENCH    |          48.4 |
JAPANESE  |          40.3 |
POLISH    |          38.3 |
SPANISH   |          30.1 |
LATIN     |          27.0 |
NORWEGIAN |          21.6 |
ENGLISH   |          19.7 |
DANISH    |           8.9 |
SWAHILI   |           6.4 |
HUNGARIAN |           6.2 |
SWEDISH   |           4.2 |
+-----------+---------------+
14 rows in set 
(1.06 sec)


blog comments powered by Disqus
MYSQL ARTICLES

- MySQL and BLOBs
- Two Lessons in ASP and MySQL
- Lord Of The Strings Part 2
- Lord Of The Strings Part 1
- Importing Data into MySQL with Navicat
- Building a Sustainable Web Site
- Creating An Online Photo Album with PHP and ...
- Creating An Online Photo Album with PHP and ...
- PhpED 3.2 More Features Than You Can Poke ...
- Creating An Online Photo Album with PHP and ...
- Creating An Online Photo Album with PHP and ...
- Security and Sessions in PHP
- Setup Your Personal Reminder System Using PHP
- Create a IP-Country Database Using PERL and ...
- Developing a Dynamic Document Search in PHP ...

Watch our Tech Videos 
Dev Articles Forums 
 RSS  Articles
 RSS  Forums
 RSS  All Feeds
Write For Us 
Weekly Newsletter
 
Developer Updates  
Free Website Content 
Contact Us 
Site Map 
Privacy Policy 
Support 

Developer Shed Affiliates

 




© 2003-2017 by Developer Shed. All rights reserved. DS Cluster - Follow our Sitemap
Popular Web Development Topics
All Web Development Tutorials