The science of shapes & language

Hieroglyphics, Metropolitan Museum NYC. Photography:

Hieroglyphics, Metropolitan Museum NYC. Photography:

A few weeks ago, in their Technology Quarterly, The Economist published a long-read on computers and language. In it, various aspects of computers processing language were addressed (speech recognition, translation, speech, etc.). Given my background in linguistics and having worked as a translator myself, I read the articles with above average attention, especially the article on machine translation.

Scientists have been working on automated translations since the early 1950’s, inspired by the success of code-breaking during WWII. Anecdotally, since then, whenever scientists in the field were asked how long it would take before computers could deliver adequate translations of texts in a different language, the answer was ‘a few years’. 65 years have gone by since and machine translation still isn’t quite up to par with human translation.

Initially, machine translation was attempted through a rule based approach (building on the aforementioned success of the code-breaking efforts in WWII). Rules upon rules - and the exceptions to the rules - were committed to computers and it was believed the machines would eventually be able to grasp the subtlest of differences and translate with an according level of precision. This approach worked in clearly delimited environments, but a huge conceptual challenge was soon encountered. The rule based approach wasn’t able to account for context, e.g. ‘bear’ being either a noun or verb, depending on the situation the word is used. As a result of this doomed approach, research into machine translation was marginalised for a few decades.

It wasn’t until the advent of the personal computer, or more specifically, the increase in calculative power & the cost decrease of computer processors which allowed personal computers to become mainstream, that a new approach became feasible, that of the statistical approach. Instead of going through a huge set of rules, machine translation was now based on defining the meaning of a word by looking at the words surrounding it, and calculating the probability of meaning based on that. IBM pioneered this technology, but it wasn’t until Google started applying both its infinitely deep pockets and the huge amounts of data it has access to (for example, the parallel texts of the EU), that this statistical approach started really bearing (sic.) fruit.

In the last five years, the statistical approach has started to benefit from another breakthrough, deep learning through so called digital neural networks (DNNs). Often presented as a close approximation of the human brain, DNNs have “neurons” connected in software, and through learning, these connections can become stronger or weaker. The underlying principle has been known for decades, but it wasn’t until the recent discovery that GPUs (as opposed to CPUs) are very well equipped at doing the math behind these neural networks, that DNNS have made a big leap.

Many challenges still need to be overcome, but automated translation has definitely become a lot better, especially for languages from the same family (such as English, German & Dutch). According to The Economist’s article, discoveries from the last decade and a half “has shifted language technology from usable at a pinch to really rather good.” Still, I’m not too worried that when the push comes to the shove (according to Google translate), I’ll be able to out-translate a computer.