What on Earth Is That?

I graduated in computational linguistics. Along with that comes the fact that I have answered the question of what exactly it is that I study countless times. My favorite response used to be: “What do you think?”. That way, I usually found out that it was “probably… some… like, studying of programming languages?!”. Well, not quite:)

In this article, I would therefore like to shed some light on what computational linguistics actually is and what it meant to me.

When I wanted to give a brief answer to those asking, I used to say it was something between informatics and linguistics. They were usually still scratching their heads, so I added that it is about “teaching computers natural language, like machine translation, text summarization, large language models, and stuff”. At this point, a satisfied smile usually appeared, showing they had categorized my field somewhere, and the conversation moved on to another topic.

I was telling the truth but a highly simplified version of it.

A Bit of History

Computational linguistics falls under mathematical linguistics—a scientific field that applies mathematics, logic, and statistics to the study of language. The goal is to describe language as a system quantitatively and through the scientific method.

Computational linguistics is then the part that uses computers a computational power to process language—that is how simply the word “computational” made it into the name. in Czech, the term is “počítačová lingvistika” (computer linguistics)—and my personal opinion is that a better name would be “komputační lingvistika”, meaning it is about computing (statistics), not computers (as in the physical boxes), but what you can do.

It all started in the 1960s. The first tasks involved examining the frequency of linguistic phenomena and early attempts at machine translation (although a reality check soon followed, proving it to be harder than it seemed). The first corpora were also created, i.e., large, organized collections of language data stored electronically. Corpora allow the study of language on real data in a natural context and on a previously unthinkable scale. For linguistics, this was a big step from intuition to evidence-based science.

Rules Versus Statistics

Alongside data analysis, the main pillar of the field is Natural Language Processing (NLP), which is that very effort to convince machines to understand human speech. When solving problems in NLP (text generation, sentiment analysis, text summarization, authorship attribution), two approaches appear side by side:

  • rule-based system: We understand the regularities of the language and design corresponding rules to capture it.

  • statistical approach: Language is essentially just a sequence of characters that follow each other with a certain probability, so we just need to calculate that probability using big data.

The rule-based approach requires detailed linguistic knowledge, while the statistical approach does not require as much. Very roughly speaking, many different rule-based systems used to be used in the past, sometimes combined with statistics or classic machine learning. With the advent of large language models, and especially with the public release of ChatGPT in the autumn of 2022, a turning point arrived. It turned out that the “lots of data and lots of computing power” method solves all tasks in NLP comparably to or better than the most sophisticated specialized rule-based system.

It might seem that linguistics no longer has a place in NLP within the current world of AI. However, the opposite is true. For instance, it holds that “a model is only as good as its input data”, and someone also has to check the quality of the output; both of which require language expertise. The fact remains that being a computational linguist today means becoming someone who actively controls AI, rather than someone who will be replaced by it.

My Journey

I started my studies in 2021, so I lived through this historical turning point in real time. And I believe I did everything I could to stand on the side that directs and tames AI systems.

Computational linguistics at Masaryk University is unique because it falls under the Faculty of Arts, but some courses are guaranteed by the Faculty of Informatics. Thanks to this, I could take the best of both worlds: from the “arts” side, a humanities perspective, communication skills, and critical thinking; from informatics, an analytical approach and hard technical skills.

Thus, my timetable brought together language typology and the historical grammar of Czech with programming in Python and Java, graph theory, databases, and machine learning. I eventually took this interdisciplinarity the furthest while writing my diploma thesis, where I used large language models in the field of bioinformatics and genetics.

But more on that some other time.