Natural language processing: Introduction

But the truth is, NLP is far from a new field. The human desire for computers to understand our language has existed since their inception. Yes, those old computers that could barely run multiple programs at the same time did get to know the complexity of natural languages.

A natural language is any human language such as English, Arabic, Russian, etc. How difficult it is to make a computer understand natural languages ​​depends on their structure. Moreover, when we speak, we often pronounce words in different ways, our accents are different, regardless of whether we use our native language or a foreign one. We also often tend to chew on words as we speak to get the message across faster, not to mention all the slang words that pop up every day. The purpose of this article is to shed light on the history of natural language processing and its subsections.

The beginning of the development of NLP

Natural language processing is an interdisciplinary field at the intersection of computer science and linguistics. There are endless ways to put words together to form a sentence. Of course, not all of these sentences will be grammatically correct or even make sense. Read more here: https://doctranslator.com/

People can tell them apart, but computers cannot. Moreover, it is unrealistic to load into it a dictionary with all possible sentences in all possible languages. In the early stages, scientists proposed to divide any sentence into a set of words that can be processed individually, which is much easier than processing the entire sentence. This approach is similar to that used to teach a new language to children and adults. When we just start learning a language, we are introduced to its parts of speech. Let`s take English as an example. It has 9 main parts of speech: nouns, verbs, adjectives, adverbs, pronouns, articles, etc. These parts of speech help to understand the purpose of each word in a sentence.

Defining the category

However, it is not enough to know the category of a word, especially for those that may have more than one meaning. For example, the word “leaves” can be the form of the verb “to leave” or the plural form of the noun “leaf”.

Therefore, computers need a basic understanding of grammar in order to refer to it in case of confusion. Thus, the rules for the structure of phrases appeared. They are a set of grammar rules by which a sentence is built. In English, it is formed using a noun phrase and a verb group. Consider the sentence “Anne ate the apple”. Here "Anne" is a noun phrase, and "ate the apple" is a verb phrase.

Different sentences are formed using different structures. As you increase the number of phrase structure rules, you can create a parsing tree to classify each word in a specific sentence and arrive at its overall meaning: Wiki