Natural Language Processing

Natural language processing (NLP) is a revolutionary technology that enables machines to understand and communicate with human language, using AI.

How can a computer, which usually understands a precise, marked out and structured programming language, understand the imprecise and ambiguous human language?

 

Definition

 

For François Yvon, a researcher specialising in NLP, this technology means " all research and development aimed at modelling and reproducing, with the help of machines, the human capacity to produce and understand linguistic statements for communication purposes ».

 

The origin of the NLP : 

 

The ability to understand human language is not a new concept; as early as 1950, the famous mathematician Alan Turing proposed a test that assessed the intelligence of a machine by its ability to hold a human conversation. 

It was during the 1950s that the first NLP tests were carried out. Firstly, by the American government, which wanted to decipher Soviet communications during the Cold War. Then in 1954, by theGeorgetown University and IBM who translated some 60 Russian sentences into English; conducting the first influential demonstration of machine translation. 

Nevertheless, the resources invested remain very small and the results insignificant. Indeed, these first trials are based on a facts and rules that present difficulties in understanding meaning and dealing with the contextualisation and ambiguities of human language. 

NLP was widely criticised at the time and only really took off with the arrival of machine learning and deep learning. 

 



 

Natural Language Processing techniques:

 

There are several ways to capture human language:  from already digitised texts or contents in an image, in a manuscript, by voice recognition or by extracting information from web pages, as search engines do.

The two main techniques used for natural language processing are syntactic analysis and semantic analysis. 

 

Syntactic analysis

Which itself includes : 

Parsing : which identifies the grammatical rules of a sentence and deciphers its meaning

Word segmentation : which splits the text into units

Morphological segmentation: which breaks words into groups

 

Semantic analysis

The process of analysing the context of a sentence to understand its meaning.



What are the areas of application of NLP? 

 

Today, natural language processing is one of the main drivers of AI and covers a very wide range of applications: 

Sentiment analysis: this involves identifying subjective information in a text to extract the author's opinion. This makes it possible to measure the level of satisfaction of customers and users.

Chatboat: this is a software robot that will dialogue with an individual or a consumer through an automated conversation service. Chatbots work in several stages: understanding the question, recognising the words, determining the meaning and context, making the right decision, formulating the answer.

Machine translation: aMachine translation algorithms that automate translation.

Text classification: this involves organising, structuring and categorising a set of texts. It is used in particular in the context of content moderation, for example to detect fake news, where the NLP will analyse keywords and compare articles with those of reliable sources to assess the credibility of information. 

but also :

 

Character recognition

Automatic correction

Automatic summary

 

The limits of technology

 

Although NLP is a major axis of artificial intelligence and this technology is now revolutionising certain business issues such as customer service, it is nevertheless faced with various challenges. The understanding of human language remains complex due to : 

Ambiguity: the different meanings of the same word according to a given context, grammar...

Synonymy: there are over 44,000 synonyms in the French language

Writing styles: depending on the author's emotions and intentions, the same idea can be expressed in different ways. The NLP will find it very difficult to discern sarcasm or irony. 

Human language is infinitely complex, even insoluble, and the machine still needs Man to grasp what remains elusive. 

 

Speech Martin Luther King
Martin Luther King delivers one of the most important speeches in history: 'I have a dream' (1963)

 

Linking artificial and human intelligence: how isahit works 

At isahit, we have chosen to bridge the gap between humans and machines by implementing a technological and agile platform of artificial intelligence, augmented by human intelligence.  

This choice of operation allows us to respond to the challenges of NLP and to support our clients by guaranteeing a real understanding of the technology.

We have developed our own tool in order to better support our clients, who, like L'Oréal for example, have called on us to better qualify their data (discover nour use case with L'Oréal).

If you want to know more, please visit our dedicated NLP page Yes make an appointment with our experts.