Natural Language Processing

A Short Introduction

Language is meant for communicating about the world. By studying language, we can come to understand more about the world. If we can succeed in building a computational model of language, we will have a powerful tool for communicating about the world.

Natural Language Processing (NLP) refers to AI method of communicating with intelligent systems using a natural language.

Natural language processing is a field of computer science concerned with the interactions between computers and human (natural) languages. The goal of the Natural Language Processing (NLP) group is to design and build software that will analyze, understand, and generate languages that humans use naturally, so that eventually you will be able to address your computer as though you were addressing another person. Natural language generation systems convert information from computer databases into readable human language. Natural language understanding systems convert samples of human language into more formal representations that are easier for computer programs to manipulate.

The term natural language is used to distinguish human languages (such as Spanish, English or Swedish) from formal or computer languages (such as C++ or Java).

The field of NLP involves making computers to perform useful tasks with the natural languages humans use. The input and output of an NLP system can be:

Speech

Written Text

Largest part of human linguistic communication occurs as speech and written language being a recent invention plays a less central role. But processing written language is easier than processing speech. To build a program that understands spoken language, we need all the facilities of written language understanding as well as enough additional knowledge to handle noise and ambiguities of the audio signal.

Thus it is useful to divide the entire language processing problem into two tasks:

• Processing written text, using lexical, syntactic and semantic knowledge of the language as well as the required real world information.

• Processing spoken language, using all the information needed above plus the additional knowledge about phonology, as well as enough added informal ion to handle the further ambiguities that arise in speech.

Components of NLP

There are two components of NLP as given −

Natural Language Understanding (NLU)

Understanding involves the following tasks :

• Mapping the given input in natural language into useful representations: We think of language as a pair (source language, target representation), together with a mapping between elements of each to the other. Thus understanding a piece of language involves mapping it into some representation appropriate to a particular situation.

• Analyzing different aspects of the language.

Natural Language Generation (NLG)

It is the process of producing meaningful phrases and sentences in the form of natural language from some internal representation. It involves :

• Text planning − It includes retrieving the relevant content from the knowledge base.

• Sentence planning − It includes choosing required words, forming meaningful phrases, setting tone of the sentence.

• Text Realization − It is mapping sentence plan into sentence structure.

Steps in NLP

The steps in Natural Language Processing are as follows:

1. Morphological Analysis: Individual worlds are scrutinized into their components and non-word tokens, like punctuation are alienated from the words.

2. Syntactic Analysis: Linear sequences of words are transformed into structures that illustrate how the words associate to each other. Some word sequences may be discarded if they disobey the language's rules for how words may be united. The sentence such as “The school goes to boy” is rejected by English syntactic analyzer.

3. Semantic Analysis: The structures generated by the syntactic analyzer are allocated meanings. It draws the exact meaning or the dictionary meaning from the text. The text is checked for meaningfulness. It is done by mapping syntactic structures and objects in the task domain. The semantic analyzer disregards sentences such as “hot ice-cream”. Example: “Colourless green ideas sleep furiously” is semantically anomalous.

4. Discourse Integration: The meaning of an individual sentences may rely on the sentences that precede it and may affect the meanings of the sentence (may rely on the sentences that come first) that follow it. The sentence “John wanted it” depends on the prior discourse context while the word “John” may influence the meaning of later sentences like “He always had”.

5. Pragmatic Analysis: The structure displaying what was said is reinterpreted to verify that what was in fact meant. It involves deriving those aspects of language which require real world knowledge. For example, the sentence “Do you know what time it is?” should be interpreted as a request to be told the time.

The boundaries between these 5 phases are often very fuzzy. They may be performed in a sequence or they are performed all in once. If they are performed in sequence, one may need to appeal for assistance to another.

To make overall language understanding problem traceable, it will help if we distinguish between the following two ways of decomposing a program:

• The process and the knowledge required to perform the task

• The global control structure that is imposed on those processes.

References

[1] AI - Natural Language Processing. Tutorialspoint. (n.d.). https://www.tutorialspoint.com/artificial_intelligence/artificial_intelligence_natural_language_processing.htm.

[2] Rich, E., & Knight, K. (1991). Artificial Intelligence. McGraw-Hill, Inc.