NATURAL LANGUAGE PROCESSING

NATURAL  LANGUAGE  PROCESSING

INTRODUCTION :

Natural language processing (NLP) is a branch of Artificial Intelligence that helps computers understand , interpret and manipulate human language . NLP draws from many disciplines , including computer science and computational linguistics, in its pursuit to fill the gap between human communication and computer understanding .

EVOLUTION :

While natural language processing isn’t a new science , the technology is rapidly advancing thanks to an increased interest in human - to - machine communications , plus an availability of Big Data , powerful computing and enhanced Algorithms . 

As a human , you may speak and write in English , Spanish or Chinese . But a computer’s native language – known as machine code or machine language – is largely incomprehensible to most people . At your device’s lowest levels , communication occurs not with words but through millions of zeros and ones that produce logical actions . 

Indeed , programmers used punch cards to communicate with the first computers 70 years ago . This manual and arduous process was understood by a relatively small number of people . Now you can say , “ Alexa , I like this song ” and a device playing music in your home will lower the volume and reply , “ OK . Rating saved ” in a humanlike voice . Then it adapts its algorithm to play that song – and others like it – the next time you listen to that music station . 

Let’s take a closer look at that interaction . Your device activated when it heard you speak , understood the unspoken intent in the comment , executed an action and provided feedback in a well - formed English sentence , all in the space of about five seconds . The complete interaction was made possible by NLP , along with other AI elements such as Machine Learning and Deep Learning .

WHY IS NLP IMPORTANT :

Large volumes of textual data :

Natural language processing helps computers communicate with humans in their own language and scales other language-related tasks . For example , NLP makes it possible for computers to read text , hear speech , interpret it , measure sentiment and determine which parts are important . 

Today’s machines can analyze more language-based data than humans , without fatigue and in a consistent , unbiased way . Considering the staggering amount of unstructured data that’s generated every day , from medical records to social media , automation will be critical to fully analyze text and speech data efficiently .

Structuring a highly unstructured data source :

Human language is astoundingly complex and diverse. We express ourselves in infinite ways , both verbally and in writing . Not only are there hundreds of languages and dialects , but within each language is a unique set of grammar and syntax rules , terms and slang . When we write , we often misspell or abbreviate words , or omit punctuation . When we speak , we have regional accents , and we mumble , stutter and borrow terms from other languages . 

While supervised and unsupervised learning , and specifically deep learning , are now widely used for modeling human language , there’s also a need for syntactic and semantic understanding and domain expertise that are not necessarily present in these machine learning approaches . NLP is important because it helps resolve ambiguity in language and adds useful numeric structure to the data for many downstream applications , such as speech recognition or text analytics . 

WORKING :

Natural language processing includes many different techniques for interpreting human language , ranging from statistical and machine learning methods to rules - based and algorithmic approaches . We need a broad array of approaches because the text and voice based data varies widely , as do the practical applications . 

Basic NLP tasks include tokenization and parsing , lemmatization / stemming , part-of-speech tagging , language detection and identification of semantic relationships . If you ever diagramed sentences in grade school , you’ve done these tasks manually before . 

In general terms , NLP tasks break down language into shorter , elemental pieces , try to understand relationships between the pieces and explore how the pieces work together to create meaning .

These underlying tasks are often used in higher - level NLP capabilities , such as :

• Content categorization - A linguistic-based document summary , including search and indexing , content alerts and duplication detection .
• Topic discovery and modeling - Accurately capture the meaning and themes in text collections , and apply Advanced analytics to text , like optimization and forecasting .
• Contextual extraction - Automatically pull structured information from text - based sources .
• Sentiment analysis - Identifying the mood or subjective opinions within large amounts of text , including average sentiment and opinion mining . 
• Speech-to-text and text-to-speech conversion - Transforming voice commands into written text , and vice versa . 
• Document summarization - Automatically generating synopses of large bodies of text .
• Machine translation - Automatic translation of text or speech from one language to another .

In all these cases , the overarching goal is to take raw language input and use linguistics and algorithms to transform or enrich the text in such a way that it delivers greater value .

Written By - Ritesh Pandita  ©