Hello! Today I will be talking about one of the most hyped topics of AI and my personal favourite Natural Language Processing, in short, NLP. Before going through what all this hype is about, let’s see how the concept originated in the first place.
A brief history of NLP
In 1950, Alan Turing proposed the well-known ‘Turing Test’ in his paper, ‘Computing Machinery and Intelligence’, where he introduces a modified version of an ‘Imitation Game’. In this test, human judge ‘C’ should determine who among A (a machine) and B (a human) is a human, over interrogations. Any machine which would be able to fool the interrogator, would pass the test. Turing argues that, any machine with sufficient physical resources, could be programmed to give answers as close a human can give.
Trending Chatbot Tutorials
1. 🤖 How to talk to Computers: A Framework for building Conversational Agents — Part 1
2. Designing a chatbot for an improved customer experience
3. Top 5 NLP Chatbot Platforms
4. Chatbot, Natural Language Processing (NLP) and Search Services and how to mash them up for a better user experience
This is believed to be the foundation of NLP that we see today. Soon after this, several attempts were made in the field of NLP — structural transformation of natural language into machine readable format, complex rule based systems were created to make computers understand the natural language. Since these were not satisfactorily good enough, NLP and AI seemed to lose its charm. By the 1980s, with the scope of machine learning increasing, NLP started shifting towards machine learning too.
In 1985, Terry Sejnowski created a neural system which could learn how to pronounce English words. Today, a tremendous amount of work is being done in the field of NLP using Deep Neural Networks or Machine Learning in general, where we are able to create state-of-the-art models in text classification, QnA generation, Sentiment Classification, etc.
Why is NLP important?
In simple words, Natural Language Processing involves machines learning to understand language that humans speak, analyse it, manipulate it and give intended results.
Well, ‘Processing’ is a broad term. In order to understand the role of NLP, we will have to look into some scenarios -
- Understanding Natural Language
Firstly, the most obvious point — Natural Language is the language that we speak. So if a machine can understand the language we speak, it makes the interaction between a machine and a human, much smoother.
- Analyse huge amount of data, efficiently
Imagine the amount of textual and speech data we have over the internet today — A lot!!! A lot of information is always good, but it is obvious that allocating human resources for processing such an enormous amount of data isn’t really practical. NLP has certainly automated the process of analysing and extracting relevant information out of large volumes of data.
- Convert unstructured data into any required structure
Suppose, the market strategist of a company wants to know how people have reacted to their new promotional event. People will have expressed their views over any social media or other internet platforms out there. What does the task include in general — The following
- Collect all the comments relevant to this promotional event from internet resources.
- Identify various levels of sentiments/reactions which could be used to classify the data.
- Classify the comments into the above various levels of sentiments/reactions.
- Find statistics for each of the reactions to get an overall view of data
When you observe the above steps, each of the above steps needs NLP in one or the other way since we are ultimately dealing with the text.
Applications of NLP
Since NLP itself is a vast field, the way concepts of NLP are being used in various domains is enormous. Here I have listed out some of the most important applications of NLP, which are being used in a variety of projects today!
- Language Translation — Whether you want to understand a catchy song from an unknown language or write ceremonial speech in another language, translators are something we all would want to have. Before machine translation was a thing, human translators were all we had. Our scholars realised the necessity of machine translation pretty early and that is why translation is one of the oldest applications of NLP. Today we have Google Translate, Bing Translator, etc providing us with free translation services. Many applications allow us to translate conversations in the app, for example — Facebook.
- Question and Answering — Amazed at seeing how conversational chatbots can answer you instantly when you ask them something? QnA maker and Answer Retrieval, has been two of the most trending research topics in the field of NLP today. You must have seen how Gmail suggests replies to the mails you receive — this is nothing but automatic answer generation.
- Speech to Text conversion — We all have used Siri, Google Assistant, etc. When we give voice commands to such bots, these commands are first converted into text and then processed to perform further actions.
- Language Modeling — Language model is probabilistic distribution over phrases/words. In simple words, it is the numerical representation of one or more languages where the semantics of the languages are preserved. Thanks to the Language Models, we are able to produce state-of-the-art results in various tasks of NLP
- Grammar/Spelling correction — While I’m writing this blog, I can’t ignore to notice how good the spelling and grammar corrector does Google Docs have and how overwhelmingly useful it is. Most of the NLP applications have been using spelling and grammar correction techniques which could result in better accuracy for other NLP tasks.
- Text classification — Text Classification finds it usage in almost any NLP related tasks — sentiment classification, anomaly detection, intent classification, etc.
- Document summarization — Document Summarization as the title explains,, refer to the generation of a shorter document without loss of any useful data. It helps primarily in indexing documents by using unbiased summarization unlike humans.
- Sentiment Analysis — Market Analysis for a retail company, Analysis of Social Media Trends, Public Opinions on ruling government are some of the examples where sentiment analysis is primarily required. Looking at the above applications of Sentiment Analysis, we can argue that while it can brands/companies in improving customer service, planning marketing strategies, product designing,etc., it can even help politicians to plan their campaigning strategies. Haha, this is why AI is for everyone, isn’t it? :D
I hope, after reading this blog, you have quite an idea of how versatile NLP is! Do follow our blog page for more such insights on AI and Conversational AI Chatbots. Also feel free to get in touch with us at curiousmuch@asksid.ai or visit our website www.asksid.ai to start a conversation.
(This article is attributed to my colleague Neha M)
Don’t forget to give us your 👏 !
What is Natural Language Processing? was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.