In today's world of the 21st century, According to the industry estimates, only 21% of the available data is in the structured format.

Loads of data are being generated as we speak, as we tweet, as we send messages on WhatsApp, Facebook, Instagram all through text messages. And the majority of this data exists in the textual form which is unstructured in nature.

In order to produce significant and actionable insights from the text data, it is important to get acquainted with the techniques of text analysis. So now let's understand what is text analysis.

Text Analysis or Text Mining?

Text Analysis is the process of deriving meaningful information from natural language text. Text mining usually involves the process of structuring the input text, driving patterns within the structured data, and finally evaluating to interpret the output.

Compared with the kind of data stored in the database, the text is unstructured, amorphous, and difficult to deal with algorithmically. But, In the modern culture text is the most common vehicle for the former exchange of information.

Now as the text analysis refers to the process of driving high-quality information from text, the overall goal here is to turn the text into data for analysis and this is done by the application of NLP or natural language processing.

What is Natural Language Processing?

NLP refers to the AI method of communicating with an intelligent system using natural language. (Usually, dealing with human languages)

By utilizing NLP and its components one can organize the massive chunks of textual data perform automated tasks and solve a wide range of problems such as

• Automatic Summarisation
• Machine Translation
• Name Entity Recognition
• Speech Recognition and
• Topic Segmentation

Applications of NLP

There are various applications of NLP,

• First of all, we have Sentiment Analysis, this field NLP is used heavily,
• Speech Recognition, now here we are also talking about voice assistants like Google Assistant, Cortana and Siri.
• In the implementation of a chatbot - which you might have used the customer care chat services of any app. It also uses NLP to process the data entered and provide the response based on the input.
• Machine Translation is also another use case of natural language processing. The most common example would be the Google translate it uses NLP and translates the data from one language to another and that too in real-time.
• The other applications of NLP include spell checking, keyword search which is also a big field where NLP is used.
• Extracting Information from any particular website for any particular document is also a use case of NLP.
• And one of the coolest applications of NLP is Advertisement Matching which is nothing but the recommendation of the ads based on your history.

Components of NLP

NLP is divided into two major components i.e.,

Natural Language Understanding (NLU)

NLU can digest a text, translate it into computer language, and produce an output in a language that humans can understand.
It involves tasks like,

• Mapping the given input into natural language into useful representations.
• Analysing different aspects of the language

Natural Language Generation (NLG)

NLG is the process of producing meaningful phrases and sentences in the form of natural language. It involves,

• Text Planning
• Sentence Planning and
• Text Realisation

Did you ever think about how whip-smart the machine is to understand the diverse languages that we use ... But under the hood, it faces many difficulties for understanding the languages.

Let's recognize the difficulties that machines face in understanding our language (and appreciate them ;P) in my next post.

till then...

Stay Peachy!!