We edited these materials from the FastAI NLP Course, so full credit to them. Here is the repo: https://github.com/fastai/course-nlp We also referenced a blog from Algorithma: https://algorithmia.com/blog/introduction-natural-language-processing-nlp
Natural language processing (NLP) is a field of artificial intelligence in which computers analyze, understand, and derive meaning from human language in a smart and useful way. By utilizing NLP, developers can organize and structure knowledge to perform tasks such as automatic summarization, translation, named entity recognition, relationship extraction, sentiment analysis, speech recognition, and topic segmentation.
Some examples of NLP are with common products such as:
Besides common word processors, NLP considers the hierarchical structure of language.
John Rehling, an NLP expert at Meltwater Group, said in How Natural Language Processing Helps Uncover Social Media Sentiment. “By analyzing language for its meaning, NLP systems have long filled useful roles, such as correcting grammar, converting speech to text and automatically translating between languages.”
NLP is used to analyze text, allowing machines to understand how humans speak. This human-computer interaction enables real-world applications like automatic text summarization, sentiment analysis, topic extraction, named entity recognition, parts-of-speech tagging, relationship extraction, stemming, and more. NLP is commonly used for text mining, machine translation, and automated question answering.
NLP is very difficult in computer science because human language is rarely precise or plainly spoken. To understand human language is to understand not only the words, but the concepts and how they’re linked together to create meaning. Despite language being one of the easiest things for the human mind to learn, the ambiguity of language is what makes natural language processing a difficult problem for computers to master.
NLP algorithms have a variety of uses. Basically, they allow developers to create a software that understands human language. Due to the complicated nature of human language, NLP can be difficult to learn and implement correctly. However, with the knowledge gained from this article, you will be better equipped to use NLP successfully. Some of the projects developers can use NLP algorithms for are:
Many techniques from NLP are useful in a variety of places, for instance, you may have text within your tabular data.
There are also interesting techniques that let you go between text and images:
Historically, NLP originally relied on hard-coded rules about a language. In the 1990s, there was a change towards using statistical & machine learning approaches, but the complexity of natural language meant that simple statistical approaches were often not state-of-the-art. We are now currently in the midst of a major change in the move towards neural networks. Because deep learning allows for much greater complexity, it is now achieving state-of-the-art for many things.
This doesn't have to be binary: there is room to combine deep learning with rules-based approaches.
https://www.youtube.com/watch?v=5ctbvkAMQO4 Stop at 5:25
Books
Here are a few helpful references if you want to get really in-depth: (Note, these are really advanced)
Speech and Language Processing, by Dan Jurafsky and James H. Martin (free PDF)
Introduction to Information Retrieval by By Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze (free online)
Natural Language Processing with PyTorch by Brian McMahan and Delip Rao (need to purchase or have O'Reilly Safari account)
Blogs
Good NLP-related blogs: