Lecture 1 – Introduction to Natural Language Processing, History, Core Applications

Natural Language Processing is one of the most important areas inside Artificial Intelligence. Almost everything we do on the internet passes through some form of language processing. Search engines. Email filters. Social media platforms. Customer support chatbots. Digital assistants. All of these systems must handle text or speech produced by humans.

This first lecture provides a complete introduction to Natural Language Processing for computing students. The goal is to build a clear mental model of what NLP is. why it matters. how it developed over time. and how it is used in practical systems such as information retrieval, language translation and text classification.

The article assumes that the reader has basic programming knowledge in Python and has already studied introductory concepts of neural networks and deep learning. That background will be used in later lectures. Here we focus on concepts and intuition.

What Is Natural Language Processing ?

Natural Language Processing is the field of computer science that focuses on how computers can work with human languages such as English and Urdu. A simple definition that works well in exams and interviews is the following.

Natural Language Processing is the use of computational and statistical techniques to analyze, understand and generate human language in text and speech form.

This definition highlights four key ideas.

1. The input is a natural language. That means the ordinary language used in everyday communication. It is not a programming language.
2. The techniques are computational. Algorithms and data structures are required to handle large text collections and to implement models efficiently.
3. The techniques are statistical. Modern NLP relies heavily on probabilities, statistics and machine learning to learn patterns from data.
4. The output can be analysis or generation. Some systems only classify or label text. Others produce new text such as translations or summaries.

NLP lies at the intersection of several areas.

1. Computer science and software engineering.
2. Linguistics.
3. Probability, statistics and linear algebra.
4. Machine learning and deep learning.

Standard references such as Speech and Language Processing by Jurafsky and Martin and Foundations of Statistical Natural Language Processing by Manning and Schütze give a comprehensive treatment of these topics and are widely used in universities around the world.

If you need to revise deep learning concepts, review our deep learning lecture series.

Deep Learning

Why Natural Language Processing Matters ?

Natural language is the main communication channel for humans. Written text and spoken speech carry information about facts, opinions, emotions and intentions. Businesses, governments and individuals all generate huge amounts of such data every day.

Without NLP, a computer only sees text as a long string of characters. With NLP, the computer can start to recognise words, phrases, entities, topics and relationships. This enables many important real world applications.

1. Search engines. Given a short query, the system finds relevant web pages from billions of documents.
2. Email filtering. Incoming emails are automatically labelled as important, promotional or spam.
3. Social media analysis. Large volumes of posts and comments are analysed to understand trends and public opinion.
4. Customer support. Chatbots and virtual assistants answer common questions and route complex issues to human agents.
5. Translation tools. Users can read content in their own language even if the original source is in a different language.
6. Legal and medical text analysis. Long documents are processed to extract key facts, dates and entities.

For a computing student, NLP is a natural next step after courses on programming, algorithms and machine learning. It connects theory to visible and high impact applications. It also provides a strong foundation for research projects and industry jobs in data science and artificial intelligence.

Historical Evolution of Natural Language Processing

To understand current methods it is useful to review how the field has evolved. There are four broad stages.

3.1 Rule based systems

The earliest systems were almost entirely rule based. Linguists and programmers created large sets of handcrafted rules describing grammar and word patterns. For example a rule might state that if a word ends with the letters e d then it is likely to be a past tense verb. Another rule might specify that a noun phrase can consist of a determiner followed by adjectives and then a noun.

These systems were transparent. For any decision it was possible to point to the exact rule used. However they were hard to scale. Language is full of exceptions and irregularities. Everyday language in emails and social media contains spelling mistakes, abbreviations and slang. Maintaining a large rule set in such conditions becomes very expensive and brittle.

For production NLP pipelines, spaCy offers fast tokenisation, tagging and parsing.

3.2 Statistical methods

With the availability of large digital text collections and faster computers, the field moved towards statistical methods. Instead of writing rules manually, researchers started to estimate probabilities from data.

Typical ideas included the following.

1. N gram language models that estimate the probability of a word given the previous few words.
2. Hidden Markov Models for modelling sequences such as part of speech tags.
3. Probabilistic grammars for scoring alternative parse trees.

In this approach the knowledge about language is not coded directly by hand. Instead it is derived from the frequencies and patterns observed in corpora. The book by Manning and Schütze gives a detailed introduction to these statistical techniques and shows how they apply to tagging, parsing and information retrieval.

3.3 Neural and deep learning methods

The next major shift occurred when neural networks and deep learning became practical for large scale training. Word embeddings such as word2vec and GloVe represented words as dense vectors that captured semantic similarity. Recurrent neural networks and long short term memory networks handled long context better than simple N gram models. Sequence to sequence models enabled end to end tasks such as machine translation.

These developments allowed NLP systems to model complex patterns without manual feature engineering. Many tasks such as sentiment analysis and translation saw large improvements in accuracy.

3.4 Transformer and large language models

The introduction of the Transformer architecture brought another significant change. Transformer based models rely on self attention to capture relationships between all words in a sentence, not only neighbouring ones. This made training more parallel and efficient.

Large pre trained language models such as BERT and GPT demonstrated that a single model trained on massive corpora can perform many tasks with minimal fine tuning. The latest editions of core textbooks now devote entire chapters to transformer based models and their applications in information retrieval, question answering and text generation.

Levels of Language in NLP

Natural language can be analyzed at several levels. Understanding these levels helps to organize the topics that appear later in the course.

1. Phonetics and phonology (speech sounds). Study of speech sounds. Mainly important for speech recognition and synthesis. This level studies how words sound.
   It is about:
   individual sounds (like “b”, “p”, “sh”)
   pronunciation differences
   stress and tone
   Example
   The words:
   bat
   pat
   They differ only in one sound: b vs p
   Humans easily hear the difference.
   Speech recognition systems must learn this too.
   Why important in NLP?
   Mainly for:
   Speech recognition (converting speech → text)
   Text-to-speech (converting text → speech)
   Voice assistants (Siri, Alexa, Google Assistant)
   If the system cannot distinguish sounds correctly, it cannot transcribe correctly.
2. Morphology (word structure). Study of word structure. Words are made up of roots plus prefixes and suffixes. For example play, played and playing share the same root. Words are built from smaller meaningful parts.
   These parts are called:
   root (base meaning)
   prefix (before root)
   suffix (after root)
   Example
   Word: unhappiness
   Break it into:
   un- (not)
   happy (root)
   -ness (state)
   So:
   un + happy + ness
   Meaning:
   “state of not being happy”
   Another example:
   play
   played
   playing
   Same root → different grammatical forms.
   Why important in NLP?
   Morphology helps in:
   Stemming (reducing words to root form)
   Lemmatization
   Handling plural/singular
   Handling verb tenses
   Without morphology, the computer treats:
   play
   played
   playing
   as completely different words.
   With morphology, it knows they are related.
3. Syntax (sentence structure). Study of sentence structure and word order. For example the difference between the cat chased the mouse and the mouse chased the cat is captured at the syntactic level. Syntax is about word order and grammar structure.
   It answers:
   Who did what to whom?
   Example
   Sentence 1:
   The cat chased the mouse.
   Sentence 2:
   The mouse chased the cat.
   Same words.
   Different meaning.
   Why?
   Because of word order.
   Syntax studies:
   Subject
   Verb
   Object
   Phrase structure
   It helps computers understand:
   Which word is doing the action
   Which word is receiving the action
   Why important in NLP?
   Used in:
   Parsing
   Question answering
   Machine translation
   Grammar checking
   Without syntax, meaning becomes ambiguous.
4. Semantics (meaning). Study of literal meaning. Which entities are involved, what actions they perform and what roles they play. Semantics studies what the sentence actually means.
   It goes deeper than grammar.
   It answers:
   What entities are involved?
   What action is happening?
   What is the relationship?
   Example:
   Ali gave Sara a book.
   Semantics identifies:
   Giver → Ali
   Receiver → Sara
   Object → book
   Action → gave
   Syntax tells structure.
   Semantics tells meaning.
   Why important in NLP?
   Needed for:
   Information extraction
   Question answering
   Knowledge graphs
   Chatbots
   Without semantics, system may know structure but not real meaning.
5. Pragmatics (context meaning). Study of meaning in context. Includes politeness, sarcasm and implied meaning. Pragmatics studies what the speaker really means, not just literal words.
   It depends on:
   Situation
   Tone
   Social context
   Shared knowledge
   Example 1:
   “It’s cold here.”
   Literal meaning: temperature is low.
   Pragmatic meaning: “Please close the window.”
   Example 2 :
   After a terrible exam:
   “Wow, that went great.”
   Literal meaning: positive.
   Actual meaning: negative.
   Why important in NLP?
   Hardest level for machines.
   Needed for:
   Detecting sarcasm
   Understanding indirect requests
   Advanced chatbots
   Human-like conversation
   Large language models try to capture pragmatics.
6. Discourse (multi-sentence coherence). Study of how sentences connect to form a coherent text. Includes reference to earlier entities and continuity of topics. Discourse studies how multiple sentences connect together.
   It handles:
   Pronouns
   Topic flow
   References
   Coherence
   Example:
   Ahmed bought a car.
   He loves it.
   Questions:
   Who is “he”?
   What is “it”?
   Discourse analysis connects:
   he → Ahmed
   it → car
   Another example:
   Sara lost her phone. She was very upset.
   “She” refers to Sara.
   This is called coreference resolution.
   Why important in NLP?
   Used in:
   Document summarization
   Chatbots
   Story understanding
   Long document QA
   Dialogue systems
   Without discourse understanding, systems get confused in longer text.

Think of language like layers:

Level	Focus
Phonetics	Sounds
Morphology	Word structure
Syntax	Sentence structure
Semantics	Literal meaning
Pragmatics	Context meaning
Discourse	Multi-sentence connection

From smallest to biggest:

Sound → Word → Sentence → Meaning → Context → Conversation

This course mainly works with text. Therefore it focuses on morphology, syntax, semantics and discourse. Speech related topics appear only as background motivation.

For hands-on NLP in Python, the Natural Language Toolkit NLTK provides corpora and basic algorithms

Overview of Core NLP Tasks

NLP covers many tasks. In this introduction the focus is on three central tasks that will reappear throughout the course. information retrieval, language translation and text classification.

5.1 Information retrieval

Information retrieval systems accept a short user query and return a ranked list of documents that are likely to be relevant.

Consider a student who types convolutional neural networks for medical imaging into an academic search engine. Behind the interface, the system performs several operations.

1. Preprocesses all documents in the collection.
2. Preprocesses the query in the same way.
3. Represents documents and query as vectors in a high dimensional space.
4. Computes similarity scores between the query vector and each document vector.
5. Returns documents in decreasing order of similarity.

Natural Language Processing helps at multiple stages. Tokenisation ensures that words are detected correctly. Normalisation handles case and punctuation. Stopword removal and stemming can reduce noise. Later in the course, vector space models, language models and evaluation metrics such as precision and recall are studied in detail.

5.2 Language translation

Language translation systems convert a sentence or document from one language into another. For example an English news sentence can be translated into Urdu for local readers.

Traditional translation systems used rules and phrase tables. Modern systems use neural networks trained on large parallel corpora that contain sentence pairs in source and target languages.

Even without implementing such a system in this lecture, it is important to understand the components that are always present.

1. A parallel corpus that provides examples of correct translations.
2. A model that maps input sequences to output sequences.
3. A decoding algorithm that selects the most likely translation.
4. Evaluation metrics such as BLEU that compare system output with reference translations.

Later lectures revisit these concepts when discussing language models and sequence to sequence architectures.

5.3 Text classification

Text classification assigns a predefined label to a given piece of text. Common labels include spam or not spam, positive or negative sentiment, or topic categories such as sports, politics or technology.

A typical pipeline for text classification includes the following steps.

1. Collect a dataset of text documents with labels.
2. Preprocess the text using techniques from basic text processing.
3. Convert each document into a numeric representation such as bag of words or TF IDF.
4. Train a machine learning classifier such as Naive Bayes or logistic regression.
5. Evaluate the classifier using metrics such as accuracy, precision and recall.

Future lectures provide detailed mathematical treatment and code examples for these methods.

If you need to revise machine learning concepts, review our machine learning lecture series.

Machine Learning

A Simple NLP Pipeline as an Algorithm

Many NLP applications share a common pipeline. It is useful to think of it as a general algorithm template that can be specialised for particular tasks.

Step 1. Data collection.
Gather raw text data from sources such as web pages, files, chat logs or databases.

Step 2. Text cleaning.
Remove unwanted characters, markup tags and other noise. Standardise encoding and handle special symbols.

Step 3. Tokenization.
Split text into units. Typically sentences first and then words.

Step 4. Normalization.
Convert all text to a common form. This may include lowercasing, handling digits, removing punctuation and expanding contractions.

Step 5. Optional morphological processing.
Apply stemming or lemmatisation if appropriate for the task.

Step 6. Feature extraction or representation.
Convert tokens into a numeric form that models can understand. This may be bag of words counts, TF IDF values, or learned embeddings.

Step 7. Model training.
Train a statistical or neural model on labelled data when available. For example a classifier or a language model.

Step 8. Evaluation.
Use suitable metrics such as accuracy, precision, recall, F1 score, perplexity or BLEU, depending on the task.

Step 9. Deployment.
Integrate the trained model into a real system such as a search engine, chatbot or recommendation component. Monitor performance over time and update the model if the data distribution changes.

This algorithm will be refined and reused in many of the later lectures including those on text classification, sentiment analysis, information retrieval and question answering.

Summary

This first lecture introduced Natural Language Processing as a core area of Artificial Intelligence that focuses on human language. The article explained what NLP is, why it is important, how it has evolved from rule based systems to statistical models and then to neural and transformer based architectures. It presented the main linguistic levels relevant for text based processing and highlighted three central application areas. information retrieval, language translation and text classification.

You also saw a general NLP pipeline and a small Python example that demonstrates basic preprocessing and word frequency analysis. These ideas form the base on which later lectures will build more advanced topics such as corpora, edit distance, language models, tagging, parsing, semantics, sentiment analysis, information extraction, question answering and summarization.

Next. Lecture 2 – Text Preprocessing and Standard Corpora in NLP

People also ask: