Text Classification Model with Attention Mechanism NLP

Project Overview

This project implements an attention mechanism for multiclass text classification, focusing on dynamically highlighting relevant features of the input data. The model gives more weightage to the most relevant words in a sentence to make predictions, building upon previous work with Naïve Bayes, skip-gram, RNN, and LSTM models.

Business Objective

To perform multiclass text classification on a dataset of customer complaints about consumer financial products, utilizing the power of attention mechanisms to improve classification accuracy.

Tech Stack

• Language: Python
• Libraries: pandas, torch, nltk, numpy, pickle, re, tqdm, sklearn
• Pre-trained word vectors: GloVe (glove.6B)

Approach

1. Install necessary packages
2. Import required libraries
3. Define configuration file paths
4. Process GloVe embeddings
5. Process text data
6. Perform data preprocessing
7. Create attention model
8. Implement training and testing functions
9. Develop prediction functionality for new text

Key Components

• Data preprocessing pipeline
• Custom PyTorch dataset implementation
• Attention mechanism model architecture
• Training and evaluation scripts
• Modular code structure for maintainability

Project Structure

The project follows a modular structure with separate directories for input data, source code, and output:

• Input: Contains the dataset and pre-trained embeddings
• Source: Includes modularized Python files (model.py, data.py, utils.py)
• Output: Stores trained models and preprocessed data
• Configuration files and main execution scripts