nextgenpdf/README.md

# PDF Rename Sort

A Python web application built with Flask that automatically renames PDF files based on form field data extracted from within the PDFs and organizes them into a structured folder hierarchy. Designed for batch processing of fillable PDFs, extracting student information and creating organized archives.

## Features

- **Web-Based Interface**: Clean, responsive web UI for easy file uploads
- **Batch Processing**: Handle multiple PDFs or ZIP archives containing PDFs
- **Form Field Extraction**: Uses PyPDF2 to read PDF form fields
- **Intelligent Renaming**: Renames files to "LastName_FirstName.pdf" format
- **Structured Organization**: Creates Episode/School/Grade/Gender folder hierarchy
- **Duplicate Handling**: Automatically appends counters for duplicate names
- **ZIP Support**: Upload ZIP files and download processed results as ZIP with folder structure
- **Job Management**: Background processing with job status tracking and history
- **Authentication**: Microsoft Entra ID and local admin login support
- **Error Handling**: Gracefully skips files that can't be processed

## Prerequisites

- Python 3.6 or higher
- Virtual environment (recommended)
- Microsoft Entra ID (Azure AD) tenant with app registration

## Microsoft Entra ID Setup

**Detailed setup instructions**: See [ENTRA_SETUP.md](ENTRA_SETUP.md) for comprehensive step-by-step guide to configure Microsoft Entra ID authentication.

### Quick Setup:
1. Register application in Azure Portal (optional for admin login)
2. Get Tenant ID, Client ID, and Client Secret (for Entra login)
3. Set environment variables:
   ```bash
   # For Entra ID login (optional)
   export TENANT_ID="your-tenant-id"
   export CLIENT_ID="your-client-id"
   export CLIENT_SECRET="your-client-secret"
   
   # For admin login
   export ADMIN_USERNAME="trbcit"
   export ADMIN_PASSWORD="atdrmbicn"
   
   # Required
   export SECRET_KEY="your-random-secret-key"
   ```

## Authentication

The application supports two authentication methods:

1. **Microsoft Entra ID**: OAuth2 authentication with Azure AD
2. **Admin Login**: Local username/password authentication

Users can choose their preferred login method on the login page.

## Installation

1. **Clone the repository**:
   ```bash
   git clone https://gitlab.com/trbc-dev/pdf-rename-sort.git
   cd pdf-rename-sort
   ```

2. **Create and activate virtual environment**:
   ```bash
   python -m venv .venv
   # On Windows:
   & .venv/Scripts/Activate.ps1
   # On macOS/Linux:
   source .venv/bin/activate
   ```

3. **Install dependencies**:
   ```bash
   pip install -r requirements.txt
   ```

## Usage

1. **Set environment variables** (see Entra ID Setup above)

2. **Run the application**:
   ```bash
   python pdf_renamer.py
   ```

3. **Access the web interface**:
   - Open `http://127.0.0.1:3000/`
   - Authenticate using Microsoft Entra ID or admin login

4. **Upload files**:
   - Select individual PDF files or ZIP archives containing PDFs
   - Click "Process PDFs" - this creates a background job

5. **Monitor jobs**:
   - Click "View Jobs" to see processing status and history
   - Jobs show as "Queued", "Processing", or "Completed"

6. **Download results**:
   - When complete, download the ZIP file from the jobs page
   - Review any skipped files listed for each job

## How It Works

1. **File Upload**: Accepts PDFs or ZIPs via web form
2. **Extraction**: If ZIP, extracts PDFs to temporary directory
3. **Field Parsing**: For each PDF, reads form fields using PyPDF2:
   - Student First Name
   - Student Last Name
   - Student Grade
   - Student School
4. **Renaming**: Creates filename as "LastName_FirstName.pdf"
5. **Organization**: Creates folder structure:
   ```
   Episode/
   ├── School Name/
   │   ├── Grade Level/
   │   │   ├── LastName_FirstName.pdf
   │   │   └── LastName_FirstName_2.pdf
   ```
6. **Packaging**: Creates a ZIP archive preserving the folder structure
7. **Download**: Provides secure download link for the organized results

## Important Notes

- **PDF Compatibility**: The system works with both fillable PDFs (containing form fields) and flattened PDFs (filled forms that have been saved)
- **Required Information**: PDFs must contain student information for: "Student First Name", "Student Last Name", "Student Grade", "Student School", and "Gender"
- **Field Extraction**: For fillable PDFs, looks for form fields with exact names. For flattened PDFs, extracts information from the text content of the first page
- **Processing Requirements**: All five pieces of student information must be present for the PDF to be processed
- **Error Handling**: Files without the required information or with extraction errors will be skipped with detailed error messages
- **Security**: All processing happens server-side with temporary file storage
- **Limits**: Maximum upload size is limited to 500MB
- **Output**: Creates a structured folder hierarchy within the downloaded ZIP (Episode/School/Grade/Gender)

## Production Deployment

For production deployment on Ubuntu with reverse proxy (multi-domain setup):

1. **Run the automated installation script**:
   ```bash
   # On your Ubuntu server (can coexist with other apps)
   wget https://gitlab.com/trbc-dev/pdf-rename-sort/-/raw/main/ubuntu-install.sh
   chmod +x ubuntu-install.sh
   sudo ./ubuntu-install.sh
   ```

2. **Configure DNS**:
   - Point a subdomain (e.g., `pdf-renamer.yourdomain.com`) to your server's IP
   - This allows the app to run alongside your existing app on port 5000

3. **Configure environment variables**:
   Edit `/etc/systemd/system/pdf-renamer.service` and update:
   - `SECRET_KEY`: Generate a secure random key
   - `TENANT_ID`, `CLIENT_ID`, `CLIENT_SECRET`: From Azure AD app registration
   - `ADMIN_USERNAME` and `ADMIN_PASSWORD`: Set secure credentials

4. **Update domain configuration**:
   Edit `/etc/nginx/sites-available/pdf-renamer` and replace `pdf-renamer.your-domain.com` with your actual subdomain

5. **Enable SSL (recommended)**:
   ```bash
   sudo apt install certbot python3-certbot-nginx
   sudo certbot --nginx -d your-subdomain.yourdomain.com
   ```

6. **Reload services**:
   ```bash
   sudo systemctl daemon-reload
   sudo systemctl restart pdf-renamer
   sudo systemctl reload nginx
   ```

The application will be accessible at your subdomain on ports 80 (HTTP) and 443 (HTTPS after SSL setup), running alongside your existing application.

## Development

To contribute or modify:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request

## License

This project is licensed under the MIT License - see the LICENSE file for details.

## Support

For issues or questions, please open an issue on the GitLab repository.