192 lines
6.8 KiB
Markdown
192 lines
6.8 KiB
Markdown
# PDF Rename Sort
|
|
|
|
A Python web application built with Flask that automatically renames PDF files based on form field data extracted from within the PDFs and organizes them into a structured folder hierarchy. Designed for batch processing of fillable PDFs, extracting student information and creating organized archives.
|
|
|
|
## Features
|
|
|
|
- **Web-Based Interface**: Clean, responsive web UI for easy file uploads
|
|
- **Batch Processing**: Handle multiple PDFs or ZIP archives containing PDFs
|
|
- **Form Field Extraction**: Uses PyPDF2 to read PDF form fields
|
|
- **Intelligent Renaming**: Renames files to "LastName_FirstName.pdf" format
|
|
- **Structured Organization**: Creates Episode/School/Grade/Gender folder hierarchy
|
|
- **Duplicate Handling**: Automatically appends counters for duplicate names
|
|
- **ZIP Support**: Upload ZIP files and download processed results as ZIP with folder structure
|
|
- **Job Management**: Background processing with job status tracking and history
|
|
- **Authentication**: Microsoft Entra ID and local admin login support
|
|
- **Error Handling**: Gracefully skips files that can't be processed
|
|
|
|
## Prerequisites
|
|
|
|
- Python 3.6 or higher
|
|
- Virtual environment (recommended)
|
|
- Microsoft Entra ID (Azure AD) tenant with app registration
|
|
|
|
## Microsoft Entra ID Setup
|
|
|
|
**Detailed setup instructions**: See [ENTRA_SETUP.md](ENTRA_SETUP.md) for comprehensive step-by-step guide to configure Microsoft Entra ID authentication.
|
|
|
|
### Quick Setup:
|
|
1. Register application in Azure Portal (optional for admin login)
|
|
2. Get Tenant ID, Client ID, and Client Secret (for Entra login)
|
|
3. Set environment variables:
|
|
```bash
|
|
# For Entra ID login (optional)
|
|
export TENANT_ID="your-tenant-id"
|
|
export CLIENT_ID="your-client-id"
|
|
export CLIENT_SECRET="your-client-secret"
|
|
|
|
# For admin login
|
|
export ADMIN_USERNAME="trbcit"
|
|
export ADMIN_PASSWORD="atdrmbicn"
|
|
|
|
# Required
|
|
export SECRET_KEY="your-random-secret-key"
|
|
```
|
|
|
|
## Authentication
|
|
|
|
The application supports two authentication methods:
|
|
|
|
1. **Microsoft Entra ID**: OAuth2 authentication with Azure AD
|
|
2. **Admin Login**: Local username/password authentication
|
|
|
|
Users can choose their preferred login method on the login page.
|
|
|
|
## Installation
|
|
|
|
1. **Clone the repository**:
|
|
```bash
|
|
git clone https://gitlab.com/trbc-dev/pdf-rename-sort.git
|
|
cd pdf-rename-sort
|
|
```
|
|
|
|
2. **Create and activate virtual environment**:
|
|
```bash
|
|
python -m venv .venv
|
|
# On Windows:
|
|
& .venv/Scripts/Activate.ps1
|
|
# On macOS/Linux:
|
|
source .venv/bin/activate
|
|
```
|
|
|
|
3. **Install dependencies**:
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
## Usage
|
|
|
|
1. **Set environment variables** (see Entra ID Setup above)
|
|
|
|
2. **Run the application**:
|
|
```bash
|
|
python pdf_renamer.py
|
|
```
|
|
|
|
3. **Access the web interface**:
|
|
- Open `http://127.0.0.1:3000/`
|
|
- Authenticate using Microsoft Entra ID or admin login
|
|
|
|
4. **Upload files**:
|
|
- Select individual PDF files or ZIP archives containing PDFs
|
|
- Click "Process PDFs" - this creates a background job
|
|
|
|
5. **Monitor jobs**:
|
|
- Click "View Jobs" to see processing status and history
|
|
- Jobs show as "Queued", "Processing", or "Completed"
|
|
|
|
6. **Download results**:
|
|
- When complete, download the ZIP file from the jobs page
|
|
- Review any skipped files listed for each job
|
|
|
|
## How It Works
|
|
|
|
1. **File Upload**: Accepts PDFs or ZIPs via web form
|
|
2. **Extraction**: If ZIP, extracts PDFs to temporary directory
|
|
3. **Field Parsing**: For each PDF, reads form fields using PyPDF2:
|
|
- Student First Name
|
|
- Student Last Name
|
|
- Student Grade
|
|
- Student School
|
|
4. **Renaming**: Creates filename as "LastName_FirstName.pdf"
|
|
5. **Organization**: Creates folder structure:
|
|
```
|
|
Episode/
|
|
├── School Name/
|
|
│ ├── Grade Level/
|
|
│ │ ├── LastName_FirstName.pdf
|
|
│ │ └── LastName_FirstName_2.pdf
|
|
```
|
|
6. **Packaging**: Creates a ZIP archive preserving the folder structure
|
|
7. **Download**: Provides secure download link for the organized results
|
|
|
|
## Important Notes
|
|
|
|
- **PDF Compatibility**: The system works with both fillable PDFs (containing form fields) and flattened PDFs (filled forms that have been saved)
|
|
- **Required Information**: PDFs must contain student information for: "Student First Name", "Student Last Name", "Student Grade", "Student School", and "Gender"
|
|
- **Field Extraction**: For fillable PDFs, looks for form fields with exact names. For flattened PDFs, extracts information from the text content of the first page
|
|
- **Processing Requirements**: All five pieces of student information must be present for the PDF to be processed
|
|
- **Error Handling**: Files without the required information or with extraction errors will be skipped with detailed error messages
|
|
- **Security**: All processing happens server-side with temporary file storage
|
|
- **File Storage**: Processed ZIP files are stored in the `processed/` folder within the application directory
|
|
- **Limits**: Maximum upload size is limited to 500MB
|
|
- **Output**: Creates a structured folder hierarchy within the downloaded ZIP (Episode/School/Grade/Gender)
|
|
|
|
## Production Deployment
|
|
|
|
For production deployment on Ubuntu with reverse proxy (multi-domain setup):
|
|
|
|
1. **Run the automated installation script**:
|
|
```bash
|
|
# On your Ubuntu server (can coexist with other apps)
|
|
wget https://gitlab.com/trbc-dev/pdf-rename-sort/-/raw/main/ubuntu-install.sh
|
|
chmod +x ubuntu-install.sh
|
|
sudo ./ubuntu-install.sh
|
|
```
|
|
|
|
2. **Configure DNS**:
|
|
- Point a subdomain (e.g., `pdf-renamer.yourdomain.com`) to your server's IP
|
|
- This allows the app to run alongside your existing app on port 5000
|
|
|
|
3. **Configure environment variables**:
|
|
Edit `/etc/systemd/system/pdf-renamer.service` and update:
|
|
- `SECRET_KEY`: Generate a secure random key
|
|
- `TENANT_ID`, `CLIENT_ID`, `CLIENT_SECRET`: From Azure AD app registration
|
|
- `ADMIN_USERNAME` and `ADMIN_PASSWORD`: Set secure credentials
|
|
|
|
4. **Update domain configuration**:
|
|
Edit `/etc/nginx/sites-available/pdf-renamer` and replace `pdf-renamer.your-domain.com` with your actual subdomain
|
|
|
|
5. **Enable SSL (recommended)**:
|
|
```bash
|
|
sudo apt install certbot python3-certbot-nginx
|
|
sudo certbot --nginx -d your-subdomain.yourdomain.com
|
|
```
|
|
|
|
6. **Reload services**:
|
|
```bash
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl restart pdf-renamer
|
|
sudo systemctl reload nginx
|
|
```
|
|
|
|
The application will be accessible at your subdomain on ports 80 (HTTP) and 443 (HTTPS after SSL setup), running alongside your existing application.
|
|
|
|
## Development
|
|
|
|
To contribute or modify:
|
|
|
|
1. Fork the repository
|
|
2. Create a feature branch
|
|
3. Make your changes
|
|
4. Test thoroughly
|
|
5. Submit a pull request
|
|
|
|
## License
|
|
|
|
This project is licensed under the MIT License - see the LICENSE file for details.
|
|
|
|
## Support
|
|
|
|
For issues or questions, please open an issue on the GitLab repository.
|