Update TODO.md
This commit is contained in:
nmemmert authored and 
GitHub committed
1 parent
309a6723d7
commit
03e6d6b628
1 file changed
+103
-60
@@ -1,89 +1,132 @@
|
|||||||
# Self-Hosted Photo Manager — TODO
|
# Plex for Photos — Self-Hosted Photo Library Architecture
|
||||||
|
|
||||||
## Architecture
|
A self-built media server for your photo collection, running as another container in your homelab. Follows the same pattern as your existing study app: React/Vite frontend + Express/SQLite backend, files stay on disk, DB just indexes them.
|
||||||
|
|
||||||
A self-built alternative to Immich: React/Vite frontend + Express/SQLite backend,
|
---
|
||||||
files stay on disk, DB indexes them, thumbnails are the only derived asset.
|
|
||||||
|
## High-Level Flow
|
||||||
|
|
||||||
```
|
```
|
||||||
┌───────────────────┐ watches ┌──────────────────────┐
|
┌───────────────────┐ watches ┌──────────────────────┐
|
||||||
│ Photo folder(s) │ ─────────────────────▶ │ Ingestion Worker │
|
│ Photo Library │ ─────────────────────▶ │ Ingestion Engine │
|
||||||
│ (disk / NAS) │ │ (chokidar file watch) │
|
│ (disk / NAS) │ │ (chokidar file watch)│
|
||||||
└───────────────────┘ └───────────┬────────────┘
|
└───────────────────┘ └───────────┬───────────┘
|
||||||
│ extract EXIF,
|
│ extract EXIF,
|
||||||
│ generate thumbnail,
|
│ generate poster,
|
||||||
│ compute pHash
|
│ compute pHash
|
||||||
▼
|
▼
|
||||||
┌──────────────────────┐
|
┌──────────────────────┐
|
||||||
│ SQLite metadata DB │
|
│ SQLite metadata DB │
|
||||||
│ (photos, tags, │
|
│ (photos, collections,│
|
||||||
│ albums, hashes) │
|
│ hashes, artwork) │
|
||||||
└───────────┬────────────┘
|
└───────────┬───────────┘
|
||||||
│
|
│
|
||||||
▼
|
▼
|
||||||
┌──────────────────────┐
|
┌──────────────────────┐
|
||||||
│ Express API server │
|
│ Express API server │
|
||||||
└───────────┬────────────┘
|
└───────────┬───────────┘
|
||||||
│ REST
|
│ REST/WS
|
||||||
▼
|
▼
|
||||||
┌──────────────────────┐
|
┌──────────────────────┐
|
||||||
│ React/Vite frontend │
|
│ React/Vite frontend │
|
||||||
│ (timeline, albums, │
|
│ (browse, collections,│
|
||||||
│ search, map) │
|
│ discover, map) │
|
||||||
└──────────────────────┘
|
└──────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
## Build Order
|
Original files never move or duplicate — the DB just points to paths on disk. Posters (thumbnails) are the only generated/derived asset stored separately (for instant browsing and discovery views).
|
||||||
|
|
||||||
- [ ] **1. Ingestion worker + SQLite schema** — watch folder, extract EXIF, generate thumbnails, populate DB
|
---
|
||||||
- [ ] **2. Basic API + timeline UI** — paginated grid, chronological browsing (usable on its own)
|
|
||||||
- [ ] **3. Detail view + manual tagging/albums**
|
|
||||||
- [ ] **4. Dedupe detection + review UI** — pHash comparison, side-by-side review, keep/delete
|
|
||||||
- [ ] **5. Search + filters** — date range, camera model, tag
|
|
||||||
- [ ] **6. Map view** — Leaflet/MapLibre pins from GPS EXIF
|
|
||||||
- [ ] **7. Face grouping** — only if wanted after v1–v5 (biggest scope jump, treat as v2+)
|
|
||||||
|
|
||||||
## Stack
|
## Components
|
||||||
|
|
||||||
| Layer | Tech |
|
### Ingestion Engine
|
||||||
|---|---|
|
|
||||||
| File watching | `chokidar` (persistent) or cron scan |
|
|
||||||
| EXIF extraction | `exifr` or `exiftool` wrapper |
|
|
||||||
| Thumbnails | `sharp` (1–2 sizes: grid thumb + preview) |
|
|
||||||
| Dedupe hashing | pHash via `sharp` + hashing lib, or `blockhash-core` |
|
|
||||||
| Database | SQLite via `better-sqlite3` |
|
|
||||||
| API | Express |
|
|
||||||
| Frontend | React + Vite |
|
|
||||||
| Map | Leaflet or MapLibre |
|
|
||||||
| Deployment | Docker container, Nginx Proxy Manager, behind Authelia |
|
|
||||||
|
|
||||||
## Database Schema
|
**Stack:** Node + chokidar (file watcher) or a scheduled scan job
|
||||||
|
|
||||||
```sql
|
**On new file detected:**
|
||||||
photos (
|
- **Extract EXIF** (date taken, GPS, camera model, orientation) — exifr or exiftool wrapper
|
||||||
id, filepath, thumbnail_path, date_taken, date_imported,
|
- **Generate posters** (e.g., sharp) at multiple sizes — grid poster + hero/preview size
|
||||||
camera_model, gps_lat, gps_lon, phash, width, height, filesize
|
- **Compute a perceptual hash** (pHash via sharp + blockhash-core) for smart duplicate detection
|
||||||
)
|
- **Insert a row** into SQLite: path, EXIF fields, poster path, hash, import timestamp
|
||||||
albums (id, name, created_at)
|
|
||||||
photo_albums (photo_id, album_id)
|
|
||||||
tags (id, name)
|
|
||||||
photo_tags (photo_id, tag_id)
|
|
||||||
```
|
|
||||||
|
|
||||||
## API Endpoints (planned)
|
Runs as a background process inside the same container, or a separate lightweight worker container if you want to isolate it from the API.
|
||||||
|
|
||||||
```
|
---
|
||||||
GET /photos paginated timeline (filter by date, tag, album)
|
|
||||||
GET /photos/:id full detail + EXIF
|
|
||||||
GET /photos/duplicates grouped by pHash similarity
|
|
||||||
GET /photos/map GPS-tagged photos for map view
|
|
||||||
POST /albums create album
|
|
||||||
POST /photos/:id/tags tag a photo
|
|
||||||
```
|
|
||||||
|
|
||||||
## Deployment Notes
|
### Database (SQLite)
|
||||||
|
|
||||||
- Mount photo folder(s) **read-only** (worker doesn't need write access to originals)
|
**Core tables to start with:**
|
||||||
- Writable volume for thumbnails + DB
|
|
||||||
- Route: `photos.necloud.us` via Nginx Proxy Manager
|
| Table | Purpose |
|
||||||
- Same homelab pattern as other services
|
|-------|---------|
|
||||||
|
| `photos` | `id, filepath, poster_path, date_taken, date_imported, camera_model, gps_lat, gps_lon, phash, width, height, filesize` |
|
||||||
|
| `collections` | `id, name, created_at, description, poster_path` |
|
||||||
|
| `photo_collections` (join) | `photo_id, collection_id` |
|
||||||
|
| `tags` | `id, name` |
|
||||||
|
| `photo_tags` (join) | `photo_id, tag_id` |
|
||||||
|
|
||||||
|
Matches the SQLite pattern you're already using in the study app — no new infra to stand up.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### API Server (Express)
|
||||||
|
|
||||||
|
- **GET /photos** — paginated library query (filter by date range, tag, collection)
|
||||||
|
- **GET /photos/:id** — full detail + EXIF metadata
|
||||||
|
- **GET /photos/similar** — grouped by similar pHash (within a distance threshold)
|
||||||
|
- **POST /collections, POST /photos/:id/tags** — manual organization endpoints
|
||||||
|
- **GET /photos/map** — GPS-tagged photos for map view
|
||||||
|
- **Poster serving** — directly via endpoint or Nginx for static file efficiency
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Frontend (React/Vite)
|
||||||
|
|
||||||
|
- **Browse view** — infinite scroll, grouped by day/month, lazy-loaded posters (like Plex's continuous feed)
|
||||||
|
- **Collection/tag views** — same grid, filtered (like Plex collections for photos)
|
||||||
|
- **Detail view** — full image, EXIF panel, tag editor, related photos
|
||||||
|
- **Duplicate review** — side-by-side similar photos with keep/flag workflow (like Plex's duplicate management)
|
||||||
|
- **Map view** — Leaflet/MapLibre pins from GPS EXIF, click to open photo
|
||||||
|
- **Discovery/search** — date range, camera, tag filters, full-text search
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Smart Duplicate Detection (High Value — Do This Early)
|
||||||
|
|
||||||
|
At import time, compute pHash for every photo. On a scheduled job (or on-demand from a "Find Similar" button), compare hashes pairwise within a similarity threshold (Hamming distance) to group near-duplicates — burst shots, slightly different crops, etc.
|
||||||
|
|
||||||
|
Surface groups in a review UI rather than auto-deleting — you make the call, app just surfaces the candidates. This is much cheaper computationally than face recognition and gives you the real "cleanup" value people want from a photo manager.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Face Grouping (Optional, Scope Carefully)
|
||||||
|
|
||||||
|
If you want this eventually:
|
||||||
|
|
||||||
|
**Options:** face-api.js (runs in Node, ONNX-based, no Python needed) or a Python microservice using face_recognition/dlib if you're OK adding a Python component.
|
||||||
|
|
||||||
|
Store face embeddings + bounding boxes in a `faces` table, cluster embeddings to group "this looks like the same person" without needing named identities upfront — you label clusters after the fact.
|
||||||
|
|
||||||
|
**This is the single biggest scope jump** — worth treating as a v2/v3 feature, not part of the initial build.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Deployment Fit
|
||||||
|
|
||||||
|
**New container** (`photo-library`), same pattern as your other homelab services:
|
||||||
|
- Mounts your existing photo backup folder(s) read-only (ingestion engine doesn't need write access to originals) plus a writable volume for posters/DB
|
||||||
|
- Nginx Proxy Manager route (photos.necloud.us), behind Authelia like your other private services
|
||||||
|
- No external dependencies — everything runs locally against files already on disk
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Suggested Build Order
|
||||||
|
|
||||||
|
1. **Ingestion engine + SQLite schema** — watch folder, extract EXIF, generate posters, populate DB (get this solid before touching UI)
|
||||||
|
2. **Basic API + browse UI** — paginated grid, chronological browsing — this alone is already usable
|
||||||
|
3. **Detail view + manual tagging/collections**
|
||||||
|
4. **Smart duplicate detection + review UI** — high value, moderate effort
|
||||||
|
5. **Search + filters**
|
||||||
|
6. **Map view** (if your library has meaningful GPS data)
|
||||||
|
7. **Face grouping** (only if you still want it after living with V1–V6 for a while)
|
||||||
Reference in new issue
Block a user