Add K-LOVE support and Podman-friendly container deployment

This commit is contained in:
Nate Emmert committed 2026-03-13 09:31:16 -04:00
1 parent c0ebe118cb
commit 18f777e3e7
19 files changed
+2556 -678

No files matched your search

+11
View File
@@ -0,0 +1,11 @@
.git
.gitignore
.venv
__pycache__/
*.pyc
*.pyo
*.pyd
*.log
*.db
nohup.out
icon.png
+5
View File
@@ -11,6 +11,7 @@ __pycache__/
# Logs # Logs
playlist_log.txt playlist_log.txt
app_log.txt
page.html page.html
# Selenium/ChromeDriver # Selenium/ChromeDriver
@@ -28,3 +29,7 @@ Thumbs.db
# Environment variables (if you add a .env file later) # Environment variables (if you add a .env file later)
.env .env
# Debug / local runtime outputs
debug_scrapes/
test_live_klove.py
+59
View File
@@ -0,0 +1,59 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1
WORKDIR /app
RUN mkdir -p /data
# System deps for headless Chromium + Selenium
RUN apt-get update && apt-get install -y --no-install-recommends \
chromium \
chromium-driver \
ca-certificates \
fonts-liberation \
libasound2 \
libatk-bridge2.0-0 \
libatk1.0-0 \
libc6 \
libcairo2 \
libcups2 \
libdbus-1-3 \
libdrm2 \
libexpat1 \
libfontconfig1 \
libgbm1 \
libglib2.0-0 \
libgtk-3-0 \
libnspr4 \
libnss3 \
libpango-1.0-0 \
libx11-6 \
libx11-xcb1 \
libxcb1 \
libxcomposite1 \
libxdamage1 \
libxext6 \
libxfixes3 \
libxrandr2 \
libxrender1 \
libxss1 \
libxtst6 \
xdg-utils \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
# Override if your distro path differs
ENV CHROME_BINARY=/usr/bin/chromium \
CHROMEDRIVER_PATH=/usr/bin/chromedriver \
JOURNEYFM_DATA_DIR=/data \
JOURNEYFM_CONTAINER=1
VOLUME ["/data"]
CMD ["python", "run_container.py"]
+70 -5
View File
@@ -7,6 +7,7 @@ Desktop app + script that pulls recently played songs from Journey FM / Spirit F
- Scrapes recently played songs from: - Scrapes recently played songs from:
- `https://www.myjourneyfm.com/recently-played/` - `https://www.myjourneyfm.com/recently-played/`
- `https://spiritfm.com/ajax/now_playing_history.txt` - `https://spiritfm.com/ajax/now_playing_history.txt`
- `https://www.klove.com/music/songs`
- Matches tracks in your Plex Music library with title/artist normalization. - Matches tracks in your Plex Music library with title/artist normalization.
- Creates or updates your configured Plex playlist. - Creates or updates your configured Plex playlist.
- Avoids adding duplicates already in the target playlist. - Avoids adding duplicates already in the target playlist.
@@ -93,17 +94,19 @@ python journey_fm_app.py
Use the GUI setup wizard (first launch) or edit `config.json` directly. Use the GUI setup wizard (first launch) or edit `config.json` directly.
The Plex token is now stored in your OS credential store via `keyring` when available.
On Windows this means the app no longer needs to persist `PLEX_TOKEN` in `config.json`.
Example `config.json`: Example `config.json`:
```json ```json
{ {
"PLEX_TOKEN": "your-plex-token",
"SERVER_IP": "192.168.1.100", "SERVER_IP": "192.168.1.100",
"PLAYLIST_NAME": "Journey FM Recently Played", "PLAYLIST_NAME": "Journey FM Recently Played",
"AUTO_UPDATE": true, "AUTO_UPDATE": true,
"UPDATE_INTERVAL": 15, "UPDATE_INTERVAL": 15,
"UPDATE_UNIT": "Minutes", "UPDATE_UNIT": "Minutes",
"SELECTED_STATIONS": "journey_fm,spirit_fm" "SELECTED_STATIONS": "journey_fm,spirit_fm,klove"
} }
``` ```
@@ -112,6 +115,10 @@ You can also provide runtime env vars:
- `PLEX_TOKEN` - `PLEX_TOKEN`
- `SERVER_IP` - `SERVER_IP`
- `PLAYLIST_NAME` - `PLAYLIST_NAME`
- `AUTO_UPDATE`
- `UPDATE_INTERVAL`
- `UPDATE_UNIT`
- `SELECTED_STATIONS`
Optional browser override: Optional browser override:
@@ -127,20 +134,75 @@ Run one update cycle from CLI:
python main.py python main.py
``` ```
## Container Deployment
Container mode is intended for scheduled/headless playlist updates. The container now runs as a long-lived loop and sleeps between syncs instead of relying on restart churn.
1. Create a `.env` file in the project root:
```env
PLEX_TOKEN=your-plex-token
SERVER_IP=192.168.1.100
PLAYLIST_NAME=Journey FM Recently Played
SELECTED_STATIONS=journey_fm,spirit_fm,klove
UPDATE_INTERVAL=15
UPDATE_UNIT=Minutes
CONTAINER_RUN_MODE=loop
```
2. Build and run with Docker Compose:
```bash
docker-compose up -d --build
```
Podman equivalent:
```bash
podman compose up -d --build
```
3. Trigger a one-off run and review logs:
```bash
docker-compose run --rm -e CONTAINER_RUN_MODE=once journey-fm-playlist
docker-compose logs -f journey-fm-playlist
```
Podman equivalent:
```bash
podman compose run --rm -e CONTAINER_RUN_MODE=once journey-fm-playlist
podman compose logs -f journey-fm-playlist
```
Notes:
- The container uses Chromium + chromedriver in headless mode.
- Runtime state is stored in the named volume mounted at `/data`, so `config.json`, `playlist_history.db`, `playlist_log.txt`, `amazon_buy_list.txt`, and `debug_scrapes/` persist without bind-mounting the repo.
- GUI (`journey_fm_app.py`) is still intended to run natively on your desktop.
- In container mode, environment variables are preferred over local config or keyring storage.
- The Compose file is Podman-friendly because it uses a named volume instead of a source bind mount.
## GUI Notes ## GUI Notes
- Minimize-to-tray on close when tray is available. - Minimize-to-tray on close when tray is available.
- Tray menu supports Show / Update Now / Quit. - Tray menu supports Show / Update Now / Quit.
- Auto-update runs on configurable interval. - Auto-update runs on configurable interval.
- Works on X11 and Wayland Linux sessions. - Works on X11 and Wayland Linux sessions.
- Updated visual style includes a dashboard hero section, richer status chips, and a refreshed control layout.
- Setup now includes Plex connection testing, keyring-backed token storage, playlist browsing with item counts, music-only filtering, and typed creation of new playlist names.
## Output Files ## Output Files
- `config.json` - runtime settings shared by GUI and CLI - `config.json` - non-secret runtime settings shared by GUI and CLI
- `playlist_history.db` - update history and statistics data - `playlist_history.db` - update history and statistics data
- `playlist_log.txt` - GUI update logs - `playlist_log.txt` - GUI update logs
- `amazon_buy_list.txt` - missing songs with Amazon links - `amazon_buy_list.txt` - missing songs with Amazon links
- `playlist_export.csv` - exported playlist from GUI action - `playlist_export.csv` - exported playlist from GUI action
- `debug_scrapes/` - captured raw station responses for scraper debugging
In container mode these files live under `/data` inside the container volume.
## Troubleshooting ## Troubleshooting
@@ -153,15 +215,18 @@ python main.py
## Core Files ## Core Files
- `journey_fm_app.py` - desktop GUI and tray app - `journey_fm_app.py` - desktop GUI and tray app
- `main.py` - scraping + Plex update pipeline - `main.py` - CLI entrypoint for a single update run
- `run_container.py` - loop/once runner used by container deployments
- `journeyfm/` - shared service modules (config, scraper, Plex, history, orchestration)
- `requirements.txt` - Python dependencies - `requirements.txt` - Python dependencies
- `install.sh` - Linux setup helper - `install.sh` - Linux setup helper
## Pre-Release Checklist ## Pre-Release Checklist
- Verify `config.json` has valid `PLEX_TOKEN`, `SERVER_IP`, and `PLAYLIST_NAME`. - Verify `config.json` has valid `SERVER_IP`, `PLAYLIST_NAME`, and station/schedule settings.
- Run one CLI update: `python main.py` and confirm songs/history are written. - Run one CLI update: `python main.py` and confirm songs/history are written.
- Run GUI update: `python journey_fm_app.py` and verify status/log output updates. - Run GUI update: `python journey_fm_app.py` and verify status/log output updates.
- Confirm tray behavior (minimize to tray, Show/Update/Quit actions). - Confirm tray behavior (minimize to tray, Show/Update/Quit actions).
- On Windows/Linux, confirm Chrome/Chromium detection or set `CHROME_BINARY`. - On Windows/Linux, confirm Chrome/Chromium detection or set `CHROME_BINARY`.
- Export playlist once and verify `playlist_export.csv` is created. - Export playlist once and verify `playlist_export.csv` is created.
- For container mode, run `docker-compose run --rm -e CONTAINER_RUN_MODE=once journey-fm-playlist` and verify logs/history output.
+24
View File
@@ -0,0 +1,24 @@
services:
journey-fm-playlist:
build:
context: .
dockerfile: Dockerfile
container_name: journey-fm-playlist
restart: unless-stopped
environment:
PLEX_TOKEN: ${PLEX_TOKEN}
SERVER_IP: ${SERVER_IP}
PLAYLIST_NAME: ${PLAYLIST_NAME:-Journey FM Recently Played}
SELECTED_STATIONS: ${SELECTED_STATIONS:-journey_fm,spirit_fm}
AUTO_UPDATE: ${AUTO_UPDATE:-true}
UPDATE_INTERVAL: ${UPDATE_INTERVAL:-15}
UPDATE_UNIT: ${UPDATE_UNIT:-Minutes}
CONTAINER_RUN_MODE: ${CONTAINER_RUN_MODE:-loop}
CHROME_BINARY: /usr/bin/chromium
CHROMEDRIVER_PATH: /usr/bin/chromedriver
JOURNEYFM_DATA_DIR: /data
volumes:
- journey-fm-data:/data
volumes:
journey-fm-data:
+1099 -148
View File
File diff suppressed because it is too large. Load diff
+1
View File
@@ -0,0 +1 @@
"""Shared services for Journey FM Playlist Creator."""
+170
View File
@@ -0,0 +1,170 @@
import json
import os
from pathlib import Path
from journeyfm.paths import data_path
try:
import keyring
except Exception:
keyring = None
CONFIG_PATH = data_path("config.json")
KEYRING_SERVICE = "JourneyFMPlaylistCreator"
DEFAULT_CONFIG = {
"PLEX_TOKEN": "",
"SERVER_IP": "",
"PLAYLIST_NAME": "Journey FM Recently Played",
"AUTO_UPDATE": False,
"UPDATE_INTERVAL": 15,
"UPDATE_UNIT": "Minutes",
"SELECTED_STATIONS": ["journey_fm", "spirit_fm"],
}
NON_SECRET_KEYS = {
"SERVER_IP",
"PLAYLIST_NAME",
"AUTO_UPDATE",
"UPDATE_INTERVAL",
"UPDATE_UNIT",
"SELECTED_STATIONS",
}
def is_containerized():
return os.path.exists("/.dockerenv") or os.getenv("JOURNEYFM_CONTAINER") == "1"
def _normalize_selected_stations(value):
if isinstance(value, str):
return [item.strip() for item in value.split(",") if item.strip()]
if isinstance(value, list):
return [str(item).strip() for item in value if str(item).strip()]
return list(DEFAULT_CONFIG["SELECTED_STATIONS"])
def _read_config_file(config_path=CONFIG_PATH):
if not Path(config_path).exists():
return {}
try:
with open(config_path, "r", encoding="utf-8") as file_handle:
return json.load(file_handle)
except Exception:
return {}
def _write_config_file(config, config_path=CONFIG_PATH):
with open(config_path, "w", encoding="utf-8") as file_handle:
json.dump(config, file_handle, indent=2)
def get_secret(key):
env_value = os.getenv(key, "").strip()
if env_value:
return env_value
if not is_containerized() and keyring is not None:
try:
stored = keyring.get_password(KEYRING_SERVICE, key)
if stored:
return stored
except Exception:
pass
file_config = _read_config_file()
return str(file_config.get(key, "")).strip()
def set_secret(key, value):
if not value:
delete_secret(key)
return
if not is_containerized() and keyring is not None:
try:
keyring.set_password(KEYRING_SERVICE, key, value)
return
except Exception:
pass
file_config = _read_config_file()
file_config[key] = value
_write_config_file(file_config)
def delete_secret(key):
if not is_containerized() and keyring is not None:
try:
keyring.delete_password(KEYRING_SERVICE, key)
except Exception:
pass
file_config = _read_config_file()
if key in file_config:
del file_config[key]
_write_config_file(file_config)
def migrate_legacy_secrets(config_path=CONFIG_PATH):
file_config = _read_config_file(config_path)
token = str(file_config.get("PLEX_TOKEN", "")).strip()
if token and not is_containerized() and keyring is not None:
try:
keyring.set_password(KEYRING_SERVICE, "PLEX_TOKEN", token)
del file_config["PLEX_TOKEN"]
_write_config_file(file_config, config_path)
except Exception:
pass
def load_runtime_config(config_path=CONFIG_PATH):
migrate_legacy_secrets(config_path)
config = dict(DEFAULT_CONFIG)
file_config = _read_config_file(config_path)
config.update({key: value for key, value in file_config.items() if key in NON_SECRET_KEYS})
config["PLEX_TOKEN"] = get_secret("PLEX_TOKEN")
env_server = os.getenv("SERVER_IP", "").strip()
env_playlist = os.getenv("PLAYLIST_NAME", "").strip()
env_stations = os.getenv("SELECTED_STATIONS", "").strip()
env_auto_update = os.getenv("AUTO_UPDATE", "").strip()
env_interval = os.getenv("UPDATE_INTERVAL", "").strip()
env_unit = os.getenv("UPDATE_UNIT", "").strip()
if env_server:
config["SERVER_IP"] = env_server
if env_playlist:
config["PLAYLIST_NAME"] = env_playlist
if env_stations:
config["SELECTED_STATIONS"] = _normalize_selected_stations(env_stations)
else:
config["SELECTED_STATIONS"] = _normalize_selected_stations(config.get("SELECTED_STATIONS"))
if env_auto_update:
config["AUTO_UPDATE"] = env_auto_update.lower() in {"1", "true", "yes", "on"}
if env_interval:
try:
config["UPDATE_INTERVAL"] = int(env_interval)
except ValueError:
pass
if env_unit:
config["UPDATE_UNIT"] = env_unit
return config
def save_runtime_config(config, config_path=CONFIG_PATH):
config = dict(config)
token = str(config.pop("PLEX_TOKEN", "")).strip()
if token:
set_secret("PLEX_TOKEN", token)
persisted = {}
for key in NON_SECRET_KEYS:
if key not in config:
continue
value = config[key]
if isinstance(value, list):
persisted[key] = ",".join(value)
else:
persisted[key] = value
_write_config_file(persisted, config_path)
def get_display_config(config_path=CONFIG_PATH):
config = load_runtime_config(config_path)
config["PLEX_TOKEN"] = get_secret("PLEX_TOKEN")
return config
+73
View File
@@ -0,0 +1,73 @@
import json
import sqlite3
from datetime import datetime
from journeyfm.paths import data_path
HISTORY_COLUMNS = {
"status": "TEXT DEFAULT 'success'",
"scraped_count": "INTEGER DEFAULT 0",
"matched_count": "INTEGER DEFAULT 0",
"duplicate_count": "INTEGER DEFAULT 0",
"skipped_count": "INTEGER DEFAULT 0",
"station_breakdown": "TEXT DEFAULT '[]'",
"skipped_songs": "TEXT DEFAULT '[]'",
"error_message": "TEXT DEFAULT ''",
}
def init_history_db(db_path=None):
db_path = db_path or data_path("playlist_history.db")
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
cursor.execute(
"""
CREATE TABLE IF NOT EXISTS history (
id INTEGER PRIMARY KEY,
date TEXT,
added_count INTEGER,
added_songs TEXT,
missing_count INTEGER,
missing_songs TEXT
)
"""
)
cursor.execute("PRAGMA table_info(history)")
existing_columns = {row[1] for row in cursor.fetchall()}
for column_name, definition in HISTORY_COLUMNS.items():
if column_name not in existing_columns:
cursor.execute(f"ALTER TABLE history ADD COLUMN {column_name} {definition}")
conn.commit()
conn.close()
def save_history_entry(result, db_path=None):
db_path = db_path or data_path("playlist_history.db")
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
cursor.execute(
"""
INSERT INTO history (
date, added_count, added_songs, missing_count, missing_songs,
status, scraped_count, matched_count, duplicate_count, skipped_count,
station_breakdown, skipped_songs, error_message
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
datetime.now().isoformat(),
result.get("added_count", 0),
json.dumps(result.get("added_songs", [])),
result.get("missing_count", 0),
json.dumps(result.get("missing_songs", [])),
result.get("status", "success"),
result.get("scraped_count", 0),
result.get("matched_count", 0),
result.get("duplicate_count", 0),
result.get("skipped_count", 0),
json.dumps(result.get("station_breakdown", [])),
json.dumps(result.get("skipped_songs", [])),
result.get("error_message", ""),
),
)
conn.commit()
conn.close()
+16
View File
@@ -0,0 +1,16 @@
import os
from pathlib import Path
def get_data_dir() -> Path:
data_dir = os.getenv("JOURNEYFM_DATA_DIR", "").strip()
if data_dir:
path = Path(data_dir)
else:
path = Path(".")
path.mkdir(parents=True, exist_ok=True)
return path
def data_path(*parts: str) -> Path:
return get_data_dir().joinpath(*parts)
+273
View File
@@ -0,0 +1,273 @@
import urllib.parse
from plexapi.myplex import MyPlexAccount
from plexapi.server import PlexServer
class PlexConnectionError(RuntimeError):
pass
def normalize_server_target(server_ip):
target = (server_ip or "").strip()
if not target:
return ""
if "://" not in target:
target = f"http://{target}"
parsed = urllib.parse.urlparse(target)
return (parsed.hostname or "").strip().lower()
def resolve_plex_server_url(token, server_ip):
target_host = normalize_server_target(server_ip)
if not token:
raise PlexConnectionError("Missing Plex token")
if not target_host:
raise PlexConnectionError("Missing Plex server address")
try:
account = MyPlexAccount(token=token)
except Exception as exc:
raise PlexConnectionError(f"Failed to authenticate with Plex: {exc}") from exc
for resource in account.resources():
for connection in resource.connections:
connection_host = normalize_server_target(connection.uri)
address = (connection.address or "").strip().lower()
if connection_host == target_host or target_host == address or target_host in address:
return connection.uri
raise PlexConnectionError(
f"Server not found at {server_ip}. Verify the configured address and that Plex is reachable."
)
def connect_to_plex_server(token, server_ip):
server_url = resolve_plex_server_url(token, server_ip)
try:
return PlexServer(server_url, token)
except Exception as exc:
raise PlexConnectionError(f"Failed to connect to Plex server at {server_url}: {exc}") from exc
def playlist_item_count(playlist):
leaf_count = getattr(playlist, "leafCount", None)
if isinstance(leaf_count, int):
return leaf_count
try:
return len(playlist.items())
except Exception:
return 0
def fetch_playlists(token, server_ip, music_only=False):
plex = connect_to_plex_server(token, server_ip)
return fetch_playlists_for_plex(plex, music_only)
def fetch_playlists_for_plex(plex, music_only=False):
playlists = []
for playlist in plex.playlists():
playlist_type = getattr(playlist, "playlistType", "unknown")
if music_only and playlist_type != "audio":
continue
playlists.append(
{
"title": playlist.title,
"playlist_type": playlist_type,
"item_count": playlist_item_count(playlist),
}
)
playlists.sort(key=lambda item: (item["title"].lower(), item["playlist_type"]))
return playlists
def validate_playlist_target(token, server_ip, playlist_name, music_only=False):
plex = connect_to_plex_server(token, server_ip)
playlist_name = (playlist_name or "").strip()
if not playlist_name:
raise PlexConnectionError("Playlist name is required")
playlists = fetch_playlists_for_plex(plex, music_only=False)
for playlist in playlists:
if playlist["title"].strip().lower() == playlist_name.lower():
if music_only and playlist["playlist_type"] != "audio":
raise PlexConnectionError("Selected playlist is not a music playlist")
return {
"exists": True,
"playlist": playlist,
"message": f"Ready to write to existing playlist '{playlist_name}'",
}
try:
plex.library.section("Music")
except Exception as exc:
raise PlexConnectionError(f"Music library is not accessible on this server: {exc}") from exc
return {
"exists": False,
"playlist": {
"title": playlist_name,
"playlist_type": "audio",
"item_count": 0,
},
"message": f"Playlist '{playlist_name}' will be created on first sync",
}
def _normalize_artist(name):
"""Return a simplified artist token for fuzzy comparison."""
import re
name = name.lower()
# Drop featured artists — everything after feat/ft/with/w/
name = re.sub(r"\s+(feat\.?|ft\.?|featuring|w/|with)\s+.*", "", name)
# Drop parenthetical / bracketed suffixes
name = re.sub(r"[\(\[].*?[\)\]]", "", name)
# Expand & / + to "and"
name = re.sub(r"\s*[&+]\s*", " and ", name)
# Collapse articles at start: "the foo" → "foo"
name = re.sub(r"^(the|a|an)\s+", "", name)
# Remove common punctuation that Plex sometimes strips
name = re.sub(r"['\.\-,]", "", name)
# Collapse whitespace
name = re.sub(r"\s+", " ", name).strip()
return name
def _artists_match(search_artist, track_artist_raw):
"""
Return True if the station artist string plausibly refers to the same
artist stored in Plex, using several normalisation / containment passes.
"""
sa = _normalize_artist(search_artist)
ta = _normalize_artist(track_artist_raw)
if not sa or not ta:
return False
# Exact normalised match
if sa == ta:
return True
# One fully contains the other (e.g. "Brandon Lake" in "Brandon Lake & Maverick")
if sa in ta or ta in sa:
return True
# Token-set overlap: every token in the shorter name appears in the longer
sa_tokens = set(sa.split())
ta_tokens = set(ta.split())
if sa_tokens and ta_tokens:
shorter = sa_tokens if len(sa_tokens) <= len(ta_tokens) else ta_tokens
longer = ta_tokens if len(sa_tokens) <= len(ta_tokens) else sa_tokens
# All tokens of the shorter name found in the longer (handles "for king country" ↔ "for king and country")
if shorter <= longer:
return True
# At least >60 % of tokens match (handles minor word-count diffs)
overlap = len(shorter & longer)
if overlap / len(shorter) >= 0.6:
return True
return False
def create_or_update_playlist(plex, songs, playlist_name, dry_run=False):
import re
music_library = plex.library.section("Music")
tracks = []
seen_keys = set()
missing = []
skipped = []
added_songs = []
duplicate_songs = []
for song in songs:
clean_title = re.sub(r"\([^)]*\)", "", song["title"]).strip()
clean_title = re.sub(r"\s+", " ", clean_title)
clean_artist = re.sub(r"\([^)]*\)", "", song["artist"]).strip()
clean_artist = re.sub(r"\s+", " ", clean_artist)
if not clean_title or len(clean_title) < 3 or clean_title.lower() in {"by", "recently played", "now playing:", "search"}:
skipped.append({"title": song["title"], "artist": song["artist"], "reason": "invalid-title"})
continue
if re.match(r"^[!?\s]+$", clean_title):
skipped.append({"title": song["title"], "artist": song["artist"], "reason": "punctuation-only-title"})
continue
search_title = re.sub(r"[!?]", "", clean_title).strip()
search_artist = re.sub(r"[!?]", "", clean_artist).strip()
# Expand & / + in title for Plex search (artist expansion handled by _artists_match)
search_title = re.sub(r"\s*[&+]\s*", " and ", search_title, flags=re.IGNORECASE)
results = music_library.searchTracks(title=search_title)
if not results:
# Secondary search: try the raw title in case Plex stores it differently
results = music_library.searchTracks(title=clean_title)
if not results:
missing.append({"title": song["title"], "artist": song["artist"], "reason": "not-found"})
continue
matched = False
for track in results:
try:
track_artist_raw = track.artist().title
except Exception:
continue
if _artists_match(search_artist, track_artist_raw):
if track.ratingKey not in seen_keys:
tracks.append((track, song))
seen_keys.add(track.ratingKey)
matched = True
break
if not matched:
missing.append({"title": song["title"], "artist": song["artist"], "reason": "artist-mismatch"})
matched_count = len(tracks)
added_count = 0
if tracks:
try:
try:
existing = plex.playlist(playlist_name)
existing_keys = {item.ratingKey for item in existing.items()}
new_tracks = []
for track, song in tracks:
if track.ratingKey in existing_keys:
duplicate_songs.append({"title": track.title, "artist": track.artist().title, "reason": "already-in-playlist"})
continue
new_tracks.append((track, song))
if new_tracks:
if not dry_run:
existing.addItems([item[0] for item in new_tracks])
for track, song in new_tracks:
track.addLabel("Journey FM")
track.rate(5)
added_songs.append(f"{track.title} by {track.artist().title} ({song['source']})")
else:
for track, song in new_tracks:
added_songs.append(f"{track.title} by {track.artist().title} ({song['source']})")
added_count = len(new_tracks)
except Exception:
if not dry_run:
plex.createPlaylist(playlist_name, [item[0] for item in tracks])
for track, song in tracks:
track.addLabel("Journey FM")
track.rate(5)
added_songs.append(f"{track.title} by {track.artist().title} ({song['source']})")
else:
for track, song in tracks:
added_songs.append(f"{track.title} by {track.artist().title} ({song['source']})")
added_count = len(tracks)
except Exception as exc:
raise PlexConnectionError(f"Failed updating playlist '{playlist_name}': {exc}") from exc
return {
"matched_count": matched_count,
"added_count": added_count,
"added_songs": added_songs,
"missing_songs": missing,
"duplicate_count": len(duplicate_songs),
"duplicate_songs": duplicate_songs,
"skipped_songs": skipped,
}
+452
View File
@@ -0,0 +1,452 @@
import logging
import os
import re
import shutil
import time
from datetime import datetime
from pathlib import Path
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from webdriver_manager.chrome import ChromeDriverManager
import requests
from journeyfm.paths import data_path
logger = logging.getLogger(__name__)
DEBUG_SCRAPE_DIR = data_path("debug_scrapes")
def detect_chrome_binary():
env_binary = os.getenv("CHROME_BINARY", "").strip()
if env_binary and os.path.exists(env_binary):
return env_binary
candidates = []
if os.name == "nt":
local_app_data = os.getenv("LOCALAPPDATA", "")
program_files = os.getenv("PROGRAMFILES", "")
program_files_x86 = os.getenv("PROGRAMFILES(X86)", "")
candidates.extend([
os.path.join(local_app_data, "Google", "Chrome", "Application", "chrome.exe"),
os.path.join(program_files, "Google", "Chrome", "Application", "chrome.exe"),
os.path.join(program_files_x86, "Google", "Chrome", "Application", "chrome.exe"),
os.path.join(program_files, "Chromium", "Application", "chrome.exe"),
os.path.join(program_files_x86, "Chromium", "Application", "chrome.exe"),
])
else:
candidates.extend([
"/usr/bin/google-chrome",
"/usr/bin/google-chrome-stable",
"/usr/bin/chromium",
"/usr/bin/chromium-browser",
"/opt/google/chrome/chrome",
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"/Applications/Chromium.app/Contents/MacOS/Chromium",
])
for binary in candidates:
if binary and os.path.exists(binary):
return binary
for name in ["google-chrome", "google-chrome-stable", "chromium", "chromium-browser", "chrome"]:
found = shutil.which(name)
if found:
return found
return None
def detect_chromedriver_path():
env_driver = os.getenv("CHROMEDRIVER_PATH", "").strip()
if env_driver and os.path.exists(env_driver):
return env_driver
for candidate in ["/usr/bin/chromedriver", "/usr/local/bin/chromedriver"]:
if os.path.exists(candidate):
return candidate
return shutil.which("chromedriver")
def _write_debug_payload(station_key, payload, extension):
DEBUG_SCRAPE_DIR.mkdir(exist_ok=True)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
file_path = DEBUG_SCRAPE_DIR / f"{station_key}_{timestamp}.{extension}"
with open(file_path, "w", encoding="utf-8") as file_handle:
file_handle.write(payload)
latest_path = DEBUG_SCRAPE_DIR / f"{station_key}_latest.{extension}"
with open(latest_path, "w", encoding="utf-8") as file_handle:
file_handle.write(payload)
return str(file_path)
class JourneyFMScraper:
station_key = "journey_fm"
display_name = "Journey FM"
url = "https://www.myjourneyfm.com/recently-played/"
SELECTOR_PATTERNS = [
{
"name": "rp-item",
"container": ("div", "rp-item"),
"title": ("h5", "song-title"),
"artist": ("p", "song-artist"),
},
{
"name": "top-rp-list",
"container": ("div", "song-item"),
"title": ("div", "title-artist"),
"artist": ("span", None),
},
]
@classmethod
def _extract_with_pattern(cls, soup, pattern):
songs = []
container_tag, container_class = pattern["container"]
title_tag, title_class = pattern["title"]
artist_tag, artist_class = pattern["artist"]
for item in soup.find_all(container_tag, class_=container_class):
if title_class:
title_elem = item.find(title_tag, class_=title_class)
else:
title_elem = item.find(title_tag)
if not title_elem:
continue
if pattern["name"] == "top-rp-list":
paragraph = title_elem.find("p")
if not paragraph:
continue
combined = paragraph.get_text(" ", strip=True)
parts = re.split(r"\s+by\s+", combined, maxsplit=1, flags=re.IGNORECASE)
if len(parts) != 2:
continue
title = parts[0].strip()
artist = parts[1].strip()
else:
if artist_class:
artist_elem = item.find(artist_tag, class_=artist_class)
else:
artist_elem = item.find(artist_tag)
if not artist_elem:
continue
title = title_elem.get_text(" ", strip=True).strip()
artist = artist_elem.get_text(" ", strip=True).strip()
if title and artist:
songs.append({"title": title, "artist": artist, "source": cls.display_name})
return songs
@classmethod
def parse_html_with_telemetry(cls, html):
soup = BeautifulSoup(html, "html.parser")
for pattern in cls.SELECTOR_PATTERNS:
songs = cls._extract_with_pattern(soup, pattern)
if songs:
return songs, pattern["name"]
return [], "none"
@staticmethod
def parse_html(html):
songs, _pattern = JourneyFMScraper.parse_html_with_telemetry(html)
return songs
@classmethod
def scrape(cls, driver):
driver.get(cls.url)
WebDriverWait(driver, 15).until(
lambda _driver: (
bool(_driver.find_elements(By.CSS_SELECTOR, "div.rp-item"))
or bool(_driver.find_elements(By.CSS_SELECTOR, "div.song-item"))
)
)
try:
more_button = WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.ID, "moreSongs")))
more_button.click()
time.sleep(2)
except Exception:
pass
html = driver.page_source
raw_path = _write_debug_payload(cls.station_key, html, "html")
songs, parse_pattern = cls.parse_html_with_telemetry(html)
return songs, raw_path, parse_pattern
@classmethod
def scrape_without_driver(cls):
response = requests.get(cls.url, timeout=20)
response.raise_for_status()
html = response.text
raw_path = _write_debug_payload(cls.station_key, html, "html")
songs, parse_pattern = cls.parse_html_with_telemetry(html)
return songs, raw_path, f"{parse_pattern}-requests-fallback"
class SpiritFMScraper:
station_key = "spirit_fm"
display_name = "Spirit FM"
url = "https://spiritfm.com/ajax/now_playing_history.txt"
@staticmethod
def parse_text(text):
songs = []
for line in text.splitlines():
line = line.strip()
if not line:
continue
match = re.match(r"^\w+\s+\d+:\d+[AP]M\s+(.+?)\s+-\s+(.+)$", line)
if not match:
continue
artist = match.group(1).strip().replace("&amp;", "&")
title = match.group(2).strip().replace("&amp;", "&")
if title and artist:
songs.append({"title": title, "artist": artist, "source": SpiritFMScraper.display_name})
return songs
@classmethod
def scrape(cls, driver):
driver.get(cls.url)
time.sleep(2)
text = BeautifulSoup(driver.page_source, "html.parser").get_text()
raw_path = _write_debug_payload(cls.station_key, text, "txt")
return cls.parse_text(text), raw_path, "plain-text-regex"
@classmethod
def scrape_without_driver(cls):
response = requests.get(cls.url, timeout=20)
response.raise_for_status()
text = response.text
raw_path = _write_debug_payload(cls.station_key, text, "txt")
return cls.parse_text(text), raw_path, "plain-text-regex-requests-fallback"
class KLOVEScraper:
station_key = "klove"
display_name = "K-LOVE"
url = "https://www.klove.com/music/songs"
# K-LOVE is a JS-rendered React app; Selenium is the primary path.
# The requests fallback may still return partial SSR content worth parsing.
_LOAD_SELECTOR = "a[href*='/music/artists/']"
@staticmethod
def parse_html(html):
songs, _pattern = KLOVEScraper.parse_html_with_telemetry(html)
return songs
@staticmethod
def parse_html_with_telemetry(html):
soup = BeautifulSoup(html, "html.parser")
songs = []
seen = set()
# Collect every anchor that points at /music/artists/{artist}/{song}
# The page renders each card as two anchors (art + title) — use hrefs
# as a dedup key so we only process each song once.
processed_hrefs = set()
for anchor in soup.find_all("a", href=re.compile(r"/music/artists/[^/]+/[^/]+")):
href = anchor.get("href", "").strip().rstrip("/")
parts = href.strip("/").split("/")
# Expect: music / artists / artist-slug / song-slug
if len(parts) < 4:
continue
title_text = anchor.get_text(" ", strip=True)
if not title_text or len(title_text) < 2:
# Image-only anchor (cover art link) — skip it; the title anchor
# for this same href will appear later in the DOM with real text.
continue
# Now that we have a text-bearing anchor, dedup by href
if href in processed_hrefs:
continue
processed_hrefs.add(href)
# Skip navigation / album-only links (e.g. artist page without song)
song_slug = parts[-1]
artist_slug = parts[-2] if len(parts) >= 4 else ""
# Find "By {artist}" text in nearby DOM — check siblings and parent
artist_text = ""
parent = anchor.parent
if parent:
full_text = parent.get_text(" ", strip=True)
by_match = re.search(r"\bBy\s+(.+)", full_text)
if by_match:
artist_text = by_match.group(1).strip()
if not artist_text and parent and parent.parent:
grandparent_text = parent.parent.get_text(" ", strip=True)
by_match = re.search(r"\bBy\s+(.+?)(?:\bBy\b|$)", grandparent_text)
if by_match:
artist_text = by_match.group(1).strip()
if not artist_text:
# Derive artist from URL slug as last resort
artist_text = " ".join(w.capitalize() for w in artist_slug.split("-"))
# Strip leftover boilerplate that sometimes bleeds in
artist_text = re.sub(r"\s*(feat\.|ft\.|featuring|Play Sample|Image).*$", "",
artist_text, flags=re.IGNORECASE).strip()
if title_text and artist_text:
key = (title_text.lower(), artist_text.lower())
if key not in seen:
seen.add(key)
songs.append({"title": title_text, "artist": artist_text,
"source": KLOVEScraper.display_name})
if songs:
return songs, "klove-anchor-by"
return [], "none"
@classmethod
def scrape(cls, driver):
driver.get(cls.url)
# Wait for song anchor links to appear (JS-rendered)
WebDriverWait(driver, 20).until(
EC.presence_of_element_located((By.CSS_SELECTOR, cls._LOAD_SELECTOR))
)
time.sleep(2) # Allow lazy-load tiles to settle
html = driver.page_source
raw_path = _write_debug_payload(cls.station_key, html, "html")
songs, parse_pattern = cls.parse_html_with_telemetry(html)
return songs, raw_path, parse_pattern
@classmethod
def scrape_without_driver(cls):
headers = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/120.0.0.0 Safari/537.36"
)
}
response = requests.get(cls.url, headers=headers, timeout=20)
response.raise_for_status()
html = response.text
raw_path = _write_debug_payload(cls.station_key, html, "html")
songs, parse_pattern = cls.parse_html_with_telemetry(html)
return songs, raw_path, f"{parse_pattern}-requests-fallback"
SCRAPERS = {
JourneyFMScraper.station_key: JourneyFMScraper,
SpiritFMScraper.station_key: SpiritFMScraper,
KLOVEScraper.station_key: KLOVEScraper,
}
def build_driver():
options = Options()
options.add_argument("--headless=new")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--disable-gpu")
options.add_argument("--window-size=1920,1080")
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option("useAutomationExtension", False)
chrome_binary = detect_chrome_binary()
if chrome_binary:
options.binary_location = chrome_binary
chromedriver_path = detect_chromedriver_path()
if chromedriver_path:
service = Service(chromedriver_path)
else:
service = Service(ChromeDriverManager().install())
return webdriver.Chrome(service=service, options=options)
def scrape_recently_played(selected_stations=None):
selected_stations = selected_stations or ["journey_fm", "spirit_fm"]
driver = None
driver_start_error = ""
all_songs = []
seen = set()
station_results = []
try:
try:
driver = build_driver()
except Exception as exc:
driver = None
driver_start_error = str(exc)
logger.warning("Browser scraper unavailable; using fallback mode: %s", exc)
for station_key in selected_stations:
scraper_cls = SCRAPERS.get(station_key)
if scraper_cls is None:
station_results.append({
"station": station_key,
"display_name": station_key,
"success": False,
"error": "unsupported-station",
"scraped_count": 0,
"raw_path": "",
})
continue
try:
if driver is not None:
try:
songs, raw_path, parse_pattern = scraper_cls.scrape(driver)
except Exception as scrape_error:
logger.warning(
"Selenium scrape failed for %s, retrying fallback mode: %s",
scraper_cls.display_name,
scrape_error,
)
songs, raw_path, parse_pattern = scraper_cls.scrape_without_driver()
else:
songs, raw_path, parse_pattern = scraper_cls.scrape_without_driver()
raw_payload_bytes = 0
if raw_path and os.path.exists(raw_path):
try:
raw_payload_bytes = os.path.getsize(raw_path)
except Exception:
raw_payload_bytes = 0
added_for_station = 0
for song in songs:
dedupe_key = (song["title"].lower(), song["artist"].lower())
if dedupe_key in seen:
continue
seen.add(dedupe_key)
all_songs.append(song)
added_for_station += 1
logger.info(" Found: %s by %s", song["title"], song["artist"])
station_results.append({
"station": station_key,
"display_name": scraper_cls.display_name,
"success": True,
"error": "",
"scraped_count": added_for_station,
"raw_path": raw_path,
"raw_payload_bytes": raw_payload_bytes,
"parse_pattern": parse_pattern,
})
except Exception as exc:
combined_error = str(exc)
if driver is None and driver_start_error:
combined_error = f"{combined_error}; browser init: {driver_start_error}"
station_results.append({
"station": station_key,
"display_name": scraper_cls.display_name,
"success": False,
"error": combined_error,
"scraped_count": 0,
"raw_path": "",
"raw_payload_bytes": 0,
"parse_pattern": "error",
})
logger.error("Error scraping %s: %s", scraper_cls.display_name, exc)
finally:
if driver is not None:
driver.quit()
logger.info("Total unique songs found: %s", len(all_songs))
return {"songs": all_songs, "station_results": station_results}
+138
View File
@@ -0,0 +1,138 @@
import logging
import os
import urllib.parse
from journeyfm.config_store import load_runtime_config
from journeyfm.history_service import init_history_db, save_history_entry
from journeyfm.paths import data_path
from journeyfm.plex_service import PlexConnectionError, connect_to_plex_server, create_or_update_playlist
from journeyfm.scraper_service import scrape_recently_played
logger = logging.getLogger(__name__)
def update_buy_list(missing_songs, buy_list_path=None):
buy_list_path = buy_list_path or data_path("amazon_buy_list.txt")
if not missing_songs:
logger.info("No new songs to add to buy list.")
return []
existing_songs = set()
if os.path.exists(buy_list_path):
try:
with open(buy_list_path, "r", encoding="utf-8") as file_handle:
lines = file_handle.read().splitlines()
for line in lines:
if line.strip() and not line.startswith("http") and " - " in line:
artist, title = line.split(" - ", 1)
existing_songs.add((artist.strip(), title.strip()))
except Exception:
pass
new_missing = []
for song in missing_songs:
key = (song["artist"], song["title"])
if key in existing_songs:
continue
existing_songs.add(key)
new_missing.append(song)
if not new_missing:
logger.info("No new songs to add to buy list.")
return []
file_exists = os.path.exists(buy_list_path)
file_empty = not file_exists or os.path.getsize(buy_list_path) == 0
with open(buy_list_path, "a", encoding="utf-8") as file_handle:
if file_empty:
file_handle.write("Songs not in your library - Amazon search links:\n\n")
for song in new_missing:
query = urllib.parse.quote(f"{song['artist']} {song['title']}")
file_handle.write(f"{song['artist']} - {song['title']}\n")
file_handle.write(f"https://www.amazon.com/s?k={query}&i=digital-music\n\n")
logger.info("Buy list updated.")
return new_missing
def format_result_summary(result):
station_bits = []
for station in result.get("station_breakdown", []):
if station.get("success"):
pattern = station.get("parse_pattern", "unknown")
payload_bytes = int(station.get("raw_payload_bytes", 0) or 0)
payload_kib = payload_bytes / 1024.0
station_bits.append(
f"{station['display_name']}: {station['scraped_count']} [{pattern}, {payload_kib:.1f} KiB]"
)
else:
station_bits.append(f"{station['display_name']}: failed")
station_summary = ", ".join(station_bits) or "No station data"
lines = [
f"Scraped {result.get('scraped_count', 0)} songs ({station_summary})",
f"Matched {result.get('matched_count', 0)} in Plex",
f"Added {result.get('added_count', 0)} to playlist",
f"Skipped {result.get('skipped_count', 0)} invalid entries",
f"Suppressed {result.get('duplicate_count', 0)} duplicates already in playlist",
f"Missing {result.get('missing_count', 0)} from Plex library",
]
if result.get("status") == "error":
lines.append(f"Error: {result.get('error_message', 'Unknown error')}")
return "\n".join(lines)
def run_update_job(config=None, dry_run=False, persist_history=True, write_buy_list=True):
config = config or load_runtime_config()
result = {
"status": "success",
"error_message": "",
"station_breakdown": [],
"scraped_count": 0,
"matched_count": 0,
"added_count": 0,
"added_songs": [],
"missing_count": 0,
"missing_songs": [],
"duplicate_count": 0,
"duplicate_songs": [],
"skipped_count": 0,
"skipped_songs": [],
}
token = config.get("PLEX_TOKEN", "").strip()
server_ip = config.get("SERVER_IP", "").strip()
playlist_name = config.get("PLAYLIST_NAME", "Journey FM Recently Played").strip()
selected_stations = config.get("SELECTED_STATIONS", ["journey_fm", "spirit_fm"])
if not token or not server_ip:
result["status"] = "error"
result["error_message"] = "Missing configuration: set Plex token and server IP before running updates"
return result
if persist_history:
init_history_db()
scrape_result = scrape_recently_played(selected_stations)
songs = scrape_result["songs"]
result["station_breakdown"] = scrape_result["station_results"]
result["scraped_count"] = len(songs)
try:
plex = connect_to_plex_server(token, server_ip)
playlist_result = create_or_update_playlist(plex, songs, playlist_name, dry_run=dry_run)
result.update(playlist_result)
result["missing_count"] = len(result.get("missing_songs", []))
result["skipped_count"] = len(result.get("skipped_songs", []))
if write_buy_list and not dry_run:
update_buy_list(result.get("missing_songs", []))
except PlexConnectionError as exc:
result["status"] = "error"
result["error_message"] = str(exc)
except Exception as exc:
result["status"] = "error"
result["error_message"] = f"Unexpected update failure: {exc}"
if persist_history:
save_history_entry(result)
return result
+23 -506
View File
@@ -1,46 +1,23 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """Journey FM Playlist Creator CLI entrypoint."""
Journey FM Playlist Creator
This script scrapes recently played songs from myjourneyfm.com,
checks if they exist in the local music library,
creates a playlist of matching songs, and imports it into Plex.
"""
import time
import sys
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup
import os
from plexapi.myplex import MyPlexAccount
from mutagen.easyid3 import EasyID3
import re
from datetime import datetime
import json
import subprocess
import urllib.parse
import sqlite3
import logging import logging
import shutil import sys
from journeyfm.config_store import load_runtime_config
from journeyfm.history_service import init_history_db
from journeyfm.paths import data_path
from journeyfm.plex_service import connect_to_plex_server
from journeyfm.update_service import format_result_summary, run_update_job
# Configuration (defaults, can be overridden by env vars or config.json)
PLEX_TOKEN = os.getenv('PLEX_TOKEN', '').strip()
SERVER_IP = os.getenv('SERVER_IP', '').strip()
PLAYLIST_NAME = os.getenv('PLAYLIST_NAME', 'Journey FM Holiday').strip()
def configure_logging(): def configure_logging():
"""Configure logging with safe handlers for pythonw/GUI environments.""" """Configure logging with quiet defaults for CLI and GUI entrypoints."""
log_format = '%(asctime)s - %(levelname)s - %(message)s' log_format = '%(asctime)s - %(levelname)s - %(message)s'
handlers = [] handlers = []
try: try:
handlers.append(logging.FileHandler('playlist_log.txt', encoding='utf-8')) handlers.append(logging.FileHandler(data_path('playlist_log.txt'), encoding='utf-8'))
except Exception: except Exception:
pass pass
@@ -49,494 +26,34 @@ def configure_logging():
handlers.append(logging.StreamHandler(stream)) handlers.append(logging.StreamHandler(stream))
logging.basicConfig(level=logging.INFO, format=log_format, handlers=handlers, force=True) logging.basicConfig(level=logging.INFO, format=log_format, handlers=handlers, force=True)
for noisy_logger in ('webdriver_manager', 'matplotlib', 'urllib3', 'plexapi'): for noisy_logger in ('webdriver_manager', 'WDM', 'matplotlib', 'urllib3', 'plexapi'):
logging.getLogger(noisy_logger).setLevel(logging.WARNING) logging.getLogger(noisy_logger).setLevel(logging.WARNING)
configure_logging() configure_logging()
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def detect_chrome_binary():
"""Detect a Chrome/Chromium executable across common OS locations."""
env_binary = os.getenv('CHROME_BINARY', '').strip()
if env_binary and os.path.exists(env_binary):
return env_binary
candidates = []
if os.name == 'nt':
local_app_data = os.getenv('LOCALAPPDATA', '')
program_files = os.getenv('PROGRAMFILES', '')
program_files_x86 = os.getenv('PROGRAMFILES(X86)', '')
candidates.extend([
os.path.join(local_app_data, 'Google', 'Chrome', 'Application', 'chrome.exe'),
os.path.join(program_files, 'Google', 'Chrome', 'Application', 'chrome.exe'),
os.path.join(program_files_x86, 'Google', 'Chrome', 'Application', 'chrome.exe'),
os.path.join(program_files, 'Chromium', 'Application', 'chrome.exe'),
os.path.join(program_files_x86, 'Chromium', 'Application', 'chrome.exe')
])
elif os.name == 'posix':
candidates.extend([
'/usr/bin/google-chrome',
'/usr/bin/google-chrome-stable',
'/usr/bin/chromium',
'/usr/bin/chromium-browser',
'/opt/google/chrome/chrome',
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
'/Applications/Chromium.app/Contents/MacOS/Chromium'
])
for binary in candidates:
if binary and os.path.exists(binary):
return binary
for name in ['google-chrome', 'google-chrome-stable', 'chromium', 'chromium-browser', 'chrome']:
found = shutil.which(name)
if found:
return found
return None
def scrape_recently_played(selected_stations=None):
"""Scrape recently played songs from Journey FM and Spirit FM"""
if selected_stations is None:
selected_stations = ['journey_fm', 'spirit_fm']
options = Options()
options.add_argument("--headless=new")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--disable-gpu")
options.add_argument("--window-size=1920,1080")
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
chrome_binary = detect_chrome_binary()
if chrome_binary:
options.binary_location = chrome_binary
else:
logger.warning("No Chrome/Chromium binary detected; Selenium will use its default browser resolution")
service = Service(ChromeDriverManager().install())
driver = None
try:
driver = webdriver.Chrome(service=service, options=options)
except Exception as e:
logger.error("Failed to start Chrome: %s", e)
return []
all_songs = []
seen = set()
# Journey FM - uses <strong> for title, normal text for artist
if 'journey_fm' in selected_stations:
try:
logger.info("Scraping Journey FM...")
driver.get('https://www.myjourneyfm.com/recently-played/')
WebDriverWait(driver, 15).until(
EC.presence_of_element_located((By.CSS_SELECTOR, 'div.rp-item'))
)
# Click "View More" button if exists
try:
more_button = WebDriverWait(driver, 5).until(
EC.element_to_be_clickable((By.ID, "moreSongs"))
)
more_button.click()
time.sleep(2)
except Exception:
pass
soup = BeautifulSoup(driver.page_source, 'html.parser')
# Find all song items using the proper class structure
song_items = soup.find_all('div', class_='rp-item')
for item in song_items:
try:
title_elem = item.find('h5', class_='song-title')
artist_elem = item.find('p', class_='song-artist')
if title_elem and artist_elem:
title = title_elem.get_text().strip()
artist = artist_elem.get_text().strip()
if title and artist:
key = (title.lower(), artist.lower())
if key not in seen:
all_songs.append({'title': title, 'artist': artist, 'source': 'Journey FM'})
seen.add(key)
logger.info(" Found: %s by %s", title, artist)
except Exception:
continue
except Exception as e:
logger.error("Error scraping Journey FM: %s", e)
# Spirit FM - loads songs from iframe text file
if 'spirit_fm' in selected_stations:
try:
logger.info("Scraping Spirit FM...")
driver.get('https://spiritfm.com/ajax/now_playing_history.txt')
time.sleep(2)
# Parse the plain text format: "Mon 01:45PM Artist - Title"
soup = BeautifulSoup(driver.page_source, 'html.parser')
text_content = soup.get_text()
# Split into lines and parse each
for line in text_content.split('\n'):
line = line.strip()
if not line:
continue
# Match pattern: Day HH:MM(AM/PM) Artist - Title
match = re.match(r'^\w+\s+\d+:\d+[AP]M\s+(.+?)\s+-\s+(.+)$', line)
if match:
artist = match.group(1).strip()
title = match.group(2).strip()
# Clean up HTML entities
artist = artist.replace('&amp;', '&')
title = title.replace('&amp;', '&')
if title and artist:
key = (title.lower(), artist.lower())
if key not in seen:
all_songs.append({'title': title, 'artist': artist, 'source': 'Spirit FM'})
seen.add(key)
logger.info(" Found: %s by %s", title, artist)
except Exception as e:
logger.error("Error scraping Spirit FM: %s", e)
if driver:
driver.quit()
logger.info("Total unique songs found: %s", len(all_songs))
return all_songs
def normalize_string(s):
"""Normalize string for comparison"""
return re.sub(r'[^\w\s]', '', s).lower().strip()
def find_song_in_library(artist, title, library_path):
"""Check if song exists in local library"""
for root, dirs, files in os.walk(library_path):
for file in files:
if file.endswith(('.mp3', '.flac', '.m4a')): # Add more formats if needed
try:
audio = EasyID3(os.path.join(root, file))
file_artist = normalize_string(audio.get('artist', [''])[0])
file_title = normalize_string(audio.get('title', [''])[0])
if normalize_string(artist) == file_artist and normalize_string(title) == file_title:
return os.path.join(root, file)
except Exception:
continue
return None
def create_playlist_in_plex(plex, songs, playlist_name):
"""Create playlist in Plex and return added count, added songs, and missing songs"""
# Assuming Plex has a music library named 'Music'
music_library = plex.library.section('Music')
tracks = []
seen_keys = set() # Track which ratingKeys we've already added to prevent duplicates within this run
missing = []
added_songs = []
for song in songs:
# Clean title and artist by removing text in parentheses and normalizing spaces
clean_title = re.sub(r'\([^)]*\)', '', song['title']).strip()
clean_title = re.sub(r'\s+', ' ', clean_title) # Replace multiple spaces with single space
clean_artist = re.sub(r'\([^)]*\)', '', song['artist']).strip()
clean_artist = re.sub(r'\s+', ' ', clean_artist) # Replace multiple spaces with single space
# Skip invalid entries
if not clean_title or len(clean_title) < 3 or clean_title.lower() in ['by', 'recently played', 'now playing:', 'search']:
continue
# Skip entries that are ONLY punctuation (but allow titles with punctuation in them)
if re.match(r'^[!?\\\\s]+$', clean_title):
continue
# Remove only ! and ? for searching, but keep apostrophes
search_title = re.sub(r'[!?]', '', clean_title).strip()
search_artist = re.sub(r'[!?]', '', clean_artist).strip()
# Normalize & and + to "and" for both title and artist
search_title = re.sub(r'\s*[&+]\s*', ' and ', search_title, flags=re.IGNORECASE)
search_artist = re.sub(r'\s*[&+]\s*', ' and ', search_artist, flags=re.IGNORECASE)
# Search for tracks by cleaned title
results = music_library.searchTracks(title=search_title)
if results:
# Try to match by artist as well
matched = False
for track in results:
track_artist = track.artist().title.lower()
search_artist_lower = search_artist.lower()
# Clean featured artists from search (e.g., "W/ Emerson Day", "feat. Someone")
search_artist_main = re.sub(r'\s+(w/|feat\.|featuring|ft\.).*', '', search_artist_lower).strip()
# Normalize & and + in track artist as well
track_artist = re.sub(r'\s*[&+]\s*', ' and ', track_artist)
# Check if artist matches (case-insensitive, partial match)
# Match if main artist name is in the track artist or vice versa
if (search_artist_main in track_artist or track_artist in search_artist_main or
search_artist_lower in track_artist or track_artist in search_artist_lower):
# Only add if we haven't already added this track in this run
if track.ratingKey not in seen_keys:
tracks.append((track, song))
seen_keys.add(track.ratingKey)
matched = True
break
# If no artist match, skip this track (don't add wrong songs)
if not matched:
missing.append(song)
added = 0
if tracks:
try:
# Try to find existing playlist
try:
existing = plex.playlist(playlist_name)
# Get existing track rating keys to avoid duplicates
existing_keys = {item.ratingKey for item in existing.items()}
# Filter out tracks already in playlist
new_tracks = [track_song for track_song in tracks if track_song[0].ratingKey not in existing_keys]
if new_tracks:
existing.addItems([ts[0] for ts in new_tracks])
for track, song in new_tracks:
# Auto-tag added songs
track.addLabel("Journey FM")
track.rate(5)
added = len(new_tracks)
added_songs.extend([f"{track.title} by {track.artist().title} ({song['source']})" for track, song in new_tracks])
else:
added = 0
except Exception:
# Playlist doesn't exist, create it
plex.createPlaylist(playlist_name, [ts[0] for ts in tracks])
for track, song in tracks:
# Auto-tag added songs
track.addLabel("Journey FM")
track.rate(5)
added = len(tracks)
added_songs.extend([f"{track.title} by {track.artist().title} ({song['source']})" for track, song in tracks])
except Exception:
added = 0
added_songs = []
else:
logger.info("No matching tracks found in Plex.")
added = 0
added_songs = []
return added, added_songs, missing
def prompt_for_config():
"""Prompt user for configuration variables"""
print("Configuration not found. Please provide the following:")
plex_token = input("Enter your Plex token: ").strip()
server_ip = input("Enter your Plex server IP: ").strip()
playlist_name = input("Enter the playlist name: ").strip()
config = {
'PLEX_TOKEN': plex_token,
'SERVER_IP': server_ip,
'PLAYLIST_NAME': playlist_name
}
with open('config.json', 'w') as f:
json.dump(config, f)
return config
def load_config(): def load_config():
"""Load runtime configuration from config.json and environment variables.""" """Compatibility wrapper for existing callers."""
config = { return load_runtime_config()
'PLEX_TOKEN': PLEX_TOKEN,
'SERVER_IP': SERVER_IP,
'PLAYLIST_NAME': PLAYLIST_NAME,
'SELECTED_STATIONS': ['journey_fm', 'spirit_fm']
}
if not os.path.exists('config.json'):
return config
try:
with open('config.json', 'r') as f:
file_config = json.load(f)
except Exception as e:
logger.error("Unable to parse config.json: %s", e)
return config
config['PLEX_TOKEN'] = file_config.get('PLEX_TOKEN', config['PLEX_TOKEN'])
config['SERVER_IP'] = file_config.get('SERVER_IP', config['SERVER_IP'])
config['PLAYLIST_NAME'] = file_config.get('PLAYLIST_NAME', config['PLAYLIST_NAME'])
selected_stations = file_config.get('SELECTED_STATIONS')
if isinstance(selected_stations, str) and selected_stations.strip():
config['SELECTED_STATIONS'] = [s.strip() for s in selected_stations.split(',') if s.strip()]
elif isinstance(selected_stations, list) and selected_stations:
config['SELECTED_STATIONS'] = [str(s).strip() for s in selected_stations if str(s).strip()]
return config
def init_history_db():
"""Initialize the history database"""
conn = sqlite3.connect('playlist_history.db')
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS history (
id INTEGER PRIMARY KEY,
date TEXT,
added_count INTEGER,
added_songs TEXT,
missing_count INTEGER,
missing_songs TEXT
)''')
conn.commit()
conn.close()
def save_history_entry(added_count, added_songs, missing_count, missing_songs):
"""Save a history entry to the database"""
conn = sqlite3.connect('playlist_history.db')
c = conn.cursor()
date = datetime.now().isoformat()
c.execute('INSERT INTO history (date, added_count, added_songs, missing_count, missing_songs) VALUES (?, ?, ?, ?, ?)',
(date, added_count, json.dumps(added_songs), missing_count, json.dumps(missing_songs)))
conn.commit()
conn.close()
def setup_scheduler(params):
"""Run the setup_scheduler.ps1 with parameters"""
cmd = ['powershell.exe', '-File', 'setup_scheduler.ps1']
for k, v in params.items():
cmd.extend(['-' + k, str(v)])
try:
subprocess.run(cmd, check=True)
logger.info("Scheduler setup completed.")
except subprocess.CalledProcessError as e:
logger.error("Scheduler setup failed: %s", e)
def main(): def main():
global PLEX_TOKEN, SERVER_IP, PLAYLIST_NAME """Run one update cycle and print a concise result summary."""
config = load_config()
PLEX_TOKEN = config['PLEX_TOKEN']
SERVER_IP = config['SERVER_IP']
PLAYLIST_NAME = config['PLAYLIST_NAME']
selected_stations = config['SELECTED_STATIONS']
if not PLEX_TOKEN or not SERVER_IP:
logger.error("Missing configuration: set PLEX_TOKEN and SERVER_IP in config.json or environment variables")
return
# Log start time
start_time = datetime.now()
# Initialize history database
init_history_db() init_history_db()
result = run_update_job(load_runtime_config())
summary = format_result_summary(result)
print(summary)
if result.get('status') == 'error':
logger.error(summary)
return result
# Scrape songs with selected stations
songs = scrape_recently_played(selected_stations)
# Connect to Plex via MyPlexAccount
try:
account = MyPlexAccount(token=PLEX_TOKEN)
resources = account.resources()
except Exception as e:
logger.error("Failed to authenticate with Plex: %s", e)
return
# Find the server by IP
server_resource = None
for r in resources:
for conn in r.connections:
if SERVER_IP in conn.address:
server_resource = r
break
if server_resource:
break
if not server_resource:
logger.error("Server not found at %s. Verify SERVER_IP and that Plex is reachable.", SERVER_IP)
return
try:
plex = server_resource.connect()
except Exception as e:
logger.error("Failed to connect to Plex server: %s", e)
return
# Find matching songs in local library (optional, since Plex search might suffice)
# But to filter only those in local library, perhaps skip or adjust
# For now, assume all scraped songs are to be added if found in Plex
# Create playlist
if songs:
added, added_songs, missing = create_playlist_in_plex(plex, songs, PLAYLIST_NAME)
else:
logger.info("No songs found.")
added, added_songs, missing = 0, [], []
# Save to history
save_history_entry(added, added_songs, len(missing), missing)
# Create Amazon buy list for missing songs
if missing:
# Read existing buy list
existing_songs = set()
if os.path.exists('amazon_buy_list.txt'):
try:
with open('amazon_buy_list.txt', 'r') as f:
content = f.read()
lines = content.split('\n')
i = 0
while i < len(lines):
if lines[i].startswith('Songs not in your library'):
i += 2 # Skip header
continue
if lines[i].strip() and not lines[i].startswith('http'):
if ' - ' in lines[i]:
artist, title = lines[i].split(' - ', 1)
existing_songs.add((artist.strip(), title.strip()))
i += 1
except Exception as e:
logger.warning("Failed reading amazon_buy_list.txt: %s", e)
# Filter new missing songs
new_missing = []
for song in missing:
key = (song['artist'], song['title'])
if key not in existing_songs:
new_missing.append(song)
existing_songs.add(key)
if new_missing:
with open('amazon_buy_list.txt', 'a') as f: # Append mode
if not os.path.exists('amazon_buy_list.txt') or os.path.getsize('amazon_buy_list.txt') == 0:
f.write("Songs not in your library - Amazon search links:\n\n")
for song in new_missing:
artist = song['artist']
title = song['title']
# Create Amazon search URL
query = urllib.parse.quote(f"{artist} {title}")
url = f"https://www.amazon.com/s?k={query}&i=digital-music"
f.write(f"{artist} - {title}\n{url}\n\n")
else:
logger.info("No new songs to add to buy list.")
logger.info("Buy list updated.")
# Log completion
end_time = datetime.now()
duration = (end_time - start_time).total_seconds()
def update_playlist(): def update_playlist():
"""Function to update playlist, callable from GUI""" """Compatibility wrapper used by the GUI."""
main() return main()
if __name__ == '__main__': if __name__ == '__main__':
main() main()
+1
View File
@@ -6,3 +6,4 @@ selenium
webdriver-manager webdriver-manager
PySide6 PySide6
matplotlib matplotlib
keyring
+38
View File
@@ -0,0 +1,38 @@
import logging
import os
import time
from journeyfm.config_store import load_runtime_config
from journeyfm.update_service import format_result_summary, run_update_job
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)
def get_sleep_seconds():
interval = os.getenv("UPDATE_INTERVAL", "15").strip()
unit = os.getenv("UPDATE_UNIT", "Minutes").strip().lower()
try:
interval_value = max(1, int(interval))
except ValueError:
interval_value = 15
if unit.startswith("hour"):
return interval_value * 3600
return interval_value * 60
def main():
os.environ.setdefault("JOURNEYFM_CONTAINER", "1")
run_mode = os.getenv("CONTAINER_RUN_MODE", "loop").strip().lower()
while True:
result = run_update_job(load_runtime_config())
logger.info(format_result_summary(result))
if run_mode == "once":
break
sleep_seconds = get_sleep_seconds()
logger.info("Sleeping %s seconds before next container sync", sleep_seconds)
time.sleep(sleep_seconds)
if __name__ == "__main__":
main()
+56 -19
View File
@@ -1,23 +1,60 @@
import requests import unittest
from bs4 import BeautifulSoup from pathlib import Path
url = 'https://www.myjourneyfm.com/recently-played/' from journeyfm.scraper_service import JourneyFMScraper, SpiritFMScraper, KLOVEScraper
response = requests.get(url)
soup = BeautifulSoup(response.content, 'html.parser')
# Try to find song elements
print("Song items:")
song_items = soup.find_all('div', class_='song-item')
for item in song_items:
print(item.prettify())
print("\nScripts with 'song' or 'played':") class ScraperParserTests(unittest.TestCase):
scripts = soup.find_all('script') def fixture_text(self, name):
for script in scripts: fixture_path = Path("tests") / "fixtures" / name
if script.string and ('song' in script.string.lower() or 'played' in script.string.lower()): return fixture_path.read_text(encoding="utf-8")
print(script.string[:2000])
print("\nAll links:") def test_parse_journey_fm_html(self):
for a in soup.find_all('a'): html = """
if 'recently' in a.get('href', '').lower(): <div class='rp-item'>
print(a) <h5 class='song-title'>No Survivors</h5>
<p class='song-artist'>Jeremy Camp</p>
</div>
<div class='rp-item'>
<h5 class='song-title'>Take It All Back</h5>
<p class='song-artist'>Tauren Wells</p>
</div>
"""
songs = JourneyFMScraper.parse_html(html)
self.assertEqual(2, len(songs))
self.assertEqual('No Survivors', songs[0]['title'])
self.assertEqual('Jeremy Camp', songs[0]['artist'])
def test_parse_journey_fm_current_markup_fixture(self):
html = self.fixture_text("journeyfm_recent_markup.html")
songs, parse_pattern = JourneyFMScraper.parse_html_with_telemetry(html)
self.assertEqual("top-rp-list", parse_pattern)
self.assertEqual(2, len(songs))
self.assertEqual('Have Your Way', songs[0]['title'])
self.assertEqual('Katy Nichole', songs[0]['artist'])
def test_parse_spirit_fm_text(self):
text = """
Mon 01:45PM Brandon Lake - Hard Fought Hallelujah
Mon 01:49PM Danny Gokey - Love God Love People
"""
songs = SpiritFMScraper.parse_text(text)
self.assertEqual(2, len(songs))
self.assertEqual('Brandon Lake', songs[0]['artist'])
self.assertEqual('Hard Fought Hallelujah', songs[0]['title'])
def test_parse_klove_fixture(self):
html = self.fixture_text("klove_recent_markup.html")
songs, parse_pattern = KLOVEScraper.parse_html_with_telemetry(html)
self.assertEqual("klove-anchor-by", parse_pattern)
self.assertGreaterEqual(len(songs), 3)
# Verify first song title and artist
titles = [s['title'] for s in songs]
artists = [s['artist'] for s in songs]
self.assertIn('Testimony', titles)
self.assertIn('Katy Nichole', artists)
self.assertIn('TobyMac', artists)
if __name__ == '__main__':
unittest.main()
+14
View File
@@ -0,0 +1,14 @@
<div class="top-rp-list" tabindex="0">
<div class="song-item">
<div class="title-artist">
<p>Have Your Way <span> by </span> Katy Nichole</p>
</div>
<p class="song-time">8:39 AM Today</p>
</div>
<div class="song-item">
<div class="title-artist">
<p>Jesus Does <span> by </span> We The Kingdom</p>
</div>
<p class="song-time">8:31 AM Today</p>
</div>
</div>
+33
View File
@@ -0,0 +1,33 @@
<!DOCTYPE html>
<html>
<body>
<!-- Minimal representative fixture matching K-LOVE's live page structure -->
<div class="song-card">
<a href="/music/artists/katy-nichole/testimony">
<img alt="Testimony" />
</a>
<div class="song-info">
<a href="/music/artists/katy-nichole/testimony">Testimony</a>
<span>By Katy Nichole</span>
</div>
</div>
<div class="song-card">
<a href="/music/artists/phil-wickham/i-believe">
<img alt="I Believe" />
</a>
<div class="song-info">
<a href="/music/artists/phil-wickham/i-believe">I Believe</a>
<span>By Phil Wickham</span>
</div>
</div>
<div class="song-card">
<a href="/music/artists/tobymac/cornerstone-feat-zach-williams">
<img alt="Cornerstone feat. Zach Williams" />
</a>
<div class="song-info">
<a href="/music/artists/tobymac/cornerstone-feat-zach-williams">Cornerstone feat. Zach Williams</a>
<span>By TobyMac</span>
</div>
</div>
</body>
</html>