diff --git a/.gitea/workflows/build.yml b/.gitea/workflows/build.yml index f81f91f..2cafb4c 100644 --- a/.gitea/workflows/build.yml +++ b/.gitea/workflows/build.yml @@ -27,6 +27,7 @@ jobs: - run: node tools/lyrics-test.js # lyrics: formats, wrong matches refused, remembered - run: node tools/mail-test.js # Outlook.com mail through Microsoft Graph: connect, refresh, read, send, privacy - run: node tools/mail-imap-test.js # Gmail and iCloud mail over IMAP/SMTP: connect, read, send, reply, privacy + - run: node tools/files-test.js # the Seafile pass-through: timeouts, retries, closed downloads, uploads - run: node tools/signal-test.js # Signal: linking, messages, sending, live updates - uses: docker/setup-qemu-action@v3 diff --git a/README.md b/README.md index c33308f..8f9b6a4 100644 --- a/README.md +++ b/README.md @@ -67,6 +67,7 @@ e.g. hub.necloud.us. Settings: `SEAFILE_URL`, `ONLYOFFICE_URL` env vars in docke - **Links**: your own bookmarks page. Add a name, address and optional group; filter as you type; opens in a new tab. Saved with your settings on the server, private to you, same on every device. Only http(s) addresses are accepted. - **Security and backups**: every response carries `nosniff`, a `frame-ancestors 'self'` policy and a same-origin referrer policy; writes that come from another website are refused; JSON files are written atomically (temp file, then rename). Admins get **Settings -> Storage -> Download a backup** (a .tar.gz of `/data`; keep it private). `/healthz` answers without signing in and the container has a health check. In-app OnlyOffice editing now requires `ONLYOFFICE_JWT_SECRET` (or `ALLOW_UNSIGNED_OO=1` for testing). `npm test` runs every test script; CI also runs `npm audit`. - **Planner and grouped menu**: Calendar and Tasks are one menu entry, **Planner**, with a Calendar | Tasks switch at the top (each still needs its own permission in Settings -> People, and `#/calendar` and `#/tasks` still work as addresses). Tasks with a due date appear on the calendar (month chips, week/day all-day strip, agenda) with a tick to complete them; tapping one opens the task. The menu, the phone "More" sheet and the Home tiles are grouped by kind: Work (Planner, Files, Office, Links), Communicate (Mail, Signal), Media (Music, Audiobooks, Podcast, Podcast Admin), with Home/Search at the top and Settings at the bottom. +- **Files, sturdier**: the Seafile pass-through gives Seafile 30 seconds to answer (2 minutes to start a file transfer), retries a failed read twice, says plainly when Seafile is slow or down, and stops reading from Seafile as soon as you close a download. In the app, an expired Seafile sign-in asks for your password again in a sheet and then carries on where you were. Each folder you open is saved (names, sizes, dates) so it shows instantly while the real list loads, and stays visible with a note when you are offline or Seafile cannot be reached; very large folders show 300 rows at a time with Show more, and the scroll position comes back when you return. Uploads wait for the connection instead of failing, retry with growing pauses (5 tries), and when a name is already there you choose Replace (the old version stays in the file's history), Keep both, Skip or Cancel. Deleting offers **Undo** (from Seafile's recycle bin), and **Recycle bin** in the folder menu (the three-dots button in the toolbar) restores anything deleted from the library. The same menu has **Search this library** and **Download as zip** (also in Select mode and on a folder's menu). **Share links** now offer a password and an expiry, list the links that already exist and can delete them. Text files (txt, md, json, csv, logs, config files, code) open in a viewer, and small ones can be edited and saved there. Not included: resumable chunked uploads (a dropped connection restarts that file; I could not confirm Seafile's chunk protocol without a server to test), editing files over 256 KB in the viewer. The recycle bin, Undo, zip, share options and text saving use Seafile's documented API but have only been tried against a fake Seafile, not a real one. - **Command palette and quick add**: press **Ctrl/Cmd-K** anywhere (or the Command palette button on Home) to jump to any page, create things, open a saved link or find a task/event. Typing a sentence offers **Add task**, and understands a date at the end ("Call mom tomorrow", "Pay rent in 3 days", "Dentist fri", "2026-12-25"). **Quick add** on Home does the same plus new event and new link. Search also finds your links. - **Tasks**: repeating tasks (daily, weekdays, weekly, monthly, yearly; finishing one moves it to its next date) and one level of subtasks, in NE Hub lists (iCloud reminder lists stay as Apple stores them). - **Calendar**: Month, Week, Day and Agenda. Week and Day show a time grid with overlapping events side by side, an all-day strip and a "now" line; tap an empty slot to add an event. diff --git a/package.json b/package.json index d938942..52bb6d8 100644 --- a/package.json +++ b/package.json @@ -5,7 +5,7 @@ "main": "server.js", "scripts": { "start": "node server.js", - "test": "for t in smoke tasks push users plex player lyrics signal mail mail-imap; do node tools/$t-test.js || exit 1; done", + "test": "for t in smoke tasks push users plex player lyrics signal mail mail-imap files; do node tools/$t-test.js || exit 1; done", "audit": "npm audit --omit=dev --audit-level=high" }, "keywords": [], diff --git a/public/app.js b/public/app.js index c3b637b..b5ff6a0 100644 --- a/public/app.js +++ b/public/app.js @@ -100,11 +100,23 @@ let cfg = { mail: '', podcast: 'https://podcast.necloud.us/', verse: 'https://ve // ---- Seafile client (via same-origin proxy) const sf = { token: () => store.get('sf_token'), - async req(path, opts = {}) { - const headers = { Authorization: 'Token ' + sf.token(), ...(opts.headers || {}) }; - const r = await fetch('/sf' + path, { ...opts, headers }); + /** A call to Seafile through the hub. Reads are retried; a slow or unreachable Seafile gives a clear message; an expired sign-in asks for the + * password again in place (sfReauth) and then carries on, so you do not lose where you were. */ + async req(path, opts = {}, tries = 0) { + const { _reauthed, timeout, ...rest } = opts, method = (rest.method || 'GET').toUpperCase(), idem = method === 'GET' || method === 'HEAD'; + const headers = { Authorization: 'Token ' + sf.token(), ...(rest.headers || {}) }; + let r; + try { r = await fetch('/sf' + path, { ...rest, headers, signal: rest.signal || AbortSignal.timeout(timeout || 45000) }); } + catch (e) { + if (idem && tries < 2 && e.name !== 'AbortError' && navigator.onLine) { await sleep(600 * (tries + 1)); return sf.req(path, opts, tries + 1); } + throw new Error(!navigator.onLine ? 'You are offline' : e.name === 'TimeoutError' ? 'Seafile took too long to answer. Try again in a moment.' : 'Could not reach the server'); + } + if (r.status === 401 && !_reauthed) { await sfReauth(); return sf.req(path, { ...opts, _reauthed: true }); } if (r.status === 401) { store.del('sf_token'); throw new Error('Session expired, sign in again'); } - if (!r.ok) throw new Error((await r.text().catch(() => '')).slice(0, 200) || 'Request failed (' + r.status + ')'); + if (!r.ok) { + let m = (await r.text().catch(() => '')).slice(0, 200); try { const j = JSON.parse(m); m = j.error || j.error_msg || j.detail || m; } catch {} + throw new Error(m || (r.status === 504 ? 'Seafile took too long to answer' : r.status >= 500 ? 'Seafile had a problem (' + r.status + ')' : 'Request failed (' + r.status + ')')); + } const t = await r.text(); try { return JSON.parse(t); } catch { return t; } }, @@ -121,7 +133,12 @@ const sf = { }, }; -const toast = (m) => { const t = $('#toast'); t.textContent = m; t.classList.add('on'); clearTimeout(toast.t); toast.t = setTimeout(() => t.classList.remove('on'), 2600); }; +/** A short message at the bottom; with `action` ({ label, run }) it has a button (Undo, say) and stays a little longer. */ +const toast = (m, action) => { + const t = $('#toast'); t.textContent = m; t.classList.toggle('act', !!action); + if (action) { const b = document.createElement('button'); b.textContent = action.label; b.onclick = () => { t.classList.remove('on'); action.run(); }; t.appendChild(b); } + t.classList.add('on'); clearTimeout(toast.t); toast.t = setTimeout(() => t.classList.remove('on'), action ? 9000 : 2600); +}; const ext = n => (n.split('.').pop() || '').toLowerCase(); const EDIT_EXT = new Set(['docx', 'doc', 'odt', 'rtf', 'xlsx', 'xls', 'ods', 'csv', 'pptx', 'ppt', 'odp', 'txt']); const OFFICE_EXT =new Set(['docx', 'doc', 'odt', 'rtf', 'xlsx', 'xls', 'ods', 'csv', 'pptx', 'ppt', 'odp', 'txt', 'pdf']); diff --git a/public/file-actions.js b/public/file-actions.js index 89792a6..66c0a56 100644 --- a/public/file-actions.js +++ b/public/file-actions.js @@ -1,6 +1,6 @@ 'use strict'; // ---- File actions -const PREVIEW = { img: 'jpg jpeg png gif webp svg bmp heic', pdf: 'pdf', video: 'mp4 mov m4v webm', audio: 'mp3 m4a wav flac ogg aac' }; +const PREVIEW = { img: 'jpg jpeg png gif webp svg bmp heic avif', pdf: 'pdf', video: 'mp4 mov m4v webm', audio: 'mp3 m4a wav flac ogg aac opus', text: 'txt md markdown json csv tsv log xml yml yaml ini conf cfg toml js ts css html htm py sh sql env gitignore' }; const previewKind = n => Object.keys(PREVIEW).find(k => PREVIEW[k].split(' ').includes(ext(n))); const FORM = { 'Content-Type': 'application/x-www-form-urlencoded' }; const encP = p => p.split('/').map(encodeURIComponent).join('/'); @@ -21,6 +21,7 @@ function openFile(repo, path, name, dir) { async function previewFile(repo, path, name) { addRecent(repo, path, name); + if (previewKind(name) === 'text') return textPreview(repo, path, name).catch(e => toast(e.message)); try { const url = (await fileUrl(repo, path)) + '?hubinline=1', kind = previewKind(name); const media = kind === 'img' ? `${esc(name)}` @@ -32,19 +33,7 @@ async function previewFile(repo, path, name) { } catch (e) { toast(e.message); } } -async function shareLink(repo, path, isDir) { - const p = isDir ? path.replace(/\/?$/, '/') : path; - try { - let link; - try { link = (await sf.req('/api/v2.1/share-links/', { method: 'POST', headers: FORM, body: new URLSearchParams({ repo_id: repo, path: p }) })).link; } - catch { link = (await sf.req(`/api/v2.1/share-links/?repo_id=${encodeURIComponent(repo)}&path=${encodeURIComponent(p)}`))[0]?.link; } - if (!link) throw new Error('Could not create a share link'); - const el = openSheet('Share link', p.split('/').filter(Boolean).pop() || '', ` -
${navigator.share ? '' : ''}
`); - el.querySelector('#cp1').onclick = async () => { try { await navigator.clipboard.writeText(link); toast('Link copied'); closeSheet(); } catch { el.querySelector('#sl').select(); toast('Press copy on the selected link'); } }; - const sh = el.querySelector('#sh1'); if (sh) sh.onclick = () => navigator.share({ url: link }).catch(() => {}); - } catch (e) { toast(e.message); } -} +const shareLink = (repo, path, isDir) => shareSheet(repo, path, isDir); function fileSheet(repo, path, name, dir) { const enc = encodeURIComponent, acts = [], ro = libPerm[repo] === 'r', it = { name, type: 'file' }; @@ -66,6 +55,7 @@ function dirSheet(repo, path, name, dir) { actionSheet(name, 'Folder', [ { label: 'Open', icon: 'files', run: () => { location.hash = '#/files/' + encodeURIComponent(repo) + path.split('/').filter(Boolean).map(s => '/' + encodeURIComponent(s)).join(''); } }, { label: 'Share link', icon: 'copy', run: () => shareLink(repo, path, true) }, + { label: 'Download as zip', icon: 'dl', run: () => zipDownload(repo, dir, [name]) }, ...(ro ? [] : [{ label: 'Rename', icon: 'edit', run: () => renameItem(repo, path, name, dir, true) }, { label: 'Move to…', icon: 'files', run: () => moveSheet('move', repo, dir, [it]) }]), { label: 'Copy to…', icon: 'copy', run: () => moveSheet('copy', repo, dir, [it]) }, ...(ro ? [] : [{ label: 'Delete', icon: 'trash', danger: true, run: () => deleteItems(repo, dir, [it]) }]), diff --git a/public/files-extra.js b/public/files-extra.js new file mode 100644 index 0000000..75b0b52 --- /dev/null +++ b/public/files-extra.js @@ -0,0 +1,209 @@ +'use strict'; +// Files, the sturdier parts: signing in again in place, a saved copy of each folder (instant, and still there offline), upload name clashes, +// undo and the recycle bin, search in a library, zip downloads, share link options, and text files you can read and edit. +// Uses sf, store, ME, jfetch, openSheet, closeSheet, confirmSheet, actionSheet, toast, esc, icon, fmtSize, fmtDate, joinP, libNames, libPerm, cfg from the other scripts. + +// ---- Seafile sign-in expired: ask again without leaving the page, then the original request carries on +let reauthP = null; +function sfReauth() { + if (reauthP) return reauthP; + reauthP = new Promise((ok, no) => { + let settled = false; + const finish = (good, err) => { if (settled) return; settled = true; reauthP = null; good ? ok() : (store.del('sf_token'), no(err || new Error('Session expired, sign in again'))); }; + const el = openSheet('Sign in to Seafile again', 'Your Seafile sign-in expired. You will stay where you are.', `
+ +
`); + el.querySelector('#sx').onclick = () => { closeSheet(); finish(false); }; + el.querySelector('.scrim').addEventListener('click', () => finish(false)); + el.querySelector('#rf').onsubmit = async e => { + e.preventDefault(); const b = el.querySelector('#rb'); b.disabled = true; el.querySelector('#re').textContent = ''; + try { await sf.login(el.querySelector('#ru').value.trim(), el.querySelector('#rp').value, el.querySelector('#ro').value.trim() || undefined); closeSheet(); finish(true); } + catch (err) { b.disabled = false; if (err.otp) el.querySelector('#rotp').hidden = false; el.querySelector('#re').textContent = err.message; } + }; + setTimeout(() => el.querySelector('#ru').focus(), 60); + }); + return reauthP; +} + +// ---- A saved copy of every folder you open (names, sizes, dates only): shown at once while the real list loads, and when Seafile cannot be reached +const dcIndex = () => 'dc:index:' + (ME ? ME.id : ''); +const dcKey = (repo, dir) => 'dc:' + (ME ? ME.id : '') + ':' + repo + ':' + dir; +function dcGet(repo, dir) { try { const o = JSON.parse(localStorage.getItem(dcKey(repo, dir))); return o && Array.isArray(o.items) ? o : null; } catch { return null; } } +function dcPut(repo, dir, items) { + const key = dcKey(repo, dir), body = JSON.stringify({ t: Date.now(), items: items.slice(0, 1500).map(x => ({ id: x.id, name: x.name, type: x.type, size: x.size, mtime: x.mtime })) }); + const touch = () => { const idx = (JSON.parse(localStorage.getItem(dcIndex()) || '[]')).filter(k => k !== key); idx.unshift(key); while (idx.length > 40) localStorage.removeItem(idx.pop()); localStorage.setItem(dcIndex(), JSON.stringify(idx)); }; + try { localStorage.setItem(key, body); touch(); } + catch { try { const idx = JSON.parse(localStorage.getItem(dcIndex()) || '[]'); for (const k of idx.splice(Math.ceil(idx.length / 2))) localStorage.removeItem(k); localStorage.setItem(dcIndex(), JSON.stringify(idx)); localStorage.setItem(key, body); touch(); } catch { /* storage is full: go without */ } } +} +const fScroll = new Map(); // where each folder was scrolled to +document.addEventListener('scroll', e => { if (e.target && e.target.id === 'view' && /^#\/files\/./.test(location.hash)) fScroll.set(location.hash, e.target.scrollTop); }, true); + +// ---- Uploading a file whose name is already there +async function resolveUploadConflicts(repo, dir, entries) { + let existing; + try { existing = new Set((await sf.req(`/api2/repos/${encodeURIComponent(repo)}/dir/?p=${encodeURIComponent(dir)}`)).map(x => x.name.toLowerCase())); } catch { return entries; } + const clash = entries.filter(e => !e.rel && existing.has(e.file.name.toLowerCase())); // (files inside dropped folders keep Seafile's own renaming) + if (!clash.length) return entries; + const choice = await new Promise(ok => { + const el = openSheet(clash.length === 1 ? 'Replace “' + clash[0].file.name + '”?' : clash.length + ' files already exist here', 'Replacing keeps the old version in the file\'s history.', + `${clash.length > 1 ? `
${clash.map(c => `
${esc(c.file.name)}
`).join('')}
` : ''} +
+
`); + let done = false; const pick = v => { if (done) return; done = true; closeSheet(); ok(v); }; + el.querySelector('#cr').onclick = () => pick('replace'); el.querySelector('#ck').onclick = () => pick('both'); el.querySelector('#cs').onclick = () => pick('skip'); el.querySelector('#cc').onclick = () => pick('cancel'); + el.querySelector('.scrim').addEventListener('click', () => pick('cancel')); + }); + if (choice === 'cancel') return []; + if (choice === 'skip') return entries.filter(e => !clash.includes(e)); + return entries.map(e => clash.includes(e) ? { ...e, replace: choice === 'replace' } : e); +} +const waitOnline = () => new Promise(ok => { if (navigator.onLine) return ok(); const f = () => { removeEventListener('online', f); ok(); }; addEventListener('online', f); }); + +// ---- Undo a delete, and the recycle bin (Seafile keeps deleted files for a while) +const trashWhen = t => typeof t === 'number' ? t * 1000 : Date.parse(t) || 0; +async function trashList(repo, path = '/', limit = 100) { + const r = await sf.req(`/api/v2.1/repos/${encodeURIComponent(repo)}/trash/?path=${encodeURIComponent(path)}&limit=${limit}`); + return (r && (r.items || r.data)) || []; +} +async function trashRestore(repo, t) { + const parent = (t.parent_dir || '/').replace(/\/+$/, '') || ''; + await sf.req(`/api2/repos/${encodeURIComponent(repo)}/${t.is_dir ? 'dir' : 'file'}/revert/`, { method: 'POST', headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, body: new URLSearchParams({ p: parent + '/' + t.obj_name, commit_id: t.commit_id }) }); +} +/** Right after a delete: find the just-deleted items in the recycle bin and offer Undo. (Quietly nothing if Seafile has no recycle bin API.) */ +async function offerUndo(repo, dir, items, afterRestore) { + try { + const names = new Set(items.map(x => x.name)), newest = new Map(); + for (const t of await trashList(repo, dir, 60)) if (names.has(t.obj_name) && Date.now() - trashWhen(t.deleted_time) < 5 * 60e3 && (!newest.has(t.obj_name) || trashWhen(t.deleted_time) > trashWhen(newest.get(t.obj_name).deleted_time))) newest.set(t.obj_name, t); + if (!newest.size) return false; + toast(items.length === 1 ? 'Deleted “' + items[0].name + '”' : items.length + ' items deleted', { label: 'Undo', run: async () => { + let bad = 0; for (const t of newest.values()) { try { await trashRestore(repo, t); } catch { bad++; } } + toast(bad ? 'Could not restore ' + bad + ' item' + (bad === 1 ? '' : 's') : 'Restored'); if (afterRestore) afterRestore(); + } }); + return true; + } catch { return false; /* no undo available */ } +} +async function recycleBin(repo, dir) { + const el = openSheet('Recycle bin', (libNames[repo] || '') + (dir && dir !== '/' ? ' · ' + dir : ''), '

Loading…

'); + el.querySelector('.sheet').classList.add('wide'); + try { + const list = (await trashList(repo, dir || '/', 100)).sort((a, b) => trashWhen(b.deleted_time) - trashWhen(a.deleted_time)); + if (!document.body.contains(el)) return; + const body = el.querySelector('.empty'); + body.outerHTML = list.length ? `
${list.map((t, i) => `
${icon(t.is_dir ? 'files' : 'file')} + ${esc(t.obj_name)}${esc((t.parent_dir || '/'))} · ${t.is_dir ? 'folder' : esc(fmtSize(t.size || 0))} · deleted ${esc(new Date(trashWhen(t.deleted_time)).toLocaleString([], { dateStyle: 'medium', timeStyle: 'short' }))} + ${libPerm[repo] === 'r' ? '' : ``}
`).join('')}

Showing the 100 most recent. Seafile empties its recycle bin after a while.

+
` + : '

Nothing here. Deleted files and folders from this library show up here so you can get them back.

'; + el.querySelector('#sx').onclick = closeSheet; + el.querySelectorAll('[data-rs]').forEach(b => b.onclick = async () => { + b.disabled = true; try { await trashRestore(repo, list[+b.dataset.rs]); b.textContent = 'Restored'; if (fv && fv.repo === repo) fv.reload(true); } catch (e) { b.disabled = false; toast(e.message); } + }); + } catch (e) { const b = el.querySelector('.empty'); if (b) b.textContent = e.message; } +} + +// ---- Search a whole library (names, from the Seafile index when it has one, otherwise by walking the folders) +function libSearch(repo) { + textSheet('Search this library', 'File or folder name', 'Search', async term => { + const el = openSheet('Results for “' + term + '”', libNames[repo] || '', '

Searching…

'); el.querySelector('.sheet').classList.add('wide'); + try { + const { results } = await searchFiles(term, [{ id: repo, name: libNames[repo] || '' }]); + const mine = results.filter(r => r.repo_id === repo).slice(0, 60); + if (!document.body.contains(el)) return; + el.querySelector('.empty').outerHTML = `${mine.length ? `
${mine.map((r, i) => ``).join('')}
` : '

Nothing found.

'}
`; + el.querySelector('#sx').onclick = closeSheet; + el.querySelectorAll('[data-r]').forEach(b => b.onclick = () => { + const r = mine[+b.dataset.r], parent = (r.fullpath || '').replace(/\/[^/]*$/, '') || '/'; closeSheet(); + if (r.is_dir) location.hash = '#/files/' + encodeURIComponent(repo) + (r.fullpath || '').split('/').filter(Boolean).map(s => '/' + encodeURIComponent(s)).join(''); + else openFile(repo, r.fullpath, r.name, parent); + }); + } catch (e) { const b = el.querySelector('.empty'); if (b) b.textContent = e.message; } + }); +} + +// ---- Download several items, or a whole folder, as one zip (Seafile builds it; we wait for it) +async function zipDownload(repo, dir, names) { + try { + const q = new URLSearchParams({ parent_dir: dir }); names.forEach(n => q.append('dirents', n)); + const task = await sf.req(`/api/v2.1/repos/${encodeURIComponent(repo)}/zip-task/?${q}`), token = task.zip_token; + if (!token) throw new Error('Seafile could not make a zip of that'); + for (let i = 0; i < 300; i++) { + const p = await sf.req(`/api/v2.1/query-zip-progress/?token=${encodeURIComponent(token)}`); + const total = +p.total || 0, zipped = +p.zipped || 0; + if (p.failed || p.failed_reason) throw new Error('Seafile could not make the zip' + (p.failed_reason ? ': ' + p.failed_reason : '')); + if (total && zipped >= total) break; + toast('Preparing the zip… ' + (total ? Math.round(100 * zipped / total) + '%' : '')); + await sleep(i < 5 ? 500 : 1500); + if (i === 299) throw new Error('The zip is taking too long. Try fewer files.'); + } + const a = document.createElement('a'); a.href = '/sf/seafhttp/zip/' + encodeURIComponent(token); a.download = ''; document.body.appendChild(a); a.click(); a.remove(); + toast('Download started'); + } catch (e) { toast(e.message); } +} + +// ---- Share links with a password and an expiry, and the ones that already exist +async function shareSheet(repo, path, isDir) { + const p = isDir ? path.replace(/\/?$/, '/') : path, enc = encodeURIComponent, FORMH = { 'Content-Type': 'application/x-www-form-urlencoded' }; + const el = openSheet('Share link', p.split('/').filter(Boolean).pop() || (libNames[repo] || ''), '

Loading…

'); + const draw = async () => { + let links = []; try { links = (await sf.req(`/api/v2.1/share-links/?repo_id=${enc(repo)}&path=${enc(p)}`)) || []; } catch (e) { if (!document.body.contains(el)) return; } + if (!document.body.contains(el)) return; + const host = el.querySelector('.empty') || el.querySelector('#shbody'); + const html = `
${links.length ? `
${links.map((l, i) => `
${l.expire_date ? (l.is_expired ? 'Expired ' : 'Expires ') + esc(new Date(l.expire_date).toLocaleDateString([], { dateStyle: 'medium' })) : 'Never expires'}${l.is_expired ? '' : ''} +
`).join('')}
` : ''} +

${links.length ? 'New link' : 'Create a link'}

+ + +
`; + host.outerHTML = html; + el.querySelector('#sx').onclick = closeSheet; + el.querySelectorAll('[data-cp]').forEach(b => b.onclick = async () => { const l = links[+b.dataset.cp]; try { await navigator.clipboard.writeText(l.link); toast('Link copied'); } catch { toast('Select the link and copy it'); } }); + el.querySelectorAll('[data-rm]').forEach(b => b.onclick = () => confirmSheet('Delete this link?', 'Anyone using it loses access.', 'Delete', async () => { try { await sf.req(`/api/v2.1/share-links/${enc(links[+b.dataset.rm].token)}/`, { method: 'DELETE' }); shareSheet(repo, path, isDir); } catch (e) { toast(e.message); } })); + el.querySelector('#shf').onsubmit = async e => { + e.preventDefault(); const pw = el.querySelector('#shp').value, days = el.querySelector('#she').value, btn = el.querySelector('#shc'); + if (pw && pw.length < 8) { el.querySelector('#she2').textContent = 'A password needs at least 8 characters.'; return; } + btn.disabled = true; + try { + const body = new URLSearchParams({ repo_id: repo, path: p }); if (pw) body.set('password', pw); if (days) body.set('expire_days', days); + const made = await sf.req('/api/v2.1/share-links/', { method: 'POST', headers: FORMH, body }); + try { await navigator.clipboard.writeText(made.link); toast('Link created and copied'); } catch { toast('Link created'); } + shareSheet(repo, path, isDir); + } catch (err) { btn.disabled = false; el.querySelector('#she2').textContent = err.message; } + }; + }; + draw(); +} + +// ---- Text files: read them here, and edit small ones (the old version stays in the file's history) +const TEXT_EXT = new Set('txt md markdown json csv tsv log xml yml yaml ini conf cfg toml js ts css html htm py sh sql env gitignore'.split(' ')); +async function textPreview(repo, path, name) { + const dir = path.replace(/\/[^/]*$/, '') || '/', ro = libPerm[repo] === 'r', url = (await fileUrl(repo, path)) + '?hubinline=1'; + const el = openSheet(name, '', '

Loading…

'); el.querySelector('.sheet').classList.add('wide'); + const dl = `${icon('dl')}Download`; + try { + const r = await fetch(url); if (!r.ok) throw new Error('Could not open the file (' + r.status + ')'); + const len = +r.headers.get('content-length') || 0; + if (len > 1024 * 1024) { await r.body.cancel(); el.querySelector('.empty').outerHTML = `

This file is ${esc(fmtSize(len))}, too big to show here.

${dl}
`; el.querySelector('#sx').onclick = closeSheet; return; } + let text = await r.text(); if (text.length > 1024 * 1024) text = text.slice(0, 1024 * 1024); + const view = () => { + if (!document.body.contains(el)) return; + const host = el.querySelector('#tpb') || el.querySelector('.empty'); + host.outerHTML = `
${esc(text)}
${ro || len > 256 * 1024 ? '' : ''}${dl}
`; + el.querySelector('#sx').onclick = closeSheet; const ed = el.querySelector('#ted'); if (ed) ed.onclick = edit; + }; + const edit = () => { + el.querySelector('#tpb').outerHTML = `
`; + el.querySelector('#tcx').onclick = view; + el.querySelector('#tsv').onclick = async () => { + const b = el.querySelector('#tsv'); b.disabled = true; const next = el.querySelector('#tta').value; + try { + const link = await sf.req(`/api2/repos/${encodeURIComponent(repo)}/update-link/?p=${encodeURIComponent(dir)}`), fd = new FormData(); + fd.append('target_file', path); fd.append('parent_dir', dir); fd.append('file', new Blob([next], { type: 'text/plain' }), name); + const up = await fetch('/sf' + new URL(link, cfg.seafile).pathname, { method: 'POST', body: fd }); + if (!up.ok) throw new Error('Could not save (' + up.status + ')'); + text = next; toast('Saved'); view(); if (fv && fv.repo === repo) fv.reload(true); + } catch (e) { b.disabled = false; el.querySelector('#ter').textContent = e.message; } + }; + }; + view(); + } catch (e) { const b = el.querySelector('.empty'); if (b) b.textContent = e.message; } +} diff --git a/public/files.js b/public/files.js index a1e7a2d..b11ffa5 100644 --- a/public/files.js +++ b/public/files.js @@ -71,7 +71,8 @@ async function loadLibs(active) { // ---- Files: browse, select, move/copy/delete, rename, upload (drag in from the computer, drag a file out to download) let fv = null, libsP = Promise.resolve(), libNames = {}, libPerm = {}; const SORTS = [['name', 'Name'], ['mtime', 'Date modified'], ['size', 'Size'], ['type', 'Kind']]; -const nameCmp = (a, b) => a.name.localeCompare(b.name, undefined, { numeric: true, sensitivity: 'base' }); +const collator = new Intl.Collator(undefined, { numeric: true, sensitivity: 'base' }); // (one collator is much faster than localeCompare in a big folder) +const nameCmp = (a, b) => collator.compare(a.name, b.name); function sortItems(items, s) { const by = { name: nameCmp, mtime: (a, b) => a.mtime - b.mtime, size: (a, b) => (a.size || 0) - (b.size || 0), type: (a, b) => ext(a.name).localeCompare(ext(b.name)) }[s.k] || nameCmp; return [...items].sort((a, b) => a.type !== b.type ? (a.type === 'dir' ? -1 : 1) // folders stay on top @@ -97,8 +98,7 @@ function deleteItems(repo, dir, items) { const one = items.length === 1; confirmSheet(one ? (items[0].type === 'dir' ? 'Delete folder?' : 'Delete file?') : `Delete ${items.length} items?`, one ? items[0].name + (items[0].type === 'dir' ? ' and everything in it' : '') : 'Folders are deleted with everything inside them', 'Delete', async () => { - try { await removeItems(repo, dir, items); toast(one ? 'Deleted' : items.length + ' items deleted'); } catch (e) { toast(e.message); } - afterChange(repo, dir); + try { await removeItems(repo, dir, items); afterChange(repo, dir); if (!(await offerUndo(repo, dir, items, () => afterChange(repo, dir)))) toast(one ? 'Deleted' : items.length + ' items deleted'); } catch (e) { toast(e.message); afterChange(repo, dir); } }); } async function transferItems(mode, repo, dir, names, dst) { @@ -130,9 +130,11 @@ async function downloadItem(repo, path) { // ---- Uploads: a queue that survives leaving the folder, with progress, retries and a way to cancel const uq = { items: [], running: 0 }; const upActive = () => uq.items.some(x => x.state === 'wait' || x.state === 'up'); -function enqueueUploads(repo, dir, entries) { +async function enqueueUploads(repo, dir, entries) { if (!entries.length) return; - for (const e of entries) uq.items.push({ repo, dir, file: e.file, rel: e.rel || '', pct: 0, state: 'wait', tries: 0 }); + entries = await resolveUploadConflicts(repo, dir, entries); // a name that is already there: replace, keep both, or skip + if (!entries.length) return; + for (const e of entries) uq.items.push({ repo, dir, file: e.file, rel: e.rel || '', replace: !!e.replace, pct: 0, state: 'wait', tries: 0 }); drawUpBar(); pumpUploads(); } function pumpUploads() { @@ -151,7 +153,7 @@ async function sendUpload(it) { const link = await sf.req(`/api2/repos/${enc(it.repo)}/upload-link/?p=${enc(it.dir)}`); // a fresh link each time, so an expired one never blocks a retry await new Promise((ok, no) => { const x = it.xhr = new XMLHttpRequest(), fd = new FormData(); - fd.append('parent_dir', it.dir); fd.append('replace', '0'); if (it.rel) fd.append('relative_path', it.rel); fd.append('file', it.file, it.file.name); + fd.append('parent_dir', it.dir); fd.append('replace', it.replace ? '1' : '0'); if (it.rel) fd.append('relative_path', it.rel); fd.append('file', it.file, it.file.name); x.open('POST', '/sf' + new URL(link, cfg.seafile).pathname + '?ret-json=1'); x.upload.onprogress = e => { if (e.lengthComputable) { it.pct = e.loaded / e.total; drawUpBar(); } }; x.onload = () => x.status >= 200 && x.status < 300 ? ok() : no(Object.assign(new Error(x.status === 413 ? 'Too large for the server' : x.status === 403 ? 'Not allowed (or out of space)' : 'Upload failed (' + x.status + ')'), { fatal: x.status >= 400 && x.status < 500 })); @@ -162,8 +164,10 @@ async function sendUpload(it) { return; } catch (e) { if (it.cancelled) throw e; - if (e.fatal || !sf.token() || it.tries >= 3) throw new Error(it.file.name + ': ' + e.message); - await sleep(1500 * it.tries); + if (e.fatal || !sf.token()) throw new Error(it.file.name + ': ' + e.message); + if (!navigator.onLine) { it.waiting = true; drawUpBar(); await waitOnline(); it.waiting = false; it.tries = Math.max(0, it.tries - 1); } // no connection: wait for it instead of failing + else if (it.tries >= 5) throw new Error(it.file.name + ': ' + e.message); + else await sleep(Math.min(15000, 1500 * 2 ** (it.tries - 1))); } } } @@ -181,7 +185,7 @@ function drawUpBar() { if (live.length) { const total = uq.items.length, doneN = uq.items.filter(x => x.state === 'done').length, cur = live.find(x => x.state === 'up') || live[0]; const pct = Math.round(((doneN + live.reduce((a, x) => a + (x.state === 'up' ? x.pct : 0), 0)) / total) * 100); - b.innerHTML = `
Uploading ${esc(live.length === 1 ? cur.file.name : live.length + ' files')}${pct}%
`; + b.innerHTML = `
${live.some(x => x.waiting) ? 'Waiting for a connection…' : 'Uploading ' + esc(live.length === 1 ? cur.file.name : live.length + ' files')}${pct}%${cur.tries > 1 && !cur.waiting ? ' · retrying' : ''}
`; $('#upx').onclick = () => { uq.items.forEach(x => { if (x.state === 'wait') x.state = 'cancel'; if (x.state === 'up') { x.cancelled = true; x.xhr && x.xhr.abort(); } }); uq.items = uq.items.filter(x => x.state === 'fail'); drawUpBar(); }; } else { b.innerHTML = `
${bad.length} ${bad.length === 1 ? 'upload' : 'uploads'} failed${esc(bad[0].err || '')}
`; @@ -221,7 +225,7 @@ async function loadDir(repo, dir) { const hash = p => '#/files/' + enc(repo) + p.split('/').filter(Boolean).map(s => '/' + enc(s)).join(''); const segs = dir.split('/').filter(Boolean); const crumbs = `` + segs.map((s, i) => `/`).join(''); - const v = fv = { root: main, repo, dir, items: [], sel: new Set(), selecting: false, last: null, filter: '', tok: null }; + const v = fv = { root: main, repo, dir, items: [], sel: new Set(), selecting: false, last: null, filter: '', tok: null, limit: 300, stale: false }; const sortCfg = () => store.get('files_sort', { k: 'name', d: 1 }); const byName = n => v.items.find(x => x.name === n); const selItems = () => v.items.filter(x => v.sel.has(x.name)); @@ -239,12 +243,12 @@ async function loadDir(repo, dir) {
${ro ? '' : ``} ${ro ? '' : ``} -
` + ` : `
${icon('back')}Libraries ${ro ? '' : ``} - +
`; }; const drawItems = () => { @@ -256,8 +260,9 @@ async function loadDir(repo, dir) { ${v.selecting ? '' : ``}`; }; - box.innerHTML = list.length ? `
${list.map(row).join('')}
` - : `

${v.items.length ? 'Nothing matches “' + esc(v.filter) + '”' : ro ? 'This folder is empty' : 'This folder is empty. Drop files here or tap Upload.'}

`; + box.innerHTML = (v.stale ? `

${v.stale === 'old' ? 'Showing the last saved list. ' : ''}${esc(v.staleWhy || '')}

` : '') + (list.length ? `
${list.slice(0, v.limit).map(row).join('')}
${list.length > v.limit ? `
` : ''}` + : `

${v.items.length ? 'Nothing matches “' + esc(v.filter) + '”' : ro ? 'This folder is empty' : 'This folder is empty. Drop files here or tap Upload.'}

`); + const fm = $('#fmore'); if (fm) fm.onclick = () => { v.limit += 300; drawItems(); }; const dirs = v.items.filter(x => x.type === 'dir').length, files = v.items.length - dirs, size = v.items.reduce((a, x) => a + (x.size || 0), 0); $('#fsum').textContent = v.items.length ? [dirs && dirs + (dirs === 1 ? ' folder' : ' folders'), files && files + (files === 1 ? ' file' : ' files'), size && fmtSize(size)].filter(Boolean).join(' · ') : ''; }; @@ -275,6 +280,7 @@ async function loadDir(repo, dir) { const p = joinP(dir, it.name), d = it.type === 'dir', a = [{ label: 'Share link', icon: 'copy', run: () => shareLink(repo, p, d) }]; if (!ro) a.push({ label: 'Rename', icon: 'edit', run: () => renameItem(repo, p, it.name, dir, d) }); if (!d) a.push({ label: 'Download', icon: 'dl', run: () => downloadItem(repo, p) }); + if (d) a.push({ label: 'Download as zip', icon: 'dl', run: () => zipDownload(repo, dir, [it.name]) }); if (!d) a.push({ label: 'Version history', icon: 'refresh', run: () => fileHistory(repo, p, it.name, ro) }); return a; }; @@ -292,6 +298,13 @@ async function loadDir(repo, dir) { done: () => { v.selecting = false; v.sel.clear(); redraw(); }, sel: () => { v.selecting = true; redraw(); }, refresh: () => v.reload(), + zip: () => v.sel.size && zipDownload(repo, dir, [...v.sel]), + lib: () => actionSheet(dir === '/' ? (libNames[repo] || 'Library') : dir.split('/').pop(), 'This folder', [ + { label: 'Search this library', icon: 'refresh', run: () => libSearch(repo) }, + { label: 'Recycle bin', icon: 'trash', run: () => recycleBin(repo, dir) }, + ...(dir === '/' ? [] : [{ label: 'Download this folder as zip', icon: 'dl', run: () => zipDownload(repo, dir.replace(/\/[^/]*$/, '') || '/', [dir.split('/').pop()]) }]), + ...(v.items.length ? [{ label: 'Download everything here as zip', icon: 'dl', run: () => zipDownload(repo, dir, v.items.map(x => x.name)) }] : []), + ]), sort: () => { const c = sortCfg(); actionSheet('Sort by', 'Folders stay on top', SORTS.map(([k, l]) => ({ label: l + (c.k === k ? (c.d > 0 ? ' ↑' : ' ↓') : ''), icon: 'sort', run: () => { store.set('files_sort', { k, d: c.k === k ? -c.d : 1 }); drawItems(); } }))); }, up: () => actionSheet('Upload', '', [{ label: 'Files', icon: 'up', run: () => $('#upf').click() }, { label: 'A folder', icon: 'newdir', run: () => $('#upd2').click() }]), @@ -368,15 +381,27 @@ async function loadDir(repo, dir) { }; v.reload = async quiet => { - const tok = v.tok = Symbol(); if (!quiet) $('#items').innerHTML = '

Loading…

'; + const tok = v.tok = Symbol(), saved = dcGet(repo, dir); + const restore = () => { if (v.scrolled) return; v.scrolled = true; const y = fScroll.get(location.hash); if (y) requestAnimationFrame(() => { const view = $('#view'); if (view) view.scrollTop = y; }); }; + if (!quiet) { + if (!v.items.length && saved) { v.items = saved.items; v.stale = 'old'; v.staleWhy = 'Updating…'; redraw(); restore(); } // the saved list at once, the real one right behind it + else $('#items').innerHTML = '

Loading…

'; + } try { const items = await sf.req(`/api2/repos/${enc(repo)}/dir/?p=${enc(dir)}`); if (v.tok !== tok || !document.contains(main)) return; - v.items = items; for (const n of [...v.sel]) if (!byName(n)) v.sel.delete(n); - redraw(); + v.items = items; v.stale = false; for (const n of [...v.sel]) if (!byName(n)) v.sel.delete(n); + dcPut(repo, dir, items); + redraw(); restore(); } catch (e) { if (v.tok !== tok || !document.contains(main)) return; if (!sf.token()) return route(); + const why = navigator.onLine ? e.message : 'You are offline.'; + if (saved && v.items.length) { // keep showing the saved list instead of an error + v.stale = 'old'; v.staleWhy = why + ' Saved ' + fmtDate(saved.t / 1000) + '. '; redraw(); restore(); + const n = $('#items .sub'); if (n) { n.insertAdjacentHTML('beforeend', 'Try again'); $('#fretry2').onclick = ev => { ev.preventDefault(); v.reload(); }; } + return; + } $('#items').innerHTML = `

${esc(navigator.onLine ? e.message : 'You are offline. ' + e.message)}

`; $('#fretry').onclick = () => v.reload(); } diff --git a/public/index.html b/public/index.html index 15ec87c..f79b3c8 100644 --- a/public/index.html +++ b/public/index.html @@ -70,6 +70,7 @@ + diff --git a/public/style.css b/public/style.css index 9d05e9d..b337d01 100644 --- a/public/style.css +++ b/public/style.css @@ -83,6 +83,8 @@ label { display: block; font-size: 13px; color: var(--muted); margin: 14px 0 6px background: var(--text); color: var(--bg); padding: 10px 16px; border-radius: 12px; opacity: 0; pointer-events: none; transition: .2s; max-width: 90vw; z-index: 10; } #toast.on { opacity: 1; transform: translate(-50%, 0); } +#toast.act { pointer-events: auto; display: flex; align-items: center; gap: 14px; } +#toast.act button { border: 0; background: none; color: var(--accent-fg, #7aa2ff); font-weight: 700; padding: 0 4px; font-size: 15px; } /* Tablet / iPad (incl. 12.9" portrait 1024 + landscape 1366) and desktop: side rail */ @media (min-width: 768px) { @@ -570,3 +572,7 @@ body.nav-mini .ngrp span { display: none; } .sgrp a.on { background: var(--accent); color: var(--accent-t); font-weight: 600; } .psel { width: auto; min-height: 38px; padding: 0 12px; border-radius: 999px; font-size: 14px; } @media (max-width: 560px) { .ptop h1 { width: 100%; } .pspace { display: none; } .ptop .btn { margin-left: auto; } } + +/* Text preview and editing */ +.ptxt { white-space: pre-wrap; overflow-wrap: anywhere; max-height: 60dvh; overflow: auto; background: var(--panel2); border-radius: 12px; padding: 12px 14px; font: 13px/1.5 ui-monospace, Menlo, Consolas, monospace; margin: 8px 0; } +.ptxt-edit { width: 100%; height: 55dvh; font: 13px/1.5 ui-monospace, Menlo, Consolas, monospace; padding: 12px 14px; border-radius: 12px; border: 1px solid var(--line); background: var(--panel); resize: vertical; } diff --git a/server.js b/server.js index 410b07d..473cedb 100644 --- a/server.js +++ b/server.js @@ -9,6 +9,7 @@ const { Readable } = require('stream'); const PORT = process.env.PORT || 8080; const SEAFILE_URL = (process.env.SEAFILE_URL || 'https://seafile.necloud.us').replace(/\/$/, ''); +const SEAFILE_TIMEOUT = Math.max(1000, parseInt(process.env.SEAFILE_TIMEOUT_MS, 10) || 30000); // (the tests shorten this) const ONLYOFFICE_URL = (process.env.ONLYOFFICE_URL || 'https://onlyoffice.necloud.us').replace(/\/$/, ''); const AUDIOBOOKS_URL = (process.env.AUDIOBOOKS_URL || 'https://audiobooks.necloud.us').replace(/\/$/, ''); const AUDIOBOOKS_SSO_URL = process.env.AUDIOBOOKS_SSO_URL || AUDIOBOOKS_URL + '/sso'; @@ -46,43 +47,60 @@ const TYPES = { '.svg': 'image/svg+xml', '.png': 'image/png', '.ico': 'image/x-icon', }; +// Seafile calls get 30 s to start answering, file transfers (seafhttp) 2 minutes; a failed read is retried twice; and when the browser goes away +// (tab closed, download cancelled) the upstream request is stopped instead of being left running. async function proxySeafile(req, res) { const tu = new URL(SEAFILE_URL + req.url.slice('/sf'.length)); const inline = tu.searchParams.has('hubinline'); // preview: show in the page instead of forcing a download tu.searchParams.delete('hubinline'); const target = tu.toString(); const headers = {}; - for (const h of ['authorization', 'content-type', 'content-length', 'accept', 'range']) { + for (const h of ['authorization', 'content-type', 'content-length', 'accept', 'range', 'if-range']) { if (req.headers[h]) headers[h] = req.headers[h]; } - const hasBody = !['GET', 'HEAD'].includes(req.method); - try { - const upstream = await fetch(target, { - method: req.method, headers, redirect: 'manual', - body: hasBody ? Readable.toWeb(req) : undefined, - duplex: hasBody ? 'half' : undefined, - }); - const out = {}; - for (const h of ['content-type', 'content-length', 'content-disposition', 'content-range', 'accept-ranges']) { - if (inline && h === 'content-disposition') continue; - if (h === 'content-length' && upstream.headers.get('content-encoding')) continue; // decompressed by fetch, so the length changed - const v = upstream.headers.get(h); - if (v) out[h] = v; + const hasBody = !['GET', 'HEAD'].includes(req.method), canRetry = !hasBody; + const wait = /^\/seafhttp\//.test(tu.pathname) ? 120000 : SEAFILE_TIMEOUT; + let gone = false, ac = null; res.on('close', () => { if (!res.writableFinished) { gone = true; if (ac) ac.abort(); } }); + let upstream, lastErr; + for (let attempt = 0; attempt < 3; attempt++) { + if (gone) return; + ac = new AbortController(); let timedOut = false; + const timer = setTimeout(() => { timedOut = true; ac.abort(); }, wait); + try { + upstream = await fetch(target, { method: req.method, headers, redirect: 'manual', signal: ac.signal, + body: hasBody ? Readable.toWeb(req) : undefined, duplex: hasBody ? 'half' : undefined }); + clearTimeout(timer); + if (canRetry && [502, 503, 504].includes(upstream.status) && attempt < 2) { try { await upstream.body.cancel(); } catch {} await new Promise(r => setTimeout(r, 300 * (attempt + 1) ** 2)); continue; } + break; + } catch (e) { + clearTimeout(timer); upstream = null; lastErr = timedOut ? Object.assign(new Error('Seafile took too long to answer'), { timeout: true }) : e; + if (gone) return; + if (canRetry && attempt < 2) { await new Promise(r => setTimeout(r, 300 * (attempt + 1) ** 2)); continue; } + break; } - // Rewrite redirects so they stay on the proxy and on the Seafile host. - const loc = upstream.headers.get('location'); - if (loc) { - const u = new URL(loc, SEAFILE_URL); - if (u.origin === new URL(SEAFILE_URL).origin) out.location = '/sf' + u.pathname + u.search; - } - res.writeHead(upstream.status, out); - if (upstream.body) Readable.fromWeb(upstream.body).pipe(res); else res.end(); - } catch (e) { - res.writeHead(502, { 'content-type': 'application/json' }); - const why = e.cause ? (e.cause.code || e.cause.message) : ''; - console.error('Seafile proxy:', target.split('?')[0], e.message, why); - res.end(JSON.stringify({ error: 'Seafile unreachable: ' + e.message + (why ? ' (' + why + ')' : '') })); } + if (!upstream) { + const e = lastErr, why = e && e.cause ? (e.cause.code || e.cause.message) : ''; + console.error('Seafile proxy:', target.split('?')[0], e && e.message, why); + if (res.headersSent) return res.destroy(); + res.writeHead(e && e.timeout ? 504 : 502, { 'content-type': 'application/json' }); + return res.end(JSON.stringify({ error: e && e.timeout ? 'Seafile took too long to answer. Try again in a moment.' : 'Seafile unreachable: ' + (e && e.message) + (why ? ' (' + why + ')' : '') })); + } + const out = {}; + for (const h of ['content-type', 'content-length', 'content-disposition', 'content-range', 'accept-ranges', 'etag', 'last-modified', 'retry-after']) { + if (inline && h === 'content-disposition') continue; + if (h === 'content-length' && upstream.headers.get('content-encoding')) continue; // decompressed by fetch, so the length changed + const v = upstream.headers.get(h); + if (v) out[h] = v; + } + // Rewrite redirects so they stay on the proxy and on the Seafile host. + const loc = upstream.headers.get('location'); + if (loc) { + const u = new URL(loc, SEAFILE_URL); + if (u.origin === new URL(SEAFILE_URL).origin) out.location = '/sf' + u.pathname + u.search; + } + res.writeHead(upstream.status, out); + if (upstream.body) Readable.fromWeb(upstream.body).on('error', () => res.destroy()).pipe(res); else res.end(); } function serveStatic(req, res) { diff --git a/tools/files-test.js b/tools/files-test.js new file mode 100644 index 0000000..cd5290f --- /dev/null +++ b/tools/files-test.js @@ -0,0 +1,67 @@ +#!/usr/bin/env node +// The Seafile pass-through (/sf/*): sign-in passes through, slow or flaky Seafile is handled, closed downloads stop the upstream, uploads stream. +// Uses a fake Seafile. Run: node tools/files-test.js +const { spawn } = require('child_process'); +const http = require('http'); +const fs = require('fs'), os = require('os'), path = require('path'), crypto = require('crypto'); +const root = path.join(__dirname, '..'); +let failed = 0; +const ok = (n, c, x) => { console.log((c ? 'PASS ' : 'FAIL ') + n + (c ? '' : ' -> ' + JSON.stringify(x))); if (!c) failed++; }; +const wait = ms => new Promise(r => setTimeout(r, ms)); +const SECRET = 'files-test-secret-'.repeat(3), UID = 'ufiles'; + +(async () => { + const seen = []; let flaky = 0, upstreamClosed = false, uploaded = 0; + const fake = http.createServer((rq, rs) => { + const u = new URL(rq.url, 'http://x'); seen.push(rq.method + ' ' + rq.url); + const json = (c, o) => { rs.writeHead(c, { 'content-type': 'application/json' }); rs.end(JSON.stringify(o)); }; + if (u.pathname === '/api2/repos/') return json(200, [{ id: 'r1', name: 'Docs' }]); + if (u.pathname === '/api2/flaky/') { if (++flaky <= 2) { rs.writeHead(503); return rs.end('busy'); } return json(200, { ok: true, after: flaky }); } + if (u.pathname === '/api2/always503/') { rs.writeHead(503, { 'retry-after': '5' }); return rs.end('down'); } + if (u.pathname === '/api2/hang/') return; // never answers + if (u.pathname === '/api2/slowstart/') return setTimeout(() => json(200, { late: true }), 200); + if (u.pathname === '/api2/echo-auth/') return json(200, { auth: rq.headers.authorization || null }); + if (u.pathname === '/seafhttp/files/big') { // a long download the browser may walk away from + rs.writeHead(200, { 'content-type': 'application/octet-stream' }); rs.on('close', () => { upstreamClosed = true; }); + const t = setInterval(() => rs.write(Buffer.alloc(16384)), 20); rs.on('close', () => clearInterval(t)); return; + } + if (u.pathname === '/seafhttp/files/small') { rs.writeHead(200, { 'content-type': 'text/plain', 'content-disposition': 'attachment; filename="a.txt"', etag: '"e1"' }); return rs.end('hello'); } + if (u.pathname === '/seafhttp/upload-api/tok' && rq.method === 'POST') { let n = 0; rq.on('data', d => n += d.length); rq.on('end', () => { uploaded = n; json(200, [{ name: 'x' }]); }); return; } + if (u.pathname === '/api2/redir/') { rs.writeHead(302, { location: 'http://' + rq.headers.host + '/api2/repos/' }); return rs.end(); } + rs.writeHead(404); rs.end(); + }); + await new Promise(r => fake.listen(0, '127.0.0.1', r)); + const fakeUrl = `http://127.0.0.1:${fake.address().port}`; + + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'nehub-files-')), port = 20000 + Math.floor(Math.random() * 20000), base = `http://localhost:${port}`; + fs.writeFileSync(path.join(dir, 'users.json'), JSON.stringify({ users: [{ id: UID, name: 'Admin', role: 'admin', tabs: [], handle: 'aGFuZGxl', epoch: 1, disabled: false, created: 1 }] })); + const exp = Date.now() + 864e5, cookie = `hub_session=${UID}.1.${exp}.${crypto.createHmac('sha256', SECRET).update(`s${UID}.1.${exp}`).digest('base64url')}`; + const child = spawn(process.execPath, ['server.js'], { cwd: root, env: { ...process.env, PORT: String(port), DATA_DIR: dir, COOKIE_SECRET: SECRET, SEAFILE_URL: fakeUrl, SEAFILE_TIMEOUT_MS: '1500' } }); + let log = ''; child.stdout.on('data', d => log += d); child.stderr.on('data', d => log += d); + for (let i = 0; i < 60; i++) { try { await fetch(base + '/login'); break; } catch { await wait(100); } } + const get = (p, o = {}) => fetch(base + p, { redirect: 'manual', ...o, headers: { cookie, ...(o.headers || {}) } }); + + let r = await get('/sf/api2/repos/', { headers: { authorization: 'Token T1' } }); ok('a Seafile call passes through', r.status === 200 && (await r.json())[0].name === 'Docs'); + r = await get('/sf/api2/echo-auth/', { headers: { authorization: 'Token T1' } }); ok('with the person\'s Seafile token, which is not changed', (await r.json()).auth === 'Token T1'); + r = await fetch(base + '/sf/api2/repos/'); ok('and only for someone signed in to the hub', r.status === 401, r.status); + let n0 = seen.length; r = await get('/sf/api2/flaky/'); const fj = await r.json(); + ok('a Seafile that answers 503 twice is retried and the third answer is used', r.status === 200 && fj.ok === true && seen.length - n0 === 3, [r.status, seen.length - n0]); + n0 = seen.length; r = await get('/sf/api2/always503/'); ok('one that stays down gives its 503 after three tries (not a crash)', r.status === 503 && seen.length - n0 === 3, [r.status, seen.length - n0]); + const t0 = Date.now(); r = await get('/sf/api2/hang/'); const hj = await r.json(); + ok('one that never answers gives a clear 504 after the time limit, instead of hanging', r.status === 504 && /too long/i.test(hj.error) && Date.now() - t0 < 12000, [r.status, hj, Date.now() - t0]); + r = await get('/sf/api2/slowstart/'); ok('a slow but working answer is still delivered', r.status === 200 && (await r.json()).late === true); + r = await get('/sf/api2/redir/'); ok('redirects stay on the hub address', r.status === 302 && r.headers.get('location') === '/sf/api2/repos/', r.headers.get('location')); + r = await get('/sf/seafhttp/files/small'); ok('a download comes through with its headers (name, tag)', r.status === 200 && (await r.text()) === 'hello' && /a\.txt/.test(r.headers.get('content-disposition')) && r.headers.get('etag') === '"e1"'); + r = await get('/sf/seafhttp/files/small?hubinline=1'); ok('a preview is shown in the page instead of downloaded', !r.headers.get('content-disposition')); + const ctl = new AbortController(); r = await get('/sf/seafhttp/files/big', { signal: ctl.signal }); const rd = r.body.getReader(); await rd.read(); ctl.abort(); try { await rd.cancel(); } catch {} + for (let i = 0; i < 30 && !upstreamClosed; i++) await wait(100); + ok('when the browser walks away from a big download, the hub stops reading from Seafile', upstreamClosed === true); + const body = Buffer.alloc(3 * 1024 * 1024, 7); r = await get('/sf/seafhttp/upload-api/tok', { method: 'POST', body, headers: { 'content-type': 'application/octet-stream' } }); + ok('an upload is streamed through (all bytes arrive)', r.status === 200 && uploaded === body.length, [r.status, uploaded]); + fake.close(); await wait(100); + r = await get('/sf/api2/repos/', { headers: { authorization: 'Token T1' } }); const dj = await r.json(); + ok('with Seafile switched off, the answer says so (502), and the hub keeps running', r.status === 502 && /unreachable/i.test(dj.error) && child.exitCode === null, [r.status, dj]); + + child.kill(); fs.rmSync(dir, { recursive: true, force: true }); + console.log(failed ? `\n${failed} check(s) failed` : '\nAll Files checks passed'); process.exit(failed ? 1 : 0); +})().catch(e => { console.error('TEST CRASHED:', e); process.exit(1); });