Synapse v2.1.17 — "a secret in a URL, and a syscall that only offers"

Patch release closing a credential written to the log in plaintext, a page write that can publish a truncated file without raising, and an inverted sort sentinel. No schema migrations, no new endpoints, no API changes, no new configuration keys.

Like 2.1.8 through 2.1.16, none came from a live failure report — all three were found by reading code during weekly maintenance. Two of them are this project's recurring shape, one sibling of N missing the guard the others have, and this week both odd ones out are odd for a structural reason rather than by oversight:

SerpApi is the only web-search backend whose credential is in the request URL, because its API accepts no auth header. vault_io.atomic_write_bytes is the only vault write that calls the write syscall directly, because it is the one handed the whole payload up front — the streaming siblings wrap the descriptor in a buffered writer and got short-write handling from the standard library without asking for it.

That is worth saying plainly: neither site is careless. Each is the one place where the cheap, obvious idiom was unavailable, and in both cases what filled the gap was an idiom that looks complete and is not.

Fixed

A page write could publish a truncated file over a good one — atomically, and in silence

os.write() is a thin wrapper over write(2), and write(2) is permitted to accept fewer bytes than it was offered and report how many it took. So a single unchecked call is not "write this payload", it is offer this payload:

tmp_fd, tmp_name = tempfile.mkstemp(dir=str(dst.parent), suffix=suffix)
os.write(tmp_fd, data)        # ← returns a COUNT. Nobody read it.
os.close(tmp_fd)
publish_tmp_file(tmp_name, dst)   # ← renames whatever did get written onto the live page

On a regular file the short write is not hypothetical: it is what the kernel does when the filesystem fills mid-write. It writes into the blocks it could allocate and returns the partial count instead of raising ENOSPC. The bytes it did not take are gone, nothing in the process notices, and the very next step is the rename that publishes the temp file. The result is a page whose body has been cut off, committed over the intact one, atomically, with no exception raised and nothing in any log.

The blast radius is exactly what 2.1.16 created. That release consolidated all six vault write sites into this one function — deliberately, and the consolidation was right — which means this defect covers every page body the system writes:

• every page the ingest loop generates (ingest/block_writer),
• the in-place rewrite behind wikilink enrichment (ingest/orchestrator.reindex_wiki_page_body),
• PUT /pages/{page_id}/content — an ordinary page edit from the UI,
• POST /ingest/upload, POST /ingest/from-text and POST /clip.

It needs no adversary, only a full disk — an ordinary event on a self-hosted TrueNAS box (CLAUDE.md §1) — and its symptom, a page that lost its body, points at anything other than a return value nobody read.

The streaming siblings were never affected, and they are where the intended shape already lived: routers/ingest.py hands the descriptor to open(fd, "wb") and ops/deep_research.py to os.fdopen(fd, "wb"), and a buffered writer loops internally and raises on ENOSPC. This one site called the raw syscall once because it had the whole payload in hand and nothing about that looked like a loop.

Root cause: a syscall's return value treated as a status rather than as a count.

_write_all now loops until the kernel has accepted every byte, and raises OSError if the descriptor stops accepting data — a 0-byte return would otherwise spin there forever — so the function's existing cleanup removes the temp file and the destination keeps its previous contents. The close-exactly-once property added in 2.1.16 is unchanged; the loop sits inside it, not around it.

A date-sorted citation list put the undated entries first, having promised to put them last

_sort_citations_by_date (app/rag/retrieval.py) re-orders the assembled [n] citations by pages.updated_at, fetched for all cited ids in one bounded read. The comment above the sort states the contract: "pages without an updated_at entry stay at the end in their original order". The fallback key for a page the read did not return was:

_SORT_MISSING = "" if not reverse else "\xff\xff"

Both branches are the wrong way round, because the sentinel has to sit on the far side of the direction being sorted. Ascending puts the largest key last, so a miss needed the high sentinel and got the empty string. Descending puts the smallest key last, so it needed the empty string and got the high one. In both directions the undated citation led the list.

Latent today. Phase 4 builds the citation list from the same pages table the sort then reads, so a miss requires a row to vanish between two reads inside a single request. Fixed rather than filed because the correction is two characters and the comment above it already specified the right answer — a tie-break that is inverted is worth less than one that admits it.

Root cause: a sentinel derived from the sort flag instead of from the position it has to land in.

Security

The SerpApi API key was written to the log in plaintext on every failed search

app/ops/web_search/keys.py opens with the invariant: "Plaintext is NEVER logged or returned by any endpoint — only a masked posture is exposed." Of the five opt-in web-search backends, SerpApi is the one that broke it, and the reason is structural. Its API accepts no authentication header, so the key has to travel as a query parameter:

params = {"engine": "google", "q": query, "api_key": api_key}   # ← part of the request URL
...
except Exception as exc:
    logger.warning("serpapi: search failed for %r: %s — returning []", query, exc)

httpx formats HTTPStatusError as Client error '401 Unauthorized' for url '<full url>' — query string included. So the best-effort except clause, written precisely to explain why the search degraded to zero hits, wrote the operator's credential into the log line:

WARNING serpapi: search failed for 'some query': Client error '401 Unauthorized' for url
'https://serpapi.com/search.json?engine=google&q=some+query&api_key=sk-…-SECRET' — returning []

The triggering responses are the ordinary ones, not an attack:

• 401 — a wrong, revoked or expired key. The most likely failure in this adapter's life.
• 429 — rate-limited. Which means a key that is perfectly valid leaks too.

The other four backends never had the problem, because none of them has the key in the URL: brave sends X-Subscription-Token, firecrawl an Authorization: Bearer header, tavily puts it in the JSON body, and ollama_web has no cloud key at all.

Honestly scoped: this backend is opt-in and off by default (ADR-0066/ADR-0070), the key is env-only and never exposed on the config-override surface, and a log file is not a public surface. What makes it worth a release rather than a note is the combination of where it goes — shipped to whoever can read container logs, which on this deployment includes anyone with access to the TrueNAS host — and when it fires: on the single most common failure the adapter has, with no error on the operator's side to suggest that anything was disclosed.

Root cause: a secret placed in a URL, and then a URL-bearing exception stringified into a log record.

Every str(exc) at that site now goes through _redact_key, which masks the raw key and its percent-encoded form. The second replacement is not decoration: httpx builds the URL from params=, so a key containing a URL-unsafe byte reaches the exception text percent-encoded, and a raw-only replacement would miss exactly the keys that are hardest to notice in a log.

Also in this release

One new entry in the 3.0 backlog (docs/reference/ROADMAP-3.0-IDEAS.md):

• katex has a production advisory, and its trust:false is what the DOMPurify bypass rests on. katex 0.16.47 is covered by GHSA-238p-pmpm-9mq7 ("existing prototype pollution can bypass trust restrictions", severity low); the fix is katex@0.19.0, a major, so it stays out of a patch. The reason it is worth scheduling rather than ignoring is where it lands: renderMarkdown.ts renders with trust: false and then re-inserts KaTeX HTML after DOMPurify, justified in a comment by "KaTeX output (trust:false) is XSS-safe by construction" — which is the proposition the advisory denies. The justification for the bypass and the content of the advisory are the same sentence, one asserted and one negated. Not exploitable on its own today: the attack needs a prototype pollution already present in the page, and there is none. It also retires a claim in the 2026-08-14 backlog entry — npm audit --omit=dev is no longer 0, and katex is the first advisory to reach the served bundle.

The long-standing pypdf entry is unchanged: pip-audit still reports every advisory against the pinned 5.9.0 as fixed only in the 6.x line, behind a <6 constraint. Still a major bump, still out of scope for a patch.

The remaining npm audit findings are dev/build-chain only (vitest, vite/esbuild, tinypool) and each needs a major bump, as already filed. source-map-js has a non-breaking fix available but is build-chain only and rode no code change this week, so it was left alone rather than bundled in for the sake of a tidier audit line.

Upgrade notes

docker compose pull && docker compose up -d

• No schema migration. No SQLAlchemy model changed, so docs/er/schema.mmd is unchanged and no Alembic revision ships with this release.
• No API change. docs/api/openapi.json differs only in the version string.
• No new configuration. Nothing new to set in .env; no existing key changes meaning.
• Rotate your SerpApi key if you have ever used that backend and keep your logs. This is the one action this release asks for, and only if WEB_SEARCH_PROVIDER=serpapi was ever selected and SERPAPI_API_KEY was set. Any failed search before this version wrote the key into the backend log, so the key should be treated as disclosed to everyone who can read those logs or any copy of them — shipped log aggregation, a backup of /var/lib/docker, a pasted bug report. Rotate at the SerpApi dashboard, update SERPAPI_API_KEY, restart. If you never enabled SerpApi — the default, since it is off by default — nothing was ever logged and there is nothing to do. The four other backends were never affected.
• Pages already written are not re-checked. The truncation fix applies to writes from here on; it does not scan the vault for pages that were cut short earlier. If the host has ever hit a full disk, the pages written in that window are the ones to look at — a truncated page is valid Markdown that simply stops, so nothing flags it. make lint / the Lint op will surface the resulting dangling wikilinks and missing frontmatter, which is the cheapest way to find them.
• No client-side state to clear. Existing OAuth grants, refresh tokens, API tokens, conversations and vault content are untouched.