Synapse v2.1.14 — "degrade, don't die"¶

Patch release closing two defects found by reading the web-search client and the import scheduler during weekly maintenance, plus a dependency sweep of the frontend build chain. No schema migrations, no new endpoints, no API changes, no new configuration keys.

Like 2.1.8 through 2.1.13, neither defect came from a live failure report, and both are the same familiar shape: code that states an intent the code does not keep. One is a function whose own abstraction documents "never raise into the caller" and that raises. The other is a loop that paces itself on a clock which two of its three exits remember to advance and the third does not — the same root cause 2.1.9 fixed, through the one door it did not cover.

Fixed¶

A transport hiccup from SearXNG failed the whole deep-research run instead of returning zero hits¶

Every web-search backend in Synapse sits behind one abstraction, and that abstraction is explicit about what a backend owes its caller:

@abstractmethod
async def _search_one(self, query: str) -> list[SearchHit]:
    """
    Run ONE query → hits. Best-effort: on ANY failure return [] and log a WARNING —
    never raise into the caller (degrade to fewer/zero hits, same as SearXNG, I9).
    """

Every opt-in adapter under app/ops/web_search/ implements exactly that, with a blanket except Exception around the whole call. The default backend did not:

try:
    async with httpx.AsyncClient(timeout=15.0) as client:
        response = await client.get(url, params=params)
except (httpx.TimeoutException, httpx.ConnectError, httpx.NetworkError) as exc:
    logger.warning("searxng_search: request failed for %r: %s — returning []", query, exc)
    return []

That reads like three failure types. It is two: ConnectError is already a subclass of NetworkError.

Root cause. A function whose contract is "never raise", written with an allow-list of the failures its author happened to think of instead of the deny-nothing guard the contract calls for.

What escaped, from the same httpx tree:

Exception	Parent	Realistic trigger
`RemoteProtocolError`	`ProtocolError` → `TransportError`	the server disconnects mid-response — the ordinary failure behind a Cloudflare tunnel, or a SearXNG container that restarts under load
`UnsupportedProtocol`	`TransportError`	an operator saves a scheme-less `searxng.local:8080` through `PUT /web-search/config`; the value is stored and used verbatim
`ProxyError`	`TransportError`	an outbound proxy refuses the tunnel
`DecodingError`	`RequestError`	a mis-declared `Content-Encoding` on the response
`InvalidURL`	`Exception`	a malformed base URL — not a `TransportError` at all

The same hole existed one layer further in, on the shape of the decoded body. That body is a remote service's JSON, not a validated schema: a top-level array made data.get(...) an AttributeError, a non-list results made raw_results[:max_results] a TypeError, and a non-string url or title made the SearchHit(...) construction a pydantic ValidationError.

The blast radius is not one query. searxng_search_many fans queries out through asyncio.gather:

results_nested = await asyncio.gather(*[_bounded_search(q) for q in queries])

A bare gather propagates the first exception and abandons its siblings, so one bad query takes the other two down with it. From there the exception climbs to run_deep_research's terminal handler, which does exactly what it is supposed to do with an unhandled error — marks the whole run error and stops:

except Exception as exc:  # noqa: BLE001
    status = "error"
    error_message = str(exc)

So a single mid-response disconnect ends a multi-iteration research run, discarding the sources already collected and skipping synthesis entirely. The identical failure through any opt-in backend returns [], and the run simply continues with fewer sources.

This never showed up as a 500, which is why it went unnoticed: the other caller, chat web-context, already wraps build_web_context in a broad except (app/chat/stream.py) and carries on without web results. The symptom was only ever "deep research sometimes dies".

Fixed by honouring the documented contract. Any transport, decode or payload-shape failure now logs a WARNING and returns []. asyncio.CancelledError is a BaseException in Python 3.11+, so except Exception still lets cancellation through untouched — the streaming and shutdown paths are unaffected.

Nine regression tests: one per escaping httpx type, one per malformed payload shape, and one that fails a single query inside the gather and asserts the other queries' hits still come back. All nine fail against the previous code. A tenth pins the happy path, so the hardening cannot quietly change parsing.

A manual import scan spun the scheduler loop at zero delay for as long as it ran¶

2.1.9 closed a zero-delay retry storm in ImportScheduler._run(). The loop computed its sleep from _last_run_at, which only advances on success, so a scan that raised left the reference frozen, max(0.0, full_interval - elapsed) clamped to 0.0 on every subsequent iteration, and the loop re-fired with no delay at all. The fix added a second clock — _last_attempt_at, stamped in the finally on every exit — and paced on the later of the two.

That covered two of the tick's three exits. Here is the third:

# Single in-flight guard (I7 — never overlap)
if self._scan_in_flight:
    logger.debug("ImportScheduler: scan already in-flight — skipping tick")
    continue

It stamps nothing. continue returns to the top of the loop, which recomputes the sleep from the same already-expired reference, clamps to 0.0 again, and comes straight back — two load_schedule() database round-trips per iteration, with only an asyncio.sleep(0) between them.

Root cause. The same one as 2.1.9: a pacing clock advanced by some exits of a tick and not others, so any exit that forgets it re-enters an interval that has already expired.

It is reachable with no adversary. The only producer of a concurrent scan is run_now() (POST /import-schedule/run-now), and a manual scan of a large import folder runs for minutes — it hashes every file it considers. The spin needs the scheduled tick to come due while that scan is in flight, which is precisely when an operator reaches for Run now: because the import is overdue.

Fixed with two stamps.

The skip path now advances the attempt clock. That is not only a brake, it is the correct semantics: the in-flight scan is this interval's scan, so the next tick belongs a full interval later.

run_now() advances it too, in its own finally, exactly as _run()'s finally already did. It was the one scan producer entirely invisible to the clock, so a tick that computed its sleep right after a manual scan still measured its interval from a reference that scan had already satisfied.

_last_attempt_at remains deliberately unpersisted, as 2.1.9 established: a manual or failed attempt must not look like a completed run across a restart. The R13-4/T4 catch-up behaviour on boot is unchanged, and a skipped tick still never touches _last_run_at — it is not a completed run.

Regression test T-SP-011 puts a scan in flight on an overdue schedule and asserts the sleeps stay at the full interval — without the fix it records [0.0, 0.0, 0.0, 0.0] — and that the loop never started a scan of its own. T-SP-011b/c pin the run_now() stamp on both the success and the raising path.

Security¶

Frontend build chain: 7 advisories closed¶

npm audit fix (without --force): transitive patch/minor bumps only, package.json untouched, 13 advisories down to 6.

Package	From → to	Advisory
`postcss`	8.5.16 → 8.5.28	two `sourceMappingURL` path-traversal advisories — arbitrary `.map` file disclosure
`js-yaml`	4.3.0 → 4.3.2	quadratic CPU consumption in `!!omap` resolution
`nanoid`	3.3.15 → 3.3.19	non-secure generators loop indefinitely on zero/negative size
`brace-expansion`	four pinned copies	ReDoS via consecutive non-expanding `{}` groups
`fast-uri`	3.1.3 → 3.1.8	host confusion via a backslash authority delimiter
`browserslist`	4.28.4 → 4.29.0	unbounded cache growth leading to OOM

Plus the caniuse-lite / electron-to-chromium / node-releases / update-browserslist-db data packages those pull in.

All of these are dev/build-chain only — npm audit --omit=dev reported 0 both before and after, so nothing here ships in the production bundle. They are worth taking anyway: postcss runs over project CSS at build time and the two path-traversal advisories are against exactly that, and the rest are a developer-machine surface.

The 6 remaining advisories all require a major bump of vite or vitest, which is out of scope for a patch release and is not attempted here.

Recorded, not fixed¶

Three findings from this pass were too large for a patch and went into docs/reference/ROADMAP-3.0-IDEAS.md instead:

pypdf is pinned to 5.9.0 by a <6 constraint, and every fix version for its 80+ known advisories lives in the 6.x line. pypdf is the default and unconditional PDF extractor, so it parses untrusted input — uploads, clipped pages, and PDFs the deep-research loop downloads from web-search results. The advisories are overwhelmingly denial-of-service (infinite loops, unbounded recursion, memory exhaustion on malformed files), not code execution. The bump is major, and it changes text-extraction output, so it needs a before/after comparison on real PDFs rather than a lockfile edit.
The RAG context includes each page's YAML frontmatter. _load_passage() reads the raw file, where every other page reader in the codebase takes frontmatter.loads(raw).content. One line to change — but it alters the text sent to the provider on every chat turn, which is a functional change to an existing flow and therefore not a patch.
cascade_delete scopes only one of its three reference-matching methods to the vault. Latent while the process serves a single vault; a data-loss bug the day one database holds several.

Upgrade notes¶

Nothing to do. Pull the image (or rebuild) and restart:

docker compose pull && docker compose up -d

No schema migration. No SQLAlchemy model changed, so docs/er/schema.mmd is unchanged and no Alembic revision ships with this release.
No API change. docs/api/openapi.json differs only in the version string. Request and response shapes for /research/start, /web-search/config and /import-schedule/run-now are identical.
No new configuration. SEARXNG_URL, the web_search_provider key and the import-schedule frequency settings keep their meanings and defaults; nothing new to set in .env.
Two observable differences, both by design:
A deep-research run whose SearXNG queries hit a transport or protocol error now continues with fewer sources and logs a WARNING per failed query, where it previously ended with status error. If you have runs that died mysteriously mid-way, this is the likely cause, and it resolves on upgrade with no action. A run that genuinely collects nothing still skips synthesis, exactly as before.
After a manual Run now import scan, the next scheduled scan is paced a full interval from that scan rather than from the previous scheduled one. This is intended: a scan just happened.
Frontend rebuild only for the dependency bumps. package.json is unchanged, so a npm ci picks the new lockfile up automatically; no action if you deploy the published image or desktop build.
No client-side state to clear. Existing OAuth grants, refresh tokens, API tokens, conversations and vault content are untouched.