Skip to content

Releases: NpgsqlRest/NpgsqlRest

NpgsqlRest v3.16.0

20 May 09:50

Choose a tag to compare

Version 3.16.0 (2026-05-20)

Full Changelog

Minor release fixing a long-standing class of bugs in the JSON-to-parameter parsers for the timestamp, timestamptz, time, and timetz PostgreSQL types: the parsers were silently shifting incoming values by the host process's UTC offset. Bumped to minor (not patch) because the corrected behavior changes how naive ISO strings (no Z, no offset) are interpreted on non-UTC hosts — see Breaking change below. The shift was invisible on UTC hosts (the default for mcr.microsoft.com/dotnet/aspnet and almost every Linux container) and only surfaced once the same image ran somewhere with TZ set to anything else — a Windows dev box, a Kubernetes pod with TZ overridden, or a non-UTC CI runner — at which point stored values diverged from the JSON the caller sent by the host's offset.

Fix: datetime parsers are now host-TZ-independent

TryParseTimestamp, TryParseTimestampTz, TryParseTime, and TryParseTimeTz in NpgsqlRest/ParameterParsers.cs all relied on the parameter-less DateTime.TryParse(value) overload. That overload's default DateTimeStyles.None converts offset-bearing strings to the host's local TZ and tags the result Kind=Local. The two *Tz parsers then called DateTime.SpecifyKind(v, DateTimeKind.Utc) on the local-shifted value — but SpecifyKind only relabels the kind, it does not convert. The result was a host-local wall-clock value labelled UTC, written to Postgres with a silent shift.

The timestamp and time parsers used the same buggy parse and sent the local-shifted value directly to Npgsql, which transmits the wall-clock verbatim for a without time zone column — the same silent shift, same size as the host's offset.

All four parsers now use:

DateTime.TryParse(
    value,
    CultureInfo.InvariantCulture,
    DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal,
    out var v)
  • AssumeUniversal treats naive ISO strings (no Z, no offset) as UTC — the canonical JSON-over-HTTP convention.
  • AdjustToUniversal converts any Z-bearing or offset-bearing value to UTC.

The result is a DateTime with Kind=Utc carrying the true UTC instant regardless of the host's TZ. The *Tz parsers use it directly. The without time zone parsers strip the kind back to Unspecified so Npgsql sends the UTC clock-time as the naive wall-clock value, matching the column semantics.

Why this was hidden so long

Almost every production container runs TZ=UTC by default. On a UTC host, DateTime.TryParse(...)'s local-conversion is a no-op and the SpecifyKind(Utc) "lie" coincidentally matches reality. The bug only manifests once the same code is deployed where TZ is anything else. The first symptom is usually a downstream report along the lines of "we send 2026-05-20T06:00:00Z and Postgres stored 08:00" — which is exactly the host's UTC offset.

The existing MultiParamsTests2 / MultiParamsQueryStringTests2 test pairs already hinted at this — both used Should().Match(t => t == "12:06:59..." || t == "11:06:59...") style assertions with a comment that read "integration server seems to have a different datetime alltogether". That was the bug, papered over. After this fix both tests assert single deterministic values.

TryParseDate left alone

DateOnly.TryParse rejects Z- and offset-bearing strings outright (verified across UTC, America/Los_Angeles, Europe/Zagreb, Pacific/Auckland) — it does not silently shift, so the date parser was not affected by the host-TZ bug class. It was however a separate papercut: callers sending full ISO timestamps (e.g. "2026-05-20T03:00:00Z") to a date column got a flat parse failure. TryParseDate now falls back to a DateTime parse and extracts the date portion when DateOnly.TryParse rejects the input, honoring the same JsonTimestampsAreUtc semantic as the other datetime parsers (UTC date when the flag is true, host-local date when false).

Breaking change

JSON timestamps are now interpreted as UTC:

  • Z-suffixed and offset-bearing ISO strings are converted to UTC.
  • Naive ISO strings (no offset, no Z) are assumed UTC rather than interpreted as host-local time.

Callers who relied on the previous "JSON is host-local" behavior — usually by accident, because the host happened to be UTC — will see no change. Callers who sent Z strings expecting UTC were silently broken on non-UTC hosts and are now correct.

Opt-out: NpgsqlRestOptions.JsonTimestampsAreUtc

Users whose downstream code genuinely depends on the legacy "naive timestamps are host-local" behavior — and who cannot update those callers to send Z-suffixed values — can restore the pre-3.16.0 behavior by setting JsonTimestampsAreUtc to false:

  • Library: new NpgsqlRestOptions { JsonTimestampsAreUtc = false, ... }.
  • Client (appsettings.json): "NpgsqlRest": { "JsonTimestampsAreUtc": false } (default is true).

When false, the four parsers fall back to the bare DateTime.TryParse(value) overload — Z/offset strings get host-local-converted and tagged Kind=Local, naive strings get parsed as Kind=Unspecified, and the *Tz parsers re-apply SpecifyKind(Utc) on top. That reproduces the exact pre-3.16.0 code path. Note that this is not recommended for new deployments: it puts you back in the bug class the rest of this release fixes. The flag exists purely as a compatibility escape hatch.

Tests

New file NpgsqlRestTests/HostTimeZoneIndependenceTests.cs covers all four parsers via echo functions and json_build_object round-trips:

  • timestamptz with Z suffix, with numeric offset, and naive (assumed UTC)
  • timestamp with Z suffix (stored as naive UTC clock-time)
  • timetz with Z suffix (round-trips as UTC)
  • time with Z suffix (UTC clock-time extracted)

Each assertion is exact — no host-TZ-tolerant ORs. The fixture forces the database to UTC at creation (alter database … set timezone to 'UTC'), so the assertions stay deterministic across runners. To verify host-TZ independence at the parser layer, run the suite under a non-UTC TZ env var (TZ=America/Los_Angeles dotnet test, for example) — the tests must still pass.

The two existing MultiParams* tests had their loose Should().Match(...) assertions for timestamptz and timetz replaced with single-value Should().Be(...) assertions, now that the parsers produce deterministic output.

Files touched

  • NpgsqlRest/ParameterParsers.cs — four parsers switched to AssumeUniversal | AdjustToUniversal.
  • NpgsqlRestTests/HostTimeZoneIndependenceTests.cs — new, six tests covering the four type variants.
  • NpgsqlRestTests/ParamTests/MultiParamsTests2.cs — tightened timestamptz / timetz assertions.
  • NpgsqlRestTests/ParamTests/MultiParamsQueryStringTests2.cs — same.

No config changes, no API surface changes.

NpgsqlRest v3.15.2

13 May 08:21

Choose a tag to compare

Version 3.15.2 (2026-05-11)

Full Changelog

Patch release finishing the config-validator fix started in 3.15.1. That release made Auth:Schemes validate by Type rather than by name; this one applies the same treatment to the two sister sections (RateLimiterOptions:Policies, CacheOptions:Profiles) that share the same "name-keyed open dictionary" shape, plus a small consistency win for ValidationOptions:Rules. Library version unchanged — fixes live entirely in NpgsqlRestClient/ConfigDefaults.cs.

Fix: RateLimiterOptions:Policies validates by Type, not by name

A configuration like

{
  "Config": { "ValidateConfigKeys": "Error" },
  "RateLimiterOptions": {
    "Enabled": true,
    "Policies": {
      "login_throttle": {
        "Type": "FixedWindow",
        "PermitLimit": 10,
        "WindowSeconds": 60,
        "Partition": { "Sources": [{ "Type": "IpAddress" }] }
      }
    }
  }
}

failed startup with Unknown configuration key: RateLimiterOptions:Policies:login_throttle. The rate limiter itself registered login_throttle and rejected over-limit requests correctly — only the validator was wrong.

Root cause

FindUnknownConfigKeys walked the user's policy name (login_throttle) against the defaults schema, which contains illustrative example names (fixed, sliding, bucket, concurrency, per_user). Any other name was flagged unknown. With ValidateConfigKeys: "Error", that killed startup.

The 3.13.0 migration explicitly grouped RateLimiterOptions:Policies, CacheOptions:Profiles, and ValidationOptions:Rules as the same "object keyed by user-chosen name" shape, but only ValidationOptions:Rules was added to the validator's open-dictionary whitelist. The other two were missed.

What changed

FindUnknownConfigKeys now intercepts the descent at RateLimiterOptions:Policies:<name> and picks a per-Type schema:

  • FixedWindow: Type, Enabled, PermitLimit, WindowSeconds, QueueLimit, AutoReplenishment, Partition
  • SlidingWindow: Type, Enabled, PermitLimit, WindowSeconds, SegmentsPerWindow, QueueLimit, AutoReplenishment, Partition
  • TokenBucket: Type, Enabled, TokenLimit, TokensPerPeriod, ReplenishmentPeriodSeconds, QueueLimit, AutoReplenishment, Partition
  • Concurrency: Type, Enabled, PermitLimit, QueueLimit, OldestFirst, Partition

The shared Partition sub-schema (Sources: [{ Type, Name, Value }], BypassAuthenticated) is appended to every type. When Type is missing/invalid the validator skips that policy silently, matching the runtime behavior in BuildRateLimiter.

Behavior after the fix

  • Any custom policy name validates by its declared Type; example names continue to validate as before.
  • Typos inside a policy (e.g. PermitLimt) are caught — they were silently ignored when Policies was treated as an opaque dictionary.
  • Cross-type keys are caught: e.g. TokensPerPeriod placed on a FixedWindow policy is flagged, since it belongs to TokenBucket.

Fix: CacheOptions:Profiles validates by shape

Same root cause, same shape of fix. Custom profile names (session_cache, api_responses, etc.) failed validation when ValidateConfigKeys: "Error" was set, because the defaults contain example names (fast_memory, shared_redis, date_range_hybrid).

All cache profiles share the same key set regardless of backend type (Memory / Redis / Hybrid) — only the backend selection varies — so a single flat schema covers every profile:

Enabled, Type, Expiration, Parameters, When

Each When rule validates as { Parameter, Value, Then }. Typos inside a profile (e.g. Expirashun) are now caught.

Improvement: ValidationOptions:Rules now validates rule bodies

ValidationOptions:Rules was previously on the open-dictionary whitelist, so custom rule names (phone_number, etc.) passed validation — but typos inside a rule (e.g. Patrn instead of Pattern) also passed silently. All validation rules share the same flat key set regardless of Type (NotNull / NotEmpty / Required / Regex / MinLength / MaxLength):

Type, Pattern, MinLength, MaxLength, Message, StatusCode

ValidationOptions:Rules has been removed from the whitelist and is now validated against this flat schema. Custom rule names still pass; typos inside any rule now surface.

Tests

NpgsqlRestTests/ConfigTests/ConfigValidationTests.cs gained 15 new tests covering all three sections: custom name acceptance, example-name regression coverage, typo flagging, cross-type key detection (rate limiter), Partition sub-block validation, When-rule sub-block validation, and missing-Type skip behavior for rate-limiter policies.

Files touched

  • NpgsqlRestClient/ConfigDefaults.cs — three new intercepts in FindUnknownConfigKeys, three new schema helpers, ValidationOptions:Rules removed from IsOpenDictionarySection.
  • NpgsqlRestTests/ConfigTests/ConfigValidationTests.cs — 15 new tests.

No changes to runtime config reading, no breaking changes, no library version bump (the bug was in NpgsqlRestClient only).

NpgsqlRest v3.15.1

11 May 14:13

Choose a tag to compare

Version 3.15.1 (2026-05-11)

Full Changelog

Two bug fixes around config-key validation that together caused legitimate Auth:Schemes setups to fail startup under default settings. No new features; no library changes — both fixes live in NpgsqlRestClient (ConfigDefaults.cs, Program.cs).

Fix: named auth schemes are validated by Type, not by name

A configuration like

{
  "Auth": {
    "Schemes": {
      "short_session": {
        "Type": "Cookies",
        "CookieName": "my_app",
        "CookieHttpOnly": true
      }
    }
  }
}

produced startup errors:

[ERR] Unknown configuration key: Auth:Schemes:short_session:CookieName
[ERR] Unknown configuration key: Auth:Schemes:short_session:CookieHttpOnly

Both keys are documented per-type override keys for Cookies-type schemes (added in 3.15.0) and are read/applied normally at scheme registration time. The validator was flagging them because of a name collision with the docs-style example entries in Auth:Schemes defaults (short_session, api_token, admin_jwt).

Root cause

FindUnknownConfigKeys descended into the defaults schema by key name. When a user's scheme name matched one of the documented examples, the validator validated against that example's incomplete key set instead of treating the entry as an open-dictionary item. The same scheme renamed to anything not in the example set took the open-dict path and validated clean — so the bug surfaced only for users whose scheme names happened to match the documentation.

What changed

Validation under Auth:Schemes:<name> is now driven by the scheme's Type field, not its name. The validator reads Type from the actual config and selects one of three type-specific schemas:

  • Cookies: Type, Enabled, CookieValid, CookieName, CookiePath, CookieDomain, CookieMultiSessions, CookieHttpOnly, CookieSameSite, CookieSecure.
  • BearerToken: Type, Enabled, BearerTokenExpire, BearerTokenRefreshPath.
  • Jwt: Type, Enabled, JwtSecret, JwtIssuer, JwtAudience, JwtExpire, JwtRefreshExpire, JwtClockSkew, JwtValidateIssuer, JwtValidateAudience, JwtValidateLifetime, JwtValidateIssuerSigningKey, JwtRefreshPath.

When Type is missing or unrecognized, the validator skips that scheme silently — RegisterAuthSchemes already throws a clearer error at startup, so double-reporting buys nothing.

Behavior after the fix

  • Every named scheme — regardless of name — is validated against the same key set per its declared Type. Typos like CooieName are still caught for both example-named and custom-named schemes.
  • Cross-type keys (e.g. JwtSecret on a Cookies-type scheme) are now flagged where they previously slipped through under custom-named schemes via the open-dict shortcut.
  • Existing configurations using the docs-example names (short_session, api_token, admin_jwt) start cleanly with any combination of valid per-type override keys.

Fix: --config and --validate CLI commands now honor ValidateConfigKeys mode

The three call sites of ValidateConfigKeys() were inconsistent. Normal startup branched on the mode (only "Error" aborts; "Warning" logs and continues; "Ignore" skips entirely). The two CLI command paths did not — both treated any warning as a fatal validation failure regardless of mode.

For a user running npgsqlrest --validate with the default Config:ValidateConfigKeys: "Warning", this meant:

  • Exit code 1 on the first unknown key, even though the runtime would have started up normally with the same config.
  • --config (dump current configuration as JSONC) suppressed its JSON output and exited 1 instead, even when the only thing wrong was a typo that would have shown up as a warning at runtime.

What changed

Both CLI paths now read the validation mode and apply the same rule as normal startup:

  • Error mode: warnings are fatal. --config prints them in red on stderr and exits 1 without dumping JSON. --validate reports configValid: false.
  • Warning mode (default): warnings are surfaced (yellow on stderr for --config, included in --validate text/JSON output) but they don't fail the run. --config proceeds to dump the JSONC. --validate reports configValid: true.
  • Ignore mode: no warnings produced at all (unchanged — the validator short-circuits earlier).

--validate --json output gains a warningsAreFatal boolean derived from the mode, so machine consumers can decide for themselves what to do with the warnings array independent of how the binary chose to exit:

{
  "valid": true,
  "configValid": true,
  "validationMode": "Warning",
  "warningsAreFatal": false,
  "warnings": ["SomeUnknown:Key"],
  "connectionTest": "ok"
}

Behavior after the fix

  • npgsqlrest --validate against a config with a typo + ValidateConfigKeys: "Warning" exits 0; the typo is surfaced as a warning. Set ValidateConfigKeys: "Error" (or pass --Config:ValidateConfigKeys=Error) to keep the old fail-fast behavior.
  • npgsqlrest --config always emits the JSONC dump unless the mode is Error and an unknown key is present. Warnings still print to stderr so typos remain visible.
  • Normal startup is unchanged — it was already correct.

Tests

  • New unit tests in NpgsqlRestTests/ConfigTests/ConfigValidationTests.cs exercise FindUnknownConfigKeys directly: per-type validation for each example scheme name and custom names, cross-type rejection, typo detection, and missing/invalid Type handling.
  • CLI tests in NpgsqlRestTests/CliTests/CliCommandTests.cs cover the Ignore / Warning / Error matrix for both --config and --validate --json.

NpgsqlRest v3.15.0

11 May 10:07

Choose a tag to compare

Version 3.15.0 (2025-05-11)

Full Changelog

This release prepares NpgsqlRest to act as an external Web API service for a separate partner or frontend application. Three pieces land together:

  1. Auth fix — named cookie schemes registered under Auth:Schemes (introduced in 3.13.0) now actually authenticate incoming requests; previously they signed users in but no endpoint accepted the cookie.
  2. Cookie attributesCookieSameSite and CookieSecure config knobs on both the main and named cookie schemes, so cookie-based auth works across origins (SPA on a different domain).
  3. OpenAPI filtering — config-level filters (IncludeSchemas / ExcludeSchemas / NameSimilarTo / NameNotSimilarTo / RequiresAuthorizationOnly) plus a per-routine @openapi hide / @openapi tag <name> comment annotation, so a single host can serve a curated OpenAPI document to partners while keeping internal endpoints out of the spec.

Changes are concentrated in NpgsqlRestClient/Builder.cs, plugins/NpgsqlRest.OpenApi/, and NpgsqlRest/Defaults/CommentParsers/. No breaking changes; existing appsettings.json works as-is — every new key defaults to "no filter" or "ASP.NET default".

Fix: named cookie schemes now actually authenticate requests

Configurations using Auth:Schemes to register a named cookie scheme alongside the main CookieAuth issued the named-scheme cookie correctly on login but no endpoint would authenticate against it. A request bearing only the named-scheme cookie was treated as anonymous — framework endpoints like /api/passkey/add/options returned 401, and SQL endpoints annotated @authorize returned 401. The feature signed users in but the sign-in was functionally inert.

Root cause

ASP.NET's authentication middleware only runs against the default authenticate scheme. The 3.13.0 implementation:

  1. Counted only the three main auth types (cookie / bearer / jwt) when choosing the default scheme — named schemes were invisible to that calculation.
  2. Registered the policy scheme (the dispatcher that picks the right scheme per request) only when more than one of the three main types was enabled. A typical setup with cookies + a named cookie scheme skipped the dispatcher entirely.
  3. Even when the dispatcher ran, its ForwardDefaultSelector only distinguished Bearer-vs-cookie header type. For any cookie-bearing request it returned the main cookie scheme regardless of which cookie was actually present.

The result: named-scheme cookies hit the main scheme's cookie handler, which couldn't decrypt them (different data-protection purpose strings per scheme), so context.User came out anonymous.

What changed

In NpgsqlRestClient/Builder.cs:

  • Pre-scan Auth:Schemes for enabled Cookie-type entries before AddAuthentication runs, so the default-scheme decision can account for them.
  • Register a policy scheme whenever the system has either (a) multiple main auth types — the existing case, unchanged — or (b) the main cookie scheme plus one or more named cookie schemes. For (b), a synthetic policy-scheme name (NpgsqlRest_PolicyScheme) is used to avoid colliding with the main cookie scheme's own registration.
  • Cookie-aware dispatchForwardDefaultSelector now walks the registered cookie schemes in order (main first, then named in registration order) and returns the first scheme whose configured cookie name appears in the request. Falls back to the main cookie scheme for cookie-less requests so anonymous traffic behaves exactly as before. Bearer/JWT header dispatch is unchanged.
  • Cookie-name tracking — every cookie scheme registration (main and named) now records its effective HTTP cookie name on Builder.CookieSchemesInOrder. Schemes without an explicit CookieName are tracked under ASP.NET's per-scheme default (.AspNetCore.<schemeName>), so the lookup is well-defined for both explicit and defaulted cookie names.

Behavior after the fix

  • A request carrying only a named-scheme cookie authenticates under that scheme. context.User.Identity.IsAuthenticated is true, context.User.Identity.AuthenticationType matches the named scheme name.
  • /api/passkey/add/options, /api/passkey/add, bearer/JWT refresh paths, and any @authorize-annotated SQL endpoint accept named-scheme cookies the same way they accept main-scheme cookies. No endpoint changes were required.
  • @authorize <role> continues to gate by role claims — a named-scheme cookie whose principal lacks the required role is still rejected. Scheme membership is orthogonal to role membership.
  • Backward compatibility is bit-for-bit identical for single-scheme configurations (cookies only, no Auth:Schemes): no policy scheme is registered, no selector logic engages, and the default authenticate scheme remains the main cookie scheme's name.

Cookie-precedence order when both are present

When a request somehow carries both a main cookie and a named-scheme cookie (rare in practice — a user is signed in under at most one scheme by SignInAsync), the walk order is main first, then named schemes in registration order, and the first match wins. This is deterministic but not configurable; if you need scheme-specific endpoint binding regardless of which cookie is present, ASP.NET's [Authorize(AuthenticationSchemes = "...")] is the right primitive and is out of scope for this release.

Feature: CookieSameSite and CookieSecure config

ASP.NET defaults the auth cookie's SameSite attribute to Lax and the Secure policy to SameAsRequest. That works for "browser and API on the same origin" but silently breaks the cross-origin case — an SPA on app.example.com calling an API on api.example.com won't have its session cookie sent on cross-site requests at all under Lax, and a None cookie without Secure is dropped outright by modern browsers.

Two new config keys make this controllable without dropping to a custom host.

Root-level (main cookie scheme)

"Auth": {
  "CookieAuth": true,
  "CookieSameSite": "None",       // "Strict" | "Lax" | "None" | "Unspecified" | null
  "CookieSecure":   "Always"      // "SameAsRequest" | "Always" | "None" | null
}

Default for both is null, which leaves ASP.NET's per-handler default in place — so existing configs see no change.

Per-scheme override under Auth:Schemes

The same two keys are accepted inside any Auth:Schemes:<name> Cookies-type entry, with the same inheritance pattern as the existing cookie fields (CookiePath, CookieDomain, CookieMultiSessions, CookieHttpOnly): scheme-level value wins, else inherit the root Auth section's value, else fall through to ASP.NET's default.

"Auth": {
  "CookieAuth": true,
  "CookieSameSite": "None",
  "CookieSecure":   "Always",
  "Schemes": {
    // Long-lived "remember me" cookie inherits the cross-origin posture from root.
    "remember_me":   { "Type": "Cookies", "CookieValid": "30 days" },

    // Short-lived sensitive-flow cookie tightens to first-party only.
    "short_session": {
      "Type": "Cookies",
      "CookieValid": "1 hour",
      "CookieSameSite": "Strict",
      "CookieSecure":   "SameAsRequest"
    }
  }
}

Validation and warnings

  • Unknown values fail fast at startup with the offending config path included in the message — typos in security-relevant config shouldn't be silently ignored. Example: Invalid value 'Loose' for Auth:CookieSameSite. Expected one of: Unspecified, None, Lax, Strict.
  • Setting SameSite=None without Secure=Always logs a startup warning at Warning level: browsers drop SameSite=None cookies that lack the Secure attribute, and the symptom ("login succeeds but the next request is anonymous") is otherwise hard to diagnose, especially during local HTTP testing.
  • Existing appsettings.json files are unaffected — both keys default to null (use ASP.NET's default).

Cross-origin checklist for an external Web API setup

Combining the cookie attributes above with the already-existing CORS support, a typical "API used by a separate SPA" config looks like:

"Cors": {
  "Enabled": true,
  "AllowedOrigins": ["https://app.example.com"],   // not "*"
  "AllowedMethods": ["GET", "POST", "PUT", "DELETE", "OPTIONS"],
  "AllowedHeaders": ["*"],
  "AllowCredentials": true                          // required for cookie auth
},
"Auth": {
  "CookieAuth": true,
  "CookieSameSite": "None",                         // cross-site
  "CookieSecure":   "Always",                       // required when SameSite=None
  "CookieHttpOnly": true,
  "CookieDomain":   ".example.com"                  // optional — share across subdomains
}

For mobile or non-browser clients the bearer/JWT path remains the recommended route (no cookie attributes apply, no CORS preflight); these new knobs only matter when the client is a browser on a different origin.

Feature: OpenAPI filtering for partner-facing documents

The NpgsqlRest.OpenApi plugin previously documented every endpoint NpgsqlRest registered, with no way to suppress an endpoint or override its tag. That works for "internal API doc" but not for "API doc handed to a partner team" — where you typically want to expose only a curated subset (e.g. routines in a partner schema, only the authenticated surface, with a partner tag for nice Swagger UI grouping).

3.15.0 adds the missing controls: five config-level filters on OpenApiOptions plus a per-routine @openapi comment annotation. All work additively — every existing config keeps its current output, since defaults are "no filter".

Config-level filters

Key Type Default Behavior
IncludeSchemas string[] empty = no filter Allow-list of schema names. Only endpoints whose routine schema appears here are documented.
...
Read more

NpgsqlRest v3.14.0

09 May 13:22

Choose a tag to compare

Version 3.14.0 (2025-05-09)

Full Changelog

This release sharpens the philosophy of the standalone client: REST endpoints come from PostgreSQL routines (functions, procedures) and from explicit SQL files — not auto-generated CRUD on tables and views. It also makes real-time push (SSE) more honest about what your annotations mean, and gets more throughput out of array- and composite-heavy responses.

Removed: auto-CRUD endpoint generation from the standalone client

The NpgsqlRest.CrudSource plugin is no longer wired into the NpgsqlRestClient standalone executable. The plugin itself still ships as a NuGet package and remains fully functional for embedded use — projects that consume the NpgsqlRest library directly can continue to register a CrudSource instance against EndpointSources exactly as before. Only the NpgsqlRestClient JSON-driven path is affected.

Why: auto-generated CRUD endpoints over arbitrary tables and views have always been a different shape from the rest of the project. The core promise is "your PostgreSQL routines become REST endpoints" — explicit, version-controlled, comment-annotated procedures and functions where the developer chose what to expose. Auto-CRUD inverts that: every table becomes ten URL patterns by default (select / insert / update / delete plus the various on conflict and returning variants), with no per-endpoint guardrails unless you opt back into them. That's a different product, and it doesn't belong in the same configuration surface.

What this means in practice:

  • appsettings.json: the entire NpgsqlRest:CrudSource block has been removed from the default template. Any existing configuration with that block will fail key validation at startup (the same Config:ValidateConfigKeys check that catches typos) — remove the block to upgrade.
  • --config output: CrudSource no longer appears in the dumped configuration.
  • --version output: NpgsqlRest.CrudSource is no longer listed in either the human-readable or --json form, since the standalone client no longer references the assembly.
  • Library use: zero change. using NpgsqlRest.CrudSource; sources.Add(new CrudSource()) works exactly as it did in 3.13.0 and remains supported.

If you had "CrudSource": { "Enabled": true } in your config and depend on the generated endpoints, the migration is one of:

  1. Switch to function-based wrappers — the recommended path. Write the CRUD shape you actually need as PostgreSQL functions or procedures with HTTP annotations. You get the same endpoints with explicit per-endpoint auth, validation, caching, rate limiting, and comment-driven URL shapes.
  2. Embed the library in a custom host and register CrudSource programmatically. The plugin code still ships under plugins/NpgsqlRest.CrudSource/.

What's new

Two new SSE annotations: @sse_publish and @sse_subscribe

If you've ever had a manager-side procedure broadcast notifications to user-side subscribers, you've probably hit the awkwardness of the all-in-one @sse annotation: every emitter procedure ended up with a phantom /info URL nobody connected to, and your generated TypeScript client included createXEventSource() helpers for procedures that should never be subscribed to.

3.14.0 splits the responsibilities:

Annotation What it does
@sse_publish This procedure's RAISE statements feed SSE subscribers. No subscribe URL is exposed.
@sse_subscribe Exposes a subscribe URL for EventSource clients. The procedure body is never executed when a client opens the stream.
@sse [path] Same as before — shorthand for both. Unchanged.

So a "manager broadcasts to users" flow now reads cleanly:

-- subscriber URL, body never runs on subscribe
comment on function subscribe_user_events() is '
HTTP GET
@sse_subscribe
';

-- emitter, no subscribe URL, no useless TS EventSource helper
comment on procedure broadcast_user_message(...) is '
HTTP POST
@sse_publish
@sse_scope authorize
';

The TypeScript client generator follows automatically: @sse_publish produces a plain POST function, @sse_subscribe keeps the EventSource helper, and @sse is unchanged.

All existing event filtering — @sse_scope, @sse_events_level, RAISE ... USING HINT, and the X-NpgsqlRest-ID execution-id header — works the same with both new annotations.

Warning when a RAISE looks like a missed @sse_publish

Forgetting @sse_publish on an emitter procedure used to fail silently: the RAISE ran, the notice logged, and zero events reached subscribers. NpgsqlRest now warns once per endpoint when a RAISE whose severity matches the configured SSE forwarding level fires in a procedure that has no @sse or @sse_publish:

WARN: RAISE INFO in endpoint /api/update-user-roles was not broadcast to SSE subscribers —
the endpoint has no @sse or @sse_publish annotation. Add @sse_publish to forward this
routine's notices, or set WarnUnboundServerSentEventsNotices=false to silence this warning.

The warning only fires when the project actually uses SSE somewhere and only when the severity matches — projects that use RAISE NOTICE for unrelated logging see no warnings, and projects that don't use SSE at all see no warnings. Configurable via the new WarnUnboundServerSentEventsNotices setting (default true).

Reliable SSE connection handshake

SSE responses now flush a small "connected" line as soon as the broadcaster has registered the new subscriber, instead of waiting for the first real event. Browsers and EventSource clients ignore comment-only lines per spec, so no consumer behavior changes — but a client that wants to do "subscribe, then publish, then receive" can now rely on the handshake completing before its publish call. Mostly visible to integration tests; in production it makes connection states more predictable.

Startup error when claim-mapped parameters use a non-text type

If Auth.UseUserParameters is on and your ParameterNameClaimsMapping references a procedure parameter whose SQL type isn't text-compatible — for example _company_id int mapped to a company_id claim — every authenticated request used to crash with this error from deep inside the driver:

System.InvalidCastException: Writing values of 'System.String' is not supported
for parameters having NpgsqlDbType 'Integer'.

The exception didn't mention claim mapping, so debugging meant a stack-trace hunt. NpgsqlRest now catches the misdeclaration at startup with a precise message naming the endpoint, parameter, claim, and the SQL type:

Endpoint POST /api/create-local-user parameter _company_id is mapped to claim
'company_id' but its type is 'int' which is not text-compatible. Claim values
are strings, so binding would fail at runtime with InvalidCastException.
Declare the parameter as text/varchar/char/json/jsonb/xml/jsonpath, or remove
'_company_id' from ParameterNameClaimsMapping.

Accepted types: text, varchar, char, name, xml, json, jsonb, jsonpath, plus unknown (the SQL-file-source case where the driver resolves the type server-side). Any other type fails fast at UseNpgsqlRest. The check only runs for endpoints with UseUserParameters enabled and only for parameters that match a configured claim mapping.

Warning when a request value is overridden by claim auto-bind

When a parameter is auto-bound from a claim, the claim wins — that's intentional, especially for security-sensitive procedures where the caller's identity must override anything the request supplies. But if the request also sent a value for that parameter, the value used to be discarded silently. With certain UI patterns (forms that POST every field) this hid real bugs: in one case update_user_roles(_user_id text, _roles text[]) looked like it was updating a target user, but every request modified the caller's own roles because _user_id was claim-mapped.

The auto-bind still wins (no behavior change for security), but a collision now emits a warning so the developer can see what happened:

Endpoint /api/update-user-roles parameter _user_id received a body value but
is auto-bound from claim 'name_identifier'. The supplied value is being ignored.

The warning fires only when the request actually supplied a value, naming the endpoint, parameter, source (body or query), and the claim.

Performance

A focused pass on response rendering. No public API or configuration changes.

Lower-allocation JSON conversion for arrays and composites

The four PostgreSQL → JSON conversion routines used to render array, composite, and tuple values now rent their working StringBuilder buffers from the existing pool instead of allocating fresh ones on every call. This affects PgArrayToJsonArray, PgCompositeArrayToJsonArray, PgTupleToJsonObject, and PgUnknownToJsonArray — all of which fire per row × per column on responses that include array or composite types.

Measured on a focused micro-benchmark (Apple M4 Pro, .NET 10, BenchmarkDotNet ShortRun, three iterations):

Function Before alloc After alloc Δ
PgArrayToJsonArray (numeric, 100 elem) 1.91 KB 1.16 KB −39%
PgArrayToJsonArray (text, 100 elem) 19.68 KB 13.86 KB −29%
PgCompositeArrayToJsonArray (50 elem) 23.02 KB 11.52 KB −50%
PgTupleToJsonObject (10 fields) 1.10 KB 1.27 KB within noise

CPU time per call moved by single-digit percent — within or near the noise band of a short BDN run. The headline win is reduced GC pressure during sustained load: a 100-row response containing several array columns can drop ~1–2 MB of allocation pe...

Read more

NpgsqlRest v3.13.0

01 May 11:44

Choose a tag to compare

Version 3.13.0 (2026-04-24)

Full Changelog

New: Auth Schemes (Named Additional Authentication Schemes)

Auth:Schemes is a named-dict section that registers additional ASP.NET Core authentication schemes alongside the main one. Each entry is a fully-fledged scheme of any of the three supported types — Cookies, BearerToken, or Jwt — with its own options. A login function selects which scheme to use by returning the scheme's name in its scheme column.

Use cases this unlocks:

  • Short-lived sensitive sessions for admin or payment flows (Cookies scheme with shorter CookieValid + CookieMultiSessions: false).
  • Per-scope JWT signing keys so a key leak has limited blast radius (separate JwtSecret per Jwt scheme).
  • Multiple bearer-token APIs with different expirations and refresh paths.
  • Single-session cookies for areas where parallel logins must be disallowed, alongside a normal long-lived session.
"Auth": {
  "CookieAuth": true,
  "CookieValid": "14 days",
  "JwtAuth": true,
  "JwtSecret": "...root-secret-32+chars...",
  "Schemes": {
    "short_session": {
      "Type": "Cookies",
      "Enabled": true,
      "CookieValid": "1 hour",
      "CookieMultiSessions": false
    },
    "api_token": {
      "Type": "BearerToken",
      "Enabled": true,
      "BearerTokenExpire": "30 minutes",
      "BearerTokenRefreshPath": "/api/api-token/refresh"
    },
    "admin_jwt": {
      "Type": "Jwt",
      "Enabled": true,
      "JwtSecret": "...separate-admin-secret-32+chars...",
      "JwtExpire": "5 minutes",
      "JwtRefreshPath": "/api/admin-jwt/refresh"
    }
  }
}
-- Standard login: returns 'Cookies' → 14-day persistent cookie
create function login(_user text, _pass text)
returns table (scheme text, name_identifier text, name text)
language sql security definer as $$
  select 'Cookies' as scheme, user_id::text, username from users where ...
$$;

-- Sensitive-area login: returns 'short_session' → 1-hour session-only cookie
create function admin_login(_user text, _pass text)
returns table (scheme text, name_identifier text, name text)
language sql security definer as $$
  select 'short_session' as scheme, user_id::text, username from users where ...
$$;

-- Admin JWT login: returns 'admin_jwt' → 5-minute JWT signed with the admin-only secret
create function admin_jwt_login(_user text, _pass text)
returns table (scheme text, name_identifier text, name text)
language sql security definer as $$
  select 'admin_jwt' as scheme, user_id::text, username from users where ...
$$;

Per-type override fields:

Type Override fields
Cookies CookieValid, CookieName, CookiePath, CookieDomain, CookieMultiSessions, CookieHttpOnly
BearerToken BearerTokenExpire, BearerTokenRefreshPath
Jwt JwtExpire, JwtRefreshExpire, JwtSecret, JwtIssuer, JwtAudience, JwtClockSkew, JwtRefreshPath, JwtValidateIssuer, JwtValidateAudience, JwtValidateLifetime, JwtValidateIssuerSigningKey

Common fields: Type (required, case-insensitive), Enabled (default true).

Inheritance. A scheme that overrides only one or two fields reuses everything else from the root Auth section, so blocks stay small. Setting CookieMultiSessions: false is the typical "single-session" override — the cookie's Max-Age becomes null (browser-session-only) while ExpireTimeSpan still bounds server-side validity. JWT schemes inherit JwtSecret from the root section if not set explicitly, so a per-scheme block can be just a shorter expiration.

Validation at startup (fail-fast).

  • Scheme name must not collide with the main scheme names (CookieAuthScheme, BearerTokenAuthScheme, JwtAuthScheme).
  • Type must be one of Cookies, BearerToken, Jwt (case-insensitive). Missing or unsupported types throw with a clear message.
  • Explicit CookieName values must be distinct across all cookie schemes. When unset, ASP.NET's per-scheme .AspNetCore.<scheme> default automatically differs and is excluded from collision tracking.
  • Refresh paths (BearerTokenRefreshPath / JwtRefreshPath) must be unique across the main scheme and every scheme that defines one — two app.Use middlewares listening on the same path would race.
  • Jwt schemes require a secret either on the scheme or on the root section; JwtSecret must be ≥32 chars for HS256.
  • Invalid interval strings (CookieValid, BearerTokenExpire, JwtExpire, JwtRefreshExpire, JwtClockSkew) throw with the offending path and value.

Refresh middleware per scheme. Each BearerToken/Jwt scheme that declares a refresh path gets its own app.Use middleware listening on that path, with that scheme's tokens validated under that scheme's options. The main-scheme refresh middleware (root BearerTokenRefreshPath / JwtRefreshPath) continues to work for the main scheme.

Logout. The existing logout pipeline accepts a list of scheme names from the logout function's result columns and signs out each — additional schemes work without changes. To clear both main and additional cookies in one logout, return both scheme names from the function.

Breaking: legacy auth time-integer fields removed

The four integer-based time fields under Auth are removed in 3.13.0. Use the interval-notation equivalents instead:

Removed (3.12 and earlier) Use instead (3.13.0+)
Auth:CookieValidDays: 14 Auth:CookieValid: "14 days"
Auth:BearerTokenExpireHours: 1 Auth:BearerTokenExpire: "1 hour"
Auth:JwtExpireMinutes: 60 Auth:JwtExpire: "60 minutes"
Auth:JwtRefreshExpireDays: 7 Auth:JwtRefreshExpire: "7 days"

The new fields accept Postgres-interval syntax ("14 days", "12 hours", "30 minutes", "45 seconds", etc.) — finer-grained durations than the legacy integers permitted.

If you upgrade with any of the four removed fields still in your config, startup will fail with a clear migration message naming the offending field, the replacement field, and an example interval string. Failing fast is deliberate — silently ignoring the legacy field would mean an "I configured 30-day cookies" intent silently flips to the new field's default of 14 days, which would be very surprising.

New: interval notation for auth time fields

Each of the four time-window settings under Auth (cookie validity, bearer-token expiration, JWT access-token expiration, JWT refresh-token expiration) is expressed as a Postgres interval string:

"Auth": {
  "CookieAuth": true,
  "CookieValid": "12 hours",
  "JwtAuth": true,
  "JwtExpire": "5 minutes",
  "JwtRefreshExpire": "1 day"
}

Setting any of these to null falls back to the framework default (14 days / 1 hour / 60 minutes / 7 days respectively). Invalid interval values fail at startup with a clear error, rather than silently falling back. The shipped appsettings.json includes the explicit defaults so users see exactly what they're getting.

Breaking: RateLimiterOptions:Policies is now a dict, not an array

This section was previously an array of objects, each with an explicit "Name" property. It is now an object keyed by policy name, matching the existing ValidationOptions:Rules and the new CacheOptions:Profiles shape. Migration is mechanical:

// Before (3.12 and earlier):
"Policies": [
  { "Name": "fixed",   "Type": "FixedWindow",  "Enabled": true,  "PermitLimit": 100, ... },
  { "Name": "sliding", "Type": "SlidingWindow", "Enabled": false, ... }
]

// After (3.13.0):
"Policies": {
  "fixed":   { "Type": "FixedWindow",  "Enabled": true,  "PermitLimit": 100, ... },
  "sliding": { "Type": "SlidingWindow", "Enabled": false, ... }
}

Move each policy's Name value to be the JSON key, then drop the Name field. No other field changes; runtime behavior is identical.

If you upgrade with the old array form still in your config, startup will fail with a clear InvalidOperationException telling you to migrate. (Failing fast is deliberate — silently registering policies under names like "0" and "1" would have made endpoint annotations referencing them stop matching, leading to silent loss of rate limiting.)

New: Per-User Rate Limiting (Partition on a policy)

Rate-limiter policies can now be partitioned at request time, so each request gets its own bucket based on a value derived from HttpContext (a claim, an IP, a header, or a static fallback). The classic use case is per-user throttling: each authenticated user gets their own quota instead of all users sharing one global bucket.

"RateLimiterOptions": {
  "Enabled": true,
  "Policies": {
    "per_user": {
      "Type": "FixedWindow",
      "Enabled": true,
      "PermitLimit": 100,
      "WindowSeconds": 60,
      "Partition": {
        "Sources": [
          { "Type": "Claim", "Name": "name_identifier" },
          { "Type": "IpAddress" },
          { "Type": "Static", "Value": "anonymous" }
        ]
      }
    },
    "throttle_anon_only": {
      "Type": "FixedWindow",
      "Enabled": true,
      "PermitLimit": 10,
      "WindowSeconds": 60,
      "Partition": {
        "BypassAuthenticated": true,
        "Sources": [{ "Type": "IpAddress" }]
      }
    }
  }
}

Partition fields:

  • Sources — ordered list. Walked top-to-bottom at request time; the first source returning a non-empty value wins. Each source has a Type:
    • Claim — reads HttpContext.User.FindFirst(Name).Value. Name is required (the claim type, e.g., "name_identifier").
    • IpAddress — reads the client IP via HttpRequest.GetClientIpAddress(), which honors X-Forwarded-For / X-Real-IP ahead of Connection.RemoteIpAddress. No Name needed.
    • Header — reads HttpContext.Request.Headers[Name]. Name is required.
    • Static — always returns the con...
Read more

NpgsqlRest v3.12.0

02 Apr 12:04

Choose a tag to compare

Full Changelog


New Endpoint Source Plugin: NpgsqlRest.SqlFileSource

In addition to the existing endpoint sources — RoutineSource (PostgreSQL functions and procedures) and CrudSource (tables and views) — NpgsqlRest now supports a third source: SQL files.

Generate REST API endpoints directly from .sql files. Place SQL files in a configured directory, and NpgsqlRest creates endpoints automatically — no PostgreSQL functions needed.

How It Works

  1. At startup, the plugin scans the directory matching the configured glob pattern (e.g., sql/**/*.sql)
  2. Each .sql file is parsed: comments are extracted as annotations, SQL is split into statements
  3. Each statement is analyzed via PostgreSQL's wire protocol (SchemaOnly) — parameter types and return columns are inferred without executing the query
  4. A REST endpoint is created for each file, with the URL path derived from the filename

Single-Command Files

A file with one SQL statement produces a standard endpoint:

-- sql/get_reports.sql
-- HTTP GET
-- @param $1 from_date
-- @param $2 to_date
-- @authorize admin
SELECT id, title, created_at
FROM reports
WHERE created_at BETWEEN $1 AND $2;

GET /api/get-reports?from_date=2024-01-01&to_date=2024-12-31[{"id": 1, "title": "Q1", "createdAt": "..."}]

HTTP verb auto-detection (when no explicit HTTP annotation):

SQL Statement HTTP Verb Rationale
SELECT / WITH ... SELECT GET Read-only
INSERT PUT Creation
UPDATE POST Modification
DELETE DELETE Removal
DO $$ ... $$ POST Anonymous script
Mixed mutations Most destructive wins DELETE > POST > PUT

An explicit HTTP GET, HTTP POST, etc. annotation always overrides auto-detection.

Note: DO blocks do not support $N parameters — this is a PostgreSQL language limitation. A DO block always produces a parameterless endpoint. In multi-command files, DO blocks work alongside parameterized statements — the other commands receive the shared parameters, the DO block receives none.

Multi-Command Files

A file with multiple statements (separated by ;) returns a JSON object. Each key corresponds to one command's result:

-- sql/process_order.sql
-- HTTP POST
-- @param $1 order_id
-- @result validate
SELECT count(*) FROM orders WHERE id = $1;
UPDATE orders SET status = 'processing' WHERE id = $1;
-- @result confirm
SELECT id, status FROM orders WHERE id = $1;

POST /api/process-order with {"order_id": 42}

{
  "validate": [1],
  "result2": 1,
  "confirm": [{"id": 42, "status": "processing"}]
}

Result set rules:

  • Commands returning rows → JSON array of row objects (same format as single-command endpoints)
  • Void commands (INSERT/UPDATE/DELETE without RETURNING) → rows-affected count as integer
  • Multi-command endpoints are never void — they always return a JSON object

Result naming:

  • Default keys: result1, result2, result3, ... (prefix configurable via ResultPrefix setting)
  • Override with the positional @result annotation placed before the statement it applies to, or inline after the semicolon:
    • @result validate — renames the next result to validate
    • @result is validate — same ("is" style)
    • Commands without @result keep their default name

Execution:

  • Uses NpgsqlBatch with one NpgsqlBatchCommand per statement — single database round-trip
  • All statements share the same parameters ($1, $2, etc.) — user sends each parameter once
  • Full retry logic via ExecuteBatchReaderWithRetryAsync with error code mapping and timeout handling
  • If any command fails, the entire request fails — no partial results

Parameters

SQL files use PostgreSQL positional parameters ($1, $2, ...). Parameters are passed via query string (GET) or JSON body (POST/PUT/DELETE):

GET /api/my-query?$1=hello&$2=42
POST /api/my-mutation {"$1": "hello", "$2": 42}

Use the @param annotation for better names:

-- @param $1 user_name
-- @param $2 age
SELECT * FROM users WHERE name = $1 AND age > $2;

Now: GET /api/my-query?user_name=hello&age=42

For multi-command files: Each statement is described individually. Parameter types are merged across all statements:

  • Same $N with same type across statements → use that type
  • Same $N with conflicting types → startup error with clear message (override with @param $1 name type)
  • $N used in only some statements → type from the statement(s) that reference it

Virtual Parameters

Use @define_param to create HTTP parameters that are NOT bound to the PostgreSQL command. These parameters exist for HTTP request matching, custom parameter placeholders, and claim mapping — without appearing in the SQL query.

Use case: custom parameter placeholders — pass HTTP parameters that control endpoint behavior (e.g., output format) without referencing them in SQL:

-- sql/users_report.sql
-- @define_param format text
-- table_format = {format}
-- @param $1 department_id
SELECT id, name, email FROM users WHERE department_id = $1;

GET /api/users-report?department_id=5&format=html_table — the format parameter feeds into the table_format custom parameter via {format} placeholder, selecting the output format (JSON, HTML table, etc.) without being part of the SQL query.

Use case: claim mapping — auto-fill from user claims without SQL reference:

-- @authorize
-- @user_parameters
-- @define_param _user_id
SELECT * FROM user_data;

Default type is text; specify a type with @define_param name type.

Comments and Annotations

All comments in the SQL file are parsed as annotations, just like COMMENT ON FUNCTION in PostgreSQL:

-- Line comments are annotations
/* Block comments are annotations too */
SELECT * FROM table;
-- Comments after statements also work

All existing NpgsqlRest annotations work: @authorize, @allow_anonymous, @tag, @sse, @request_param_type, @path, @timeout, @cached, @raw, @header, @separator, @login, @logout, @encrypt, @decrypt, etc.

New annotations for SQL files:

Annotation Description Example
@param $N name Rename positional parameter -- @param $1 user_id
@param $N name type Rename + retype parameter -- @param $1 user_id integer
@param $N is name Rename ("is" style) -- @param $1 is user_id
@result name Rename multi-command result key (positional) -- @result validate
@result is name Rename result key ("is" style, positional) -- @result is validate
@define_param name [type] Define virtual parameter (not bound to SQL) -- @define_param _user_id

CommentScope setting controls which comments are parsed:

  • All (default) — every comment in the file, regardless of position
  • Header — only comments before the first SQL statement

Wire Protocol Introspection

At startup, each statement is analyzed via PostgreSQL's Parse → Describe → Sync cycle (CommandBehavior.SchemaOnly):

  • Parameter types inferred from ParameterDescription message (authoritative OIDs)
  • Return columns inferred from RowDescription message (column names and types)
  • No query planning, no execution — roughly the cost of SELECT 1
  • Uses reader.GetName() / reader.GetDataTypeName() instead of GetColumnSchema() to avoid .NET type mapping failures for custom composite types
  • Unknown type OIDs (custom types returning "-.-") resolved via pg_catalog.pg_type query

Custom / Composite Type Support

Composite type columns in SQL file endpoints behave the same way as routine endpoints (functions and procedures) — flat by default, nested with the @nested annotation or NestedJsonForCompositeTypes setting. Arrays of composite types are also supported.

Unnamed and Duplicate Columns

SQL without column aliases:

SELECT $1, $2;

Produces valid JSON with unique fallback names instead of duplicate ?column? keys:

[{"column1": "hello", "column2": "world"}]

Use AS aliases for meaningful names: SELECT $1 AS name, $2 AS value.

URL Path Derivation

The endpoint path is derived from the filename (without .sql extension) using the same NameConverter as functions. For example, with the default camelCase converter:

  • get_reports.sql/api/get-reports
  • user_profile.sql/api/user-profile

Override with the @path annotation: -- @path /custom/path/{id}

Error Handling

Mode Behavior Use Case
ParseErrorMode.Exit (default) Logs error, exits process Fail-fast — catches SQL errors at startup
ParseErrorMode.Skip Logs error, skips file, continues Production — tolerate partial failures

All SQL file errors are logged at Error level. In Exit mode, a Critical log explains the exit and how to switch to Skip mode. PostgreSQL errors include compiler-like formatting with line:column position, source line excerpt, and a caret pointing at the error location:

SqlFileSource: /path/to/get-posts.sql:
error 42703: column u.id does not exist
  at line 3, column 12
  select u.id, u.name from users u
             ^

A warning is logged when the configured file pattern matches no files.

Errors caught at startup:

  • Parse errors (malformed SQL, unclosed strings/quotes)
  • Describe errors (PostgreSQL syntax errors, invalid table/column references)
  • Parameter type conflicts in multi-command files

Feature Parity

SQL file endpoints support all features available to function/procedure endpoints:

  • Composite type expansion (flat by default, nested with @nested annotation)
  • Response caching (cached, cache_expires_in)
  • Raw mode (raw, raw_value_separator, raw_new_line_separator, raw_column_names)
  • Binary mode
  • Encryption/decryption (encrypt...
Read more

NpgsqlRest v3.11.1

13 Mar 13:33

Choose a tag to compare

Full Changelog

TsClient: proxy Passthrough Endpoint Support

The TypeScript client generator (NpgsqlRest.TsClient) now recognizes proxy passthrough endpoints and generates functions that return the raw Response object, matching the existing proxy_out behavior. Previously, passthrough proxy endpoints (which typically use returns void) would generate Promise<void>, which was incorrect since the actual response comes from the upstream service.

Now, both proxy and proxy_out endpoints generate Promise<Response>:

// Generated for a proxy passthrough endpoint
export async function tsclientTestProxyPassthrough() : Promise<Response> {
    const response = await fetch(baseUrl + "/api/tsclient-test/proxy-passthrough", {
        method: "GET",
    });
    return response;
}

This allows callers to handle the upstream response appropriately (.json(), .blob(), .text(), etc.), just like proxy_out endpoints.

authorize Annotation Now Matches User ID and User Name Claims

The authorize comment annotation previously only matched against role claims (DefaultRoleClaimType). It now also matches against user ID (DefaultUserIdClaimType) and user name (DefaultNameClaimType) claims, aligning with the behavior that sse_scope authorize already had.

This means you can now restrict endpoint access to specific users, not just roles:

-- Authorize by role (existing behavior)
comment on function get_reports() is 'authorize admin';

-- Authorize by user name (new)
comment on function get_my_profile() is 'authorize john';

-- Authorize by user ID (new)
comment on function get_account() is 'authorize user123';

-- Mix of roles and user identifiers (new)
comment on function get_data() is 'authorize admin, user123, jane';

The SSE matching scope was also aligned to check all three claim types, making authorization behavior consistent across all features.


NpgsqlRest v3.11.0

10 Mar 13:52

Choose a tag to compare

Full Changelog

New Feature: proxy_out Annotation (Post-Execution Proxy)

A new proxy mode that reverses the existing proxy flow: execute the PostgreSQL function first, then forward the function's result body to an upstream service. The upstream response is returned to the client.

This enables a common pattern where business logic in PostgreSQL prepares a payload, and an external service performs processing that PostgreSQL cannot do — PDF rendering, image processing, ML inference, email sending, etc.

Syntax

@proxy_out [ METHOD ] [ host_url ]

Also aliased as forward_proxy (with or without @ prefix).

How It Works

Client Request → NpgsqlRest
  → Execute PostgreSQL function
  → Forward function result as request body to upstream service
  → Forward original query string to upstream URL
  → Return upstream response to client

Unlike the existing proxy annotation (which forwards the incoming request to upstream), proxy_out forwards the outgoing function result. The original request query string is forwarded to the upstream URL as-is. The client-facing HTTP method and the upstream HTTP method are independent — the client can send a GET while the upstream receives a POST.

Basic Usage

create function generate_report(report_id int)
returns json
language plpgsql as $$
begin
    return json_build_object(
        'title', 'Monthly Report',
        'data', (select json_agg(row_to_json(t)) from sales t where month = report_id)
    );
end;
$$;

comment on function generate_report(int) is 'HTTP GET
@proxy_out POST https://render-service.internal/render';

The client calls GET /api/generate-report/?reportId=3. The server:

  1. Executes generate_report(3) in PostgreSQL.
  2. Takes the returned JSON and POSTs it to https://render-service.internal/render/api/generate-report/?reportId=3 (original query string forwarded).
  3. Returns the upstream response (e.g., a rendered PDF) directly to the client with the upstream's content-type and status code.

Query String Forwarding

The original client query string is forwarded to the upstream service as-is. This allows the upstream to receive the same parameters that were used to invoke the function:

create function generate_report(p_format text, p_id int)
returns json
language plpgsql as $$
begin
    return json_build_object('id', p_id, 'data', 'report');
end;
$$;

comment on function generate_report(text, int) is 'HTTP GET
@proxy_out POST';

Calling GET /api/generate-report/?pFormat=pdf&pId=123 executes the function, then POSTs the result to the upstream with ?pFormat=pdf&pId=123 appended to the URL.

HTTP Method Override

Specify which HTTP method to use for the upstream request:

comment on function my_func() is 'HTTP GET
@proxy_out PUT';

The client sends GET, but the upstream receives PUT with the function's result as the body.

Custom Host

Override the default ProxyOptions.Host per-endpoint:

comment on function my_func() is 'HTTP GET
@proxy_out POST https://my-other-service.internal';

Error Handling

  • If the function fails (database error, exception), the error is returned directly to the client — the proxy call is never made.
  • If the upstream fails (5xx, timeout, connection error), the upstream's error status and body are forwarded to the client (502 for connection errors, 504 for timeouts).

Configuration

Uses the same ProxyOptions configuration as the existing proxy annotation. ProxyOptions.Enabled must be true:

{
  "NpgsqlRest": {
    "ProxyOptions": {
      "Enabled": true,
      "Host": "https://api.example.com",
      "DefaultTimeout": "30 seconds"
    }
  }
}

Performance

  • Zero overhead for non-proxy_out endpoints. The implementation adds only branch-not-taken boolean/null checks on the normal execution path (~4 nanoseconds).
  • Efficient byte forwarding. Function output is captured as raw bytes and forwarded directly via ByteArrayContent — no intermediate string allocation or double UTF-8 encoding.

TsClient: proxy_out Endpoint Support

The TypeScript client generator (NpgsqlRest.TsClient) now recognizes proxy_out endpoints and generates functions that return the raw Response object instead of a typed return value. Since the actual response comes from the upstream proxy service (not from the PostgreSQL function's return type), the generated function returns Promise<Response>, allowing the caller to handle the response appropriately (.json(), .blob(), .text(), etc.):

// Generated for a proxy_out endpoint
export async function generateReport() : Promise<Response> {
    const response = await fetch(baseUrl + "/api/generate-report", {
        method: "GET",
    });
    return response;
}

NpgsqlRest v3.10.0

25 Feb 14:03

Choose a tag to compare

Full Changelog

New Feature: Resolved Parameter Expressions

When using HTTP Client Types, sensitive values like API tokens or secrets are often needed in outgoing HTTP requests (e.g., in an Authorization header). Previously, these values had to be supplied as regular HTTP parameters — exposing them to the client and requiring an insecure round-trip: database → client → server → external API.

Resolved parameter expressions solve this by allowing function parameters to be resolved server-side via SQL expressions defined in comment annotations. The resolved values are used in HTTP Client Type placeholder substitution (headers, URL, body) and are also passed to the PostgreSQL function — but they never appear in or originate from the client HTTP request.

How It Works

If a comment annotation uses the existing key = value syntax and the key matches an actual function parameter name, the value is treated as a SQL expression to execute at runtime:

create type my_api_response as (body json, status_code int);
comment on type my_api_response is 'GET https://api.example.com/data
Authorization: Bearer {_token}';

create function get_secure_data(
    _user_id int,
    _req my_api_response,
    _token text default null
)
returns table (body json, status_code int)
language plpgsql as $$
begin
    return query select (_req).body, (_req).status_code;
end;
$$;
comment on function get_secure_data(int, my_api_response, text) is '
_token = select api_token from user_tokens where user_id = {_user_id}
';

The client calls GET /api/get-secure-data/?user_id=42. The server:

  1. Fills _user_id from the query string (value 42).
  2. Executes the resolved expression: select api_token from user_tokens where user_id = $1 (parameterized, with $1 = 42).
  3. Sets _token to the result (e.g., "secret-abc").
  4. Substitutes {_token} in the outgoing HTTP request header: Authorization: Bearer secret-abc.
  5. Makes the HTTP call and returns the response.

The token never leaves the server. The client never sees it.

Behavior

  • Server-side only: Resolved parameters cannot be overridden by client input. Even if the client sends &token=hacked, the DB-resolved value is used.
  • NULL handling: If the SQL expression returns no rows or NULL, the parameter is set to DBNull.Value (empty string in placeholder substitution).
  • Name-based placeholders, parameterized execution: Placeholders like {_user_id} reference other function parameters by name — the value is always looked up by name, regardless of position. Internally, placeholders are converted to positional $N parameters for safe execution (preventing SQL injection).
  • Sequential execution: When multiple parameters are resolved, expressions execute one-by-one on the same connection, in annotation order.
  • Works with user_params: Resolved expressions can reference parameters auto-filled from JWT claims via user_params, enabling fully zero-parameter authenticated calls.

Multiple Resolved Parameters

Multiple parameters can each have their own resolved expression:

comment on function my_func(text, my_type, text, text) is '
_token = select api_token from tokens where user_name = {_name}
_api_key = select ''static-key-'' || api_token from tokens where user_name = {_name}
';

Resolved Parameters in URL, Headers, and Body

Resolved values participate in all HTTP Client Type placeholder locations — URL path segments, headers, and request body templates:

-- URL: GET https://api.example.com/resource/{_secret_path}
-- Header: Authorization: Bearer {_token}
-- Body: {"token": "{_token}", "data": "{_payload}"}

New Feature: HTTP Client Type Retry Logic

When using HTTP Client Types, outgoing HTTP requests to external APIs can fail transiently — rate limiting (429), temporary server errors (503), network timeouts. Previously, a single failure was passed directly to the PostgreSQL function with no opportunity to retry.

The new @retry_delay directive adds configurable automatic retries with delays, defined in the HTTP type comment alongside existing directives like timeout.

Syntax

-- Retry on any failure (non-2xx status, timeout, or network error):
comment on type my_api_type is '@retry_delay 1s, 2s, 5s
GET https://api.example.com/data';

-- Retry only on specific HTTP status codes:
comment on type my_api_type is '@retry_delay 1s, 2s, 5s on 429, 503
GET https://api.example.com/data';

-- Combined with timeout:
comment on type my_api_type is 'timeout 10s
@retry_delay 1s, 2s, 5s on 429, 503
GET https://api.example.com/data';

The delay list defines both the number of retries and the delay before each retry. 1s, 2s, 5s means 3 retries with 1-second, 2-second, and 5-second delays respectively. Delay values use the same format as timeout100ms, 1s, 5m, 30, 00:00:01, etc.

Behavior

  • Without on filter: Retries on any non-success HTTP response, timeout, or network error.
  • With on filter: Retries only when the HTTP response status code matches one of the listed codes (e.g., 429, 503). Timeouts and network errors always trigger retry regardless of the filter, since they have no status code.
  • Retry exhaustion: If all retries fail, the last error (status code, error message) is passed to the PostgreSQL function — the same as if retries were not configured.
  • Unexpected exceptions: Non-HTTP errors (e.g., invalid URL) are never retried.
  • Parallel execution: Each HTTP type in a function retries independently within its own parallel task. No changes to the parallel execution model.
  • No external dependencies: Built-in retry loop, no Polly or other libraries required. Matches the existing PostgreSQL command retry pattern.

Example

create type rate_limited_api as (body json, status_code int, error_message text);
comment on type rate_limited_api is '@retry_delay 1s, 2s, 5s on 429, 503
GET https://api.example.com/data
Authorization: Bearer {_token}';

create function get_rate_limited_data(
    _token text,
    _req rate_limited_api
)
returns table (body json, status_code int, error_message text)
language plpgsql as $$
begin
    return query select (_req).body, (_req).status_code, (_req).error_message;
end;
$$;

If the external API returns 429 (rate limited), the request is automatically retried after 1s, then 2s, then 5s. If it returns 400 (bad request), no retry occurs and the error is returned immediately.

New Feature: Data Protection Encrypt/Decrypt Annotations

Two new comment annotations — encrypt and decrypt — enable transparent application-level column encryption using ASP.NET Data Protection. Parameter values are encrypted before being sent to PostgreSQL, and result column values are decrypted before being returned to the API client. The database stores ciphertext; the API consumer sees plaintext. No pgcrypto or client-side encryption required.

This is useful for storing PII (SSN, medical records, credit card numbers) or other sensitive data that must be encrypted at rest but is only ever looked up by an unencrypted key (e.g., user_id, patient_id).

Prerequisite: The DataProtection section must be enabled in appsettings.json (it is by default). The DefaultDataProtector is automatically created from Data Protection configuration and passed to the NpgsqlRest authentication options.

Encrypt Parameters

Mark specific parameters to encrypt before they are sent to PostgreSQL:

create function store_patient_ssn(_patient_id int, _ssn text)
returns void
language plpgsql as $$
begin
    insert into patients (id, ssn) values (_patient_id, _ssn)
    on conflict (id) do update set ssn = excluded.ssn;
end;
$$;
comment on function store_patient_ssn(int, text) is '
HTTP POST
encrypt _ssn
';

The client calls POST /api/store-patient-ssn/ with {"patientId": 1, "ssn": "123-45-6789"}. The server encrypts _ssn using Data Protection before executing the SQL — the database stores ciphertext like CfDJ8N..., never the plaintext SSN.

Use encrypt without arguments to encrypt all text parameters:

comment on function store_all_secrets(text, text) is '
HTTP POST
encrypt
';

Decrypt Result Columns

Mark specific result columns to decrypt before returning to the client:

create function get_patient(_patient_id int)
returns table(id int, ssn text, name text)
language plpgsql as $$
begin
    return query select p.id, p.ssn, p.name from patients p where p.id = _patient_id;
end;
$$;
comment on function get_patient(int) is '
decrypt ssn
';

The client calls GET /api/get-patient/?patientId=1. The ssn column is decrypted from ciphertext back to "123-45-6789" before being included in the JSON response. The id and name columns are returned as-is.

Use decrypt without arguments to decrypt all result columns:

comment on function get_all_secrets(text) is '
decrypt
';

Decrypt also works on scalar (single-value) return types:

create function get_secret(_id int) returns text ...
comment on function get_secret(int) is 'decrypt';

Full Roundtrip Example

-- Store with encryption
create function store_secret(_key text, _value text) returns void ...
comment on function store_secret(text, text) is '
HTTP POST
encrypt _value
';

-- Retrieve with decryption
create function get_secret(_key text) returns table(key text, value text) ...
comment on function get_secret(text) is '
decrypt value
';
POST /api/store-secret/  {"key": "api-key", "value": "sk-abc123"}
GET  /api/get-secret/?key=api-key  →  {"key": "api-key", "value": "sk-abc123"}
``...
Read more