Skip to content

fix(#11838): describe schemaless columns of a prepared statement from its rows, and bind a uuid parameter - #11920

Open
shaunpatterson wants to merge 2 commits into
ArcadeData:mainfrom
shaunpatterson:fix/11838-pg-prepared-column-types
Open

shaunpatterson wants to merge 2 commits into
ArcadeData:mainfrom
shaunpatterson:fix/11838-pg-prepared-column-types

Conversation

@shaunpatterson

Copy link
Copy Markdown
Contributor

Fixes #11838.

What was wrong

A Describe('P') of a portal bound from a named statement announced every property the schema does not declare as varchar (text[] for a list), the "stable layout" from #9009. Drivers name a statement on their own once it has run a few times (psycopg 3 from the 6th execution, pgjdbc from the 5th), so the same query returned True, then 't'. A uuid parameter (OID 2950) was refused at Bind: Type with code 2950 not supported for deserializing.

The change

  1. A portal is described from its own rows, named statement or not. resolvePortalColumns no longer swaps undeclared properties for text when the portal belongs to a named statement, so a Describe('P') answers the same OIDs as the unnamed statement and the simple query protocol (16, 23, 701, int[]/int8[], 1114). libpq (and so psycopg) sends Bind, Describe('P'), Execute on every execution of a prepared statement, so each execution is described from its own rows.
  2. A layout kept from those rows is checked, not trusted. pgjdbc describes the portal once and then sends Bind, Execute with no Describe, reusing that layout and asking binary transfer for the numeric columns. The layout is still recorded on the statement (rememberDescribedLayout), now flagged columnsFromRows. A later portal executed under it without its own Describe is materialized (no longer streamed under the layout) and its rows are compared with it: each column must have the type the rows alone would be described with, or be null on every row. Rows that fit are sent under the layout. Rows that do not fit are refused the way PostgreSQL refuses a prepared statement whose result type changed: SQLSTATE 0A000, message cached plan must not change result type, routine RevalidateCachedQuery. pgjdbc reads that routine as an error a new Parse heals (QueryExecutorBase.willHealViaReparse) and runs the statement again, described from its new rows. writeError gained an optional routine field for this.
  3. uuid parameters. OID 2950 is read in text (any spelling PostgreSQL accepts: upper case, braces, no hyphens) or binary (16 bytes) and bound as the canonical lower-case text, which is how ArcadeDB stores a uuid. An invalid uuid is refused at Bind.

Describe('S') is unchanged: it runs before any parameter is bound, so it keeps the schema/sample layout with undeclared properties as text, and that layout is still applied without a check (asyncpg, npgsql and pgx use this path).

Trade-off

pgjdbc retries a RevalidateCachedQuery error only when the transaction is not already failed: with the default autosave=never, inside an explicit transaction block the refusal reaches the application and aborts the block. So a pgjdbc application that, inside one transaction, runs the same prepared statement over rows whose schemaless property changes type between executions now gets this error where it used to get every value as text. That is what PostgreSQL itself does when a prepared statement's result type changes, and declaring the property in the schema avoids it. [INFERENCE from the pgjdbc 42.7.14 source, not run.]

Tests

Issue11838PreparedSchemalessTypesIT (new, 4 tests):

  • aNamedStatementDescribesEachPortalAsTheUnnamedOneDoes: the libpq exchange over a raw socket (Parse+Sync, then Bind/Describe('P')/Execute/Sync); for bool, int, double, list and timestamp values the named statement's OID and text equal the unnamed statement's, and the same named statement describes a bool portal as 16 and an int portal as 23.
  • aPortalExecutedUnderAnotherPortalsLayoutIsRefusedWhenItsRowsDoNotFit: the pgjdbc exchange; an undescribed portal whose rows fit the kept layout gets its DataRow, one whose rows do not gets 0A000 / RevalidateCachedQuery.
  • jdbcKeepsTheTypesOnceItPreparesTheStatement: pgjdbc with default settings, 8 executions per value: getObject returns the same class and value every time (Boolean, Integer, Double, int array, timestamp), then values of alternating types on one prepared statement, each read as its own type.
  • aUuidParameterBinds: pgjdbc setObject(UUID) over 6 executions, and a text uuid declared as OID 2950 in four spellings; an invalid uuid is refused.

Failing first: with postgresw/src/main from origin/main, all 4 fail (3 failures, 1 error: Type with code 2950 not supported for deserializing); with the fix all 4 pass.

Issue9008And9009MixedTypesIT.aPreparedStatementKeepsOneLayoutForRowsOfDifferentTypes (pgjdbc defaults, ids 1,1,1,1,1,2,3,1) is updated: it now also checks that u is described as int4, int8 and float8 for the three records, and it reads a as absent or null for the records that do not have it. Before, every execution from the 6th reused the text layout of record 1, so a was a null column; now the 6th and 7th executions are prepared again and described from records 2 and 3, which have no a, exactly as the unprepared statement describes them. The prepareThreshold=-1 variant (Describe('S')) is unchanged.

Module suite (-pl postgresw, every test under com/arcadedb/postgres): 1615 run, 0 failures, 1 error, 0 skipped. The error was the 9009 assertion above, before it was updated; after the update Issue9008And9009MixedTypesIT passes 5/5 and Issue11838PreparedSchemalessTypesIT 4/4.

End to end (container arcadedata/arcadedb:26.12.1-SNAPSHOT with the Postgres plugin, main jars a7304ab2aa, psycopg 3)

The issue's script, main postgresw jar:

bool   unprepared: type oid [16] -> [(True,)]                                 prepared: type oid [1043] -> [('t',)]
int    unprepared: type oid [23] -> [(42,)]                                   prepared: type oid [1043] -> [('42',)]
double unprepared: type oid [701] -> [(2.5,)]                                  prepared: type oid [1043] -> [('2.5',)]
list   unprepared: type oid [1016] -> [([1, 2, 3],)]                            prepared: type oid [1009] -> [(['1', '2', '3'],)]
date   unprepared: type oid [1114] -> [(datetime.datetime(2026, 1, 2, 0, 0),)]  prepared: type oid [1043] -> [('2026-01-02 00:00:00.000000',)]

With the fix:

bool   unprepared: type oid [16] -> [(True,)]                                 prepared: type oid [16] -> [(True,)]
int    unprepared: type oid [23] -> [(42,)]                                   prepared: type oid [23] -> [(42,)]
double unprepared: type oid [701] -> [(2.5,)]                                  prepared: type oid [701] -> [(2.5,)]
list   unprepared: type oid [1016] -> [([1, 2, 3],)]                            prepared: type oid [1016] -> [([1, 2, 3],)]
date   unprepared: type oid [1114] -> [(datetime.datetime(2026, 1, 2, 0, 0),)]  prepared: type oid [1114] -> [(datetime.datetime(2026, 1, 2, 0, 0),)]

Default psycopg auto-prepare, 8 executions of one query over bool/int/double rows, plus a uuid.UUID parameter, main:

exec 1 bool   oid   16 -> True
exec 1 int    oid   23 -> 42
exec 1 double oid  701 -> 2.5
exec 5 bool   oid 1043 -> 't'
exec 5 int    oid 1043 -> '42'
exec 5 double oid 1043 -> '2.5'
exec 6 bool   oid 1043 -> 't'
exec 6 int    oid 1043 -> '42'
exec 6 double oid 1043 -> '2.5'
exec 8 bool   oid 1043 -> 't'
exec 8 int    oid 1043 -> '42'
exec 8 double oid 1043 -> '2.5'
uuid param: InternalError_ Error on parsing bind message: Type with code 2950 not supported for deserializing

With the fix:

exec 1 bool   oid   16 -> True
exec 1 int    oid   23 -> 42
exec 1 double oid  701 -> 2.5
exec 5 bool   oid   16 -> True
exec 5 int    oid   23 -> 42
exec 5 double oid  701 -> 2.5
exec 6 bool   oid   16 -> True
exec 6 int    oid   23 -> 42
exec 6 double oid  701 -> 2.5
exec 8 bool   oid   16 -> True
exec 8 int    oid   23 -> 42
exec 8 double oid  701 -> 2.5
uuid param (7 executions): [('u',)]

@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 16 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8f40d66d-a99d-472e-9c3f-95260c9f8a2e
Commits

Reviewing files that changed from the base of the PR and between 8da9049 and f2cb55c.

Files selected for processing: 5

  • postgresw/src/main/java/com/arcadedb/postgres/PostgresNetworkExecutor.java
  • postgresw/src/main/java/com/arcadedb/postgres/PostgresPortal.java
  • postgresw/src/main/java/com/arcadedb/postgres/PostgresType.java
  • postgresw/src/test/java/com/arcadedb/postgres/Issue11838PreparedSchemalessTypesIT.java
  • postgresw/src/test/java/com/arcadedb/postgres/PostgresTypeEncodingIT.java
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@mergify

mergify Bot commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue). If it does not match the queue conditions yet, it enters the queue once it does.

  • Queue this pull request

@codacy-production

codacy-production Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 8 complexity

Metric Results
Complexity 8

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@github-actions

Copy link
Copy Markdown
Contributor

Triage: triage:lgtm · severity:minor · areas: postgresql-wire · has-issue

Summary: The PR fixes #11838 in three parts, all in postgresw:

  • A Describe('P') of a named statement is now typed from the portal's own rows, the same as the unnamed statement.
  • A layout kept from those rows is checked before reuse. A portal executed under it without its own Describe is materialized, and its rows are compared with the layout. Rows that don't fit get 0A000 / RevalidateCachedQuery, which pgjdbc heals by re-parsing.
  • uuid parameters (OID 2950) are accepted in text and binary form.

What I checked:

  • Compilation: Dropping the 4th getColumns argument at the portal call sites still resolves, because the 3-argument overload remains. stabilizeProjectedUndeclaredColumns is removed together with its only caller. stableTypeForUndeclared is still used by the Describe('S') and sample paths and by PostgresTypeTest.
  • Layout state: columnsFromRows is copied in bindFrom and set in rememberDescribedLayout. fixedColumnsOf no longer fixes the columns for such a layout, which is consistent with the check in executeCommand.
  • writeError: The length computation includes the optional routine field, and the field is written before the terminator, so the message framing is correct.
  • uuid parsing: parseUuidText validates the length and the hex digits before Long.parseUnsignedLong, and the binary path checks for exactly 16 bytes. Invalid input raises PostgresProtocolException and is refused at Bind. I found no injection or resource hazard.
  • Tests: The new IT covers the libpq exchange, the pgjdbc exchange, the pgjdbc type round trip and uuid binding. The 9009 IT is updated to match the new behavior. This follows the repository's regression-test conventions.

Minor suggestions (not blocking):

  1. parseUuidText is a package-private static method, so a small unit test would be cheap. It could cover malformed input such as a wrong length, non-hex characters and unbalanced braces. Today it is only exercised through the IT.
  2. The accepted spellings are slightly more lenient than PostgreSQL, which allows hyphens only between 4-digit groups, and this PR strips them anywhere. That is harmless for a lookup parameter, but worth a comment.
  3. The PR description already notes the trade-off for pgjdbc inside an explicit transaction with autosave=never, where the RevalidateCachedQuery refusal now reaches the application. The behavior should be called out in the release notes or docs. The inference about the pgjdbc source is unverified, as the author says.
  4. CLAUDE.md asks for the final keyword where possible. parseUuidText reassigns hex, which is fine, but the rest of the diff looks consistent with the repository style.

No security concerns and no unrelated changes.

@github-actions github-actions Bot added triage:lgtm PR triage: nothing blocking, ready for a maintainer and removed triage:lgtm PR triage: nothing blocking, ready for a maintainer labels Nov 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Triage: triage:lgtm · severity:minor · areas: postgresql-wire · has-issue

Reviewed the diff for #11838 (postgresw only). I did not run the build or the tests.

What it does

  • A Describe('P') of a named statement is now typed from the portal's own rows, so psycopg and pgjdbc get the same OIDs as the unnamed statement.
  • A layout kept from those rows is flagged columnsFromRows. A later portal executed under it without its own Describe is checked with fitsDescribedLayout. If its rows do not fit, it is refused with 0A000 / RevalidateCachedQuery.
  • uuid parameters (OID 2950) are now bound in both text and binary form.

Findings

  • Scope is tight and matches the issue.
  • Security: the uuid parsing is bounded, with a strict 32 hex digit check and a 16-byte check for binary. Bad input raises PostgresProtocolException. No new dependencies.
  • Tests: Issue11838PreparedSchemalessTypesIT is new, and the Issue9008And9009 assertion is updated. They cover the named/unnamed OID parity, the refusal path and uuid binding.
  • Style follows CLAUDE.md: final locals, imports instead of fully qualified names, no debug output.
  • Imports: ProjectionItem and Expression are still used elsewhere in PostgresNetworkExecutor, so removing stabilizeProjectedUndeclaredColumns leaves no unused imports.

Minor suggestions, not blockers

  • fitsDescribedLayout runs getColumns over the whole materialized result for each undescribed portal. That is acceptable, but the cost grows with row count on large pgjdbc results.
  • The PR body says the pgjdbc retry behaviour is inferred from source and not run. The new jdbcKeepsTheTypesOnceItPreparesTheStatement test exercises it, which is good.
  • Check whether PostgresType.stableTypeForUndeclared is still used. It is called at PostgresNetworkExecutor.java:2045 in the base branch, so it likely is.

The one behaviour change to be aware of is that a pgjdbc application can now get 0A000 inside an explicit transaction when a schemaless property changes type between executions. The PR body documents this trade-off.

lvca added a commit that referenced this pull request Nov 21, 2026
…me values they wrap (#11979)

Fixes #11955.

### Problem

openCypher hands out its temporals as `CypherDate`,
`CypherLocalDateTime`, `CypherDateTime`, `CypherLocalTime` and
`CypherTime`. That covers function results and a stored DATE / DATETIME
read as `n.prop`. Bolt unwrapped them to `java.time` in a private
`BoltStructureMapper.unwrapCypherTemporal`. The gRPC encoder
(`GrpcTemporalCodec.encode`) and the Postgres wire
(`PostgresType.getTypeForValue` / `toText` / `serializeAsBinary`) had no
arm for them, so they went out as untagged `string_value` and as
`varchar`. SQL sends the same property as a tagged `timestamp_value` and
as `date` / `timestamp`.

### Change

- **Engine:** a new static `CypherTemporalValue.toJavaTime(Object)`
returns the `java.time` value a Cypher date, time or datetime wraps, and
any other value unchanged (one `instanceof` check for a value that is
not a Cypher temporal). `CypherDuration` stays as it is, since each
protocol already has its own encoding for it.
- **Bolt:** uses it in place of its private `unwrapCypherTemporal`,
which is removed. Same behaviour.
- **gRPC:** `GrpcTemporalCodec.encode` unwraps first, so a Cypher
temporal gets the kind and `logical_type` its `java.time` value already
gets (#9969). The codec is shared with the gRPC client's `ProtoUtils`.
- **Postgres:** `getTypeForValue`, `toText` and `serializeAsBinary`
unwrap first. A Cypher date is now announced as `date` (1082) and a
local datetime as `timestamp` (1114). A zoned datetime, time or local
time stays `varchar`, as the SQL `ZonedDateTime` / `OffsetTime` /
`LocalTime` does. The elements of a list are unwrapped too
(`arrayElementToText`, `serializeArrayToString`, `toJsonElement`), so a
Cypher `collect(n.dt)` is written as SQL writes the same list.

### Measured (E2E, `proto_repros.py F1`, container with this branch's
engine, server, network, grpcw, postgresw and bolt jars)

| column | gRPC before | gRPC after | Postgres before | Postgres after |
|---|---|---|---|---|
| `n.da` (DATE) | `string_value` "2024-02-29" | `timestamp_value` /
`date` (same as SQL) | varchar | date 1082 `2024-02-29` (same as SQL) |
| `n.dt` (DATETIME) | `string_value` "2024-02-29T13:14:15.123" |
`timestamp_value` / `datetime` (same as SQL) | varchar | timestamp 1114
`2024-02-29 13:14:15.123000` (same as SQL) |
| `date()` | `string_value` | `timestamp_value` / `date` | varchar |
date |
| `localdatetime()` | `string_value` | `timestamp_value` / `datetime` |
varchar | timestamp |
| `datetime(...Z)` | `string_value` | `string_value` / `zoned_datetime`
| varchar | varchar (as SQL's ZonedDateTime) |
| `localtime()` | `string_value` "10:00" | `string_value` / `local_time`
"10:00:00" | varchar | varchar |
| `time()` | `string_value` "10:00+01:00" | `string_value` /
`offset_time` "10:00:00+01:00" | varchar | varchar |

Bolt is unchanged (`Date`, `DateTime`, `Time`). The server log no longer
prints the "has no encoding of its own" warning for these classes.

### Tests

- `Issue11955GrpcCypherTemporalIT`: the Cypher columns are equal to the
SQL ones for the stored properties, and each function result carries its
kind and tag.
- `Issue11955GrpcCypherTemporalWireTest`: the gRPC converter encodes
each wrapper as the `java.time` value it wraps, alone and inside a list
or a map.
- `PostgresColumnTypeResolutionIT.CypherTemporalColumn` (pgjdbc and a
raw wire client): column types DATE / TIMESTAMP, the decoded `LocalDate`
/ `LocalDateTime` values, a list of datetimes written as SQL writes it,
and the binary result format.
- `PostgresTypeTest.CypherTemporalValues` and
`CypherTemporalValueToJavaTimeTest`: the type, the text and the binary
encoding of every wrapper on the Postgres wire (alone, in a list, in a
nested list), and the engine helper.
- They fail on main: the gRPC test where `n.da` from Cypher differs from
SQL (`string_value`), the Postgres one on `n.da` typed VARCHAR (12)
instead of DATE (91), the list one on `{2024-02-29T13:14:15.123}`
instead of `{"2024-02-29 13:14:15.123000"}`. They pass with the fix.

### Overlap

#11967 (#11958) also edits `PostgresType.java`, but in the json
rendering (`toText`'s record/map branch and the json array helpers).
This PR adds one line at the top of `toText`, `serializeAsBinary` and
`getTypeForValue`, and one in each of the three list element helpers. If
git reports a conflict it is textual, and either PR can merge first. Not
related to #11920.

#11968 (#11960) conflicts with this PR in the `PostgresType.java` import
block: #11968 adds the `CypherDuration` import and this PR adds
`CypherTemporalValue` on the same line. The conflict is textual only.
Whichever PR merges second needs a rebase that keeps both imports.

---------

Co-authored-by: lvca <l.garulli@arcadedata.com>
lvca added a commit that referenced this pull request Nov 21, 2026
…oes (#11967)

Fixes #11958.

### Problem

A Postgres-wire json column holding a record (`@this`, a Cypher node) or
a map (`properties(n)`, a map literal) was rendered by
`record.toJSON(true)` or `new JSONObject(map)`. Temporals came out as
epoch numbers: a DATE as epoch days in a record and epoch milliseconds
in a map, and a map dropped the zone of a ZONED_DATETIME and the
sub-millisecond digits of a DATETIME_NANOS. HTTP sends the same row as
formatted text.

### Change

- `PostgresType` renders a record, a result row, a map or a path through
`JsonSerializer`, the serializer the HTTP API uses, wherever a json
column carries one: the scalar column, an element of a json column
holding an array (a path included, e.g. `collect(p)`), and an element of
a `text[]` or `json[]` array in text and binary format. Vertex edge
counts are left out, as `toJSON(true)` left them out.
- The date patterns come from the database the value belongs to: the
record's, or the row's (`Result.getDatabase()`) when the row is not an
element. A value no database owns (a map literal, a row built without
one) uses the configured default patterns.
- One `JsonSerializer` instance serves every connection
(`PostgresType.JSON_DOCUMENT_SERIALIZER`). It is configured once in the
static initializer and never changed, and a serialization keeps no state
in the serializer (date patterns and type hints live in locals of each
call), the same pattern as gRPC's shared `FAST`/`SAFE` serializers. No
per-value allocation of a serializer on the row path.
- A record that is not a document (an edge segment, a stripe directory,
the payload of an EXTERNAL property) keeps its own `toJSON(true)`:
`JsonSerializer` has no form for those and would hand the record back,
so the column would carry its `toString()`.
`Issue11958JsonDocumentShapesTest.aRecordThatIsNotADocumentKeepsItsOwnJson`
pins it.
- `JsonSerializer` gets a public `serializeValue(Database, Object)` that
delegates to the existing private `serializeValue(database, value,
null)`. The schema pattern lookups now go through two helpers that fall
back to `GlobalConfiguration.DATE_FORMAT` / `DATE_TIME_FORMAT` when the
database is null. Every HTTP call site passes a database, so their
output is unchanged;
`Issue11958SerializeValueTest.httpRowRenderingIsUnchanged` pins the HTTP
rendering of these values.

### Behaviour change for clients (release notes)

This changes what a Postgres client receives in a json column. Scalar
temporal columns (DATE, TIMESTAMP OIDs) are unchanged.

- **Temporals inside json documents are text, not epoch numbers.** A
client that read a DATE or DATETIME inside a json column as a number
(epoch days or epoch milliseconds) must parse the formatted text
instead, the same text and the same schema `dateFormat` /
`dateTimeFormat` the HTTP API sends.
- **Paths** (`MATCH p = ... RETURN p`) keep the `{_type, nodes,
relationships, length}` object with the same members in the same order.
Each node and relationship gains the record attributes the HTTP API
sends with it: `@rid`, `@type`, `@cat`, plus `@in` / `@out` on a
relationship. `path.toMap()` wrote each record with `toJSON(false)`,
which left them out. Non-temporal property values are the same. Measured
in `aPathKeepsTheShapeOfItsMapAndGainsTheRecordAttributes`:
- before:
`{"_type":"path","nodes":[{"name":"a","n":7,"da":19782,"dt":1709212455123,"dtn":"2024-02-29
13:14:15.123456789",...}],"relationships":[{"da":19782,"w":2}],"length":1}`
- after:
`{"_type":"path","nodes":[{"@Rid":"#1:0","@type":"T11958","name":"a","n":7,"da":"2024-02-29","dt":"2024-02-29
13:14:15.123","dtn":"2024-02-29
13:14:15.123456789",...,"@cat":"v"}],"relationships":[{"@Rid":"#4:0","@type":"E11958","da":"2024-02-29","w":2,"@cat":"e","@in":"#1:1","@out":"#1:0"}],"length":1}`
- **Rows that are not elements** (a projection such as `SELECT da AS d,
name FROM T` yields one) keep the members and the order of
`result.toJSON()`; only the temporals change. Measured in
`aRowThatIsNotAnElementKeepsTheShapeOfItsJson`:
- before:
`{"name":"a","n":7,"da":1709164800000,"dt":1709212455123,"dtn":1709212455123,"nested":{"da":1709164800000}}`
- after: `{"name":"a","n":7,"da":"2024-02-29","dt":"2024-02-29
13:14:15.123","dtn":"2024-02-29
13:14:15.123456789","nested":{"da":"2024-02-29"}}`
- **DECIMAL still differs from HTTP.** A DECIMAL in those documents is
unchanged: a JSON number with its exact digits
(`12345678901234567890.123456789`), where HTTP sends a string.

### Measured (E2E, psycopg 3.3.9, container with this branch's engine
and postgresw jars)

`SELECT @this AS rec, {'da': da, 'dt': dt, 'dtn': dtn, 'zdt': zdt} AS m
FROM T` and `{cypher}MATCH (n:T) RETURN n, properties(n) AS props`:

| property | before, record | before, map | after, record and map (same
as HTTP) |
|---|---|---|---|
| DATE | `19782` | `1709164800000` | `"2024-02-29"` |
| DATETIME | `1709212455123` | `1709212455123` | `"2024-02-29
13:14:15.123"` |
| DATETIME_NANOS | `"2024-02-29 13:14:15.123456789"` | `1709212455123` |
`"2024-02-29 13:14:15.123456789"` |
| ZONED_DATETIME | `"2024-03-10T03:30-04:00[America/New_York]"` |
`1710055800000` | `"2024-03-10T03:30-04:00[America/New_York]"` |

This E2E table was measured on the first commit; the follow-up commit's
paths are covered by the tests below, not by a second psycopg run.

### Tests

- `Issue11958JsonColumnTemporalsIT` (pgjdbc): a record and a map in SQL
and Cypher (fails on main: DATE `19782` vs `"2024-02-29"`); a Cypher
path (`MATCH p = ... RETURN p`); Cypher lists of nodes, maps and paths
(`collect(n)`, `collect(properties(n))`, `collect(p)`); a map inside a
`text[]` (`SELECT ['text', {...}]`); the rows of a subquery (`SELECT
(SELECT da, dt, dtn, zdt FROM T) AS rows`).
- `Issue11958JsonDocumentShapesTest` (postgresw, embedded): a json
column holding a list of records, maps, nested lists and paths; a json
document inside a `text[]` literal and inside binary `text[]` and
`json[]` arrays; path and non-element row shapes compared with
`path.toMap()` and `result.toJSON()`; a non-element query row rendered
with its database's custom `dateFormat`; the non-document record branch.
- `Issue11958SerializeValueTest` (engine): pins the HTTP row rendering
and checks `serializeValue` matches it, with and without a database.
- Run on the latest commit: `mvn -o -pl postgresw -am verify
-DskipITs=false
-Dtest='com/arcadedb/postgres/**/*Test,Issue11958SerializeValueTest'
-Dit.test='com/arcadedb/postgres/**/*IT'`: postgresw unit 1001 run, ITs
624 run, engine 2 run; 0 failures, 0 errors, 0 skipped.

### Overlap with #11920

#11920 (#11838) also edits `PostgresType.java`, in the Bind parameter
decoding (`deserializeText` / `deserializeBinary`, UUID). This PR
touches only the json rendering of result values (`toText`,
`arrayElementToText`, the json array helpers), so the two do not change
the same code.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed JSON serialization of date and date-time values when records are
returned through PostgreSQL, including values nested in maps, lists,
arrays, and paths.
* Temporal values now retain their expected formatting in
database-backed and detached records, including date precision and time
zones.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: lvca <l.garulli@arcadedata.com>
…t as the unprepared one does, and bind a uuid parameter

A Describe('P') of a portal of a named statement announced every undeclared property as varchar (text[] for a list), so a client
that prepares a statement (psycopg from its 6th execution, pgjdbc from its 5th) read true as 't' and 42 as '42'. The portal is now
typed from its rows as the unnamed statement and the simple query protocol type it. A layout kept for the statement from those rows
is applied to a later portal executed without a Describe only when its rows have the same types; otherwise the Execute is refused
with 0A000 'cached plan must not change result type' (routine RevalidateCachedQuery), which pgjdbc heals by preparing again.

A uuid parameter (OID 2950, text or binary) is bound as its canonical text.
@shaunpatterson

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (8da9049) to resolve conflicts in PostgresType and the deleted Issue9008And9009MixedTypesIT. Resolution: kept main's new temporal and interval constants alongside this PR's UUID_CODE; main folded the #9008/#9009 test into PostgresTypeEncodingIT, so the #11838 re-describe assertions (column types of u, a missing column a) are ported there. Re-ran Issue11838PreparedSchemalessTypesIT (4), PostgresTypeEncodingIT (23), PostgresDescribeIT (31), PostgresTypeTest (239): green.

@shaunpatterson
shaunpatterson force-pushed the fix/11838-pg-prepared-column-types branch from e71f2fc to f2cb55c Compare November 21, 2026 21:06
@github-actions github-actions Bot added triage:needs-changes PR triage: the PR needs changes before it can merge and removed triage:lgtm PR triage: nothing blocking, ready for a maintainer labels Nov 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Triage: triage:needs-changes · severity:minor · areas: postgresql-wire · has-issue

Thanks for the thorough write-up and tests. The approach is sound. Describing each portal from its own rows fixes the inconsistent 't' vs True results, and the uuid parameter support is a clean addition. A few things need attention before merge.

Please address

  1. Behavior regression risk for non-pgjdbc clients. A portal executed under a layout that was read from rows, with no Describe of its own, is now refused with 0A000 cached plan must not change result type. Only pgjdbc treats RevalidateCachedQuery as "re-Parse and retry". Other drivers that Describe a portal once and then send Bind, Execute will see a hard error where they previously got text values. Your own trade-off section notes the same for pgjdbc inside an explicit transaction. Please say which drivers you verified this against (psycopg, asyncpg, npgsql, pgx, node-postgres), or add a test for at least one non-pgjdbc path. Consider whether the refusal should apply only when the client negotiated binary transfer, since only then do the rows really "not fit". A text-format client could still be served.
  2. Unused code after the removal. stabilizeProjectedUndeclaredColumns is deleted. Please check that PostgresType.stableTypeForUndeclared, the 4-arg getColumns overload and the imports SelectStatement, ProjectionItem and Expression are still used, and remove any that are now dead. I could not confirm this from the diff alone.
  3. parseUuidText is looser than PostgreSQL. hex.replace("-", "") accepts hyphens in any position, for example ----0f8f.... It also uses Character.digit, which accepts non-ASCII digits such as full-width forms. Restrict it to ASCII hex and accept hyphens only at the 8-4-4-4-12 positions, or at least document the looseness. The error message echoes the raw input, so please truncate it.
  4. Performance. fitsDescribedLayout calls getColumns over the full materialized row set on every Bind/Execute under a rows-derived layout, and it also materializes portals that used to be streamed. Please confirm this cannot affect large result sets, or bound it. For example, a streamed portal could be checked row by row, or only while the first N rows are checked.

Minor

  • Test classes in this module are named *IT, which is fine. Please make sure the new tests do not rely on fixed ports (the base class should handle this).
  • The [INFERENCE ... not run] pgjdbc claim in the description would be better backed by a test of the in-transaction case.

The changes follow CLAUDE.md on the points I could check: final parameters, no new dependencies, and regression tests present.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

has-issue The PR links an issue it resolves postgresql-wire severity:minor Cosmetic, documentation, edge case, or a workaround exists triage:needs-changes PR triage: the PR needs changes before it can merge

Projects

None yet

1 participant