Helpers for working with Cloud Spanner’s spanner.GenericColumnValue and related client types: format values to text (literals, JSON, CLI-style output) and construct values from Go types.
Requires Go 1.25 or later (see go.mod).
| Package | Role |
|---|---|
github.com/apstndb/spanvalue |
Format spanner.GenericColumnValue and *spanner.Row using FormatConfig and presets such as LiteralFormatConfig, JSONFormatConfig, SpannerCLICompatibleFormatConfig. |
github.com/apstndb/spanvalue/gcvctor |
Build spanner.GenericColumnValue (scalars, ARRAY, STRUCT, typed nulls). Types are often composed with github.com/apstndb/spantype/typector. |
github.com/apstndb/spanvalue/protofmt |
Opt-in descriptor-aware PROTO and ENUM display plugins for FormatConfig. |
github.com/apstndb/spanvalue/writer |
Stream Spanner rows to CSV, TSV, JSONL, or SQL INSERT (writer/README.md). |
github.com/apstndb/spanvalue/dbsqlrows |
Experimental. Driver-agnostic database/sql export — see package documentation. |
QuoteIdentifier and QuoteQualifiedIdentifier are conservative quoting
helpers. They always quote for the selected dialect, escape embedded quote
characters (for GoogleSQL, string-literal escapes: backslash to \\ and
backtick to \` ), and do not attempt a minimal "quote only when
necessary" strategy.
DATABASE_DIALECT_UNSPECIFIEDfollows the Spanner default and uses GoogleSQL quoting.QuoteQualifiedIdentifierquotes each dotted path segment independently.- The helpers do not validate empty identifiers or empty path segments; callers that reject those shapes must do so before calling them.
quotedTable := spanvalue.QuoteQualifiedIdentifier(
databasepb.DatabaseDialect_GOOGLE_STANDARD_SQL,
"analytics.daily_metrics",
)
quotedColumn := spanvalue.QuoteIdentifier(
databasepb.DatabaseDialect_GOOGLE_STANDARD_SQL,
"select",
)
// quotedTable == "`analytics`.`daily_metrics`"
// quotedColumn == "`select`"SpannerCLICompatibleFormatConfig
matches official spanner-cli
output, including bracket-style STRUCT in arrays ([[1, east]]). For tuple
parentheses ([(1, east)]) while keeping CLI scalar rules, prepend a
PluginForStruct
override with FormatTupleStruct
(prepended plugins run before the preset handlers):
fc := spanvalue.SpannerCLICompatibleFormatConfig().WithComplexPlugin(
spanvalue.PluginForStruct(spanvalue.FormatSimpleStructField, spanvalue.FormatTupleStruct))See ExampleSpannerCLICompatibleFormatConfig_tupleStruct. Keep product-specific combinations in your application (not as new spanvalue presets).
FormatConfig holds exactly two fields: NullString and the ordered FormatComplexPlugins chain. Preset constructors (LiteralFormatConfig, SimpleFormatConfig, and others) return configs that pass FormatConfig.Validate. Prefer assembling custom configs with NewFormatConfig, which validates the canonical array/struct/scalar handlers at build time. After hand-assembling or mutating a config—Clone() then edit FormatComplexPlugins—call Validate before the first format or export so an empty chain or an empty NullString fails at construction time rather than on the first row.
Validate cannot prove chain coverage: a non-NULL value that every plugin defers fails at format time with ErrUnhandledValue. Writers accept any *FormatConfig via writer.WithFormatter but do not call Validate today—validate hand-built formatters before passing them to writers.
fc, err := spanvalue.NewFormatConfig(
spanvalue.WithNullString("NULL"),
spanvalue.WithPlugin(myPlugin), // optional overrides, most recent first
spanvalue.WithArrayFormat(spanvalue.FormatUntypedArray),
spanvalue.WithStructFormat(spanvalue.FormatSimpleStructField, spanvalue.FormatTupleStruct),
spanvalue.WithScalarFormatter(spanvalue.FormatNullableSpannerCLICompatible),
)
if err != nil {
return err
}Use the small helper APIs directly when replacing ad hoc downstream formatting code:
jsonLine, err := spanvalue.FormatRowJSONObjectFromColumns(
spanvalue.JSONFormatConfig(),
columnNames,
gcvs,
spanvalue.IndexedUnnamedFieldNamer,
)w, err := writer.NewSQLInsertWriter(out, "analytics.daily_metrics")
if err != nil {
return err
}
if err := w.WriteValues(columnNames, gcvs); err != nil {
return err
}
return w.Flush()Nested ARRAY/STRUCT test fixtures: gcvctor.MustStructValueOf and gcvctor.MustArrayValueOf wrap the error-returning constructors and panic on construction errors—use only in tests with schema-known inputs (see gcvctor package docs).
elemType := typector.CodeToSimpleType(sppb.TypeCode_STRING)
row := gcvctor.MustStructValueOf(
[]string{"id", "tags"},
[]spanner.GenericColumnValue{
gcvctor.Int64Value(1),
gcvctor.MustArrayValueOf(elemType, gcvctor.StringValue("a"), gcvctor.StringValue("b")),
},
)Package writer streams
*spanner.Row, structpb cells, or []spanner.GenericColumnValue to CSV, quoted TSV,
JSONL, or SQL INSERT using spanvalue formatters. writer/README.md
covers RowIterator lifecycle (WriteRowIterator,
RunRowIterator,
hooks and decorators, RowsRead), schema registration, format edge cases, and v0.5.x
expectations if writer becomes a separate module.
When every query returns at least one row, iter.Do plus WriteRow and Flush is enough
(the first row registers column names). When a SELECT may return zero rows but metadata
still lists columns, use WriteRowIterator (see writer README).
iter := txn.Query(ctx, stmt)
w, err := writer.NewCSVWriter(out, writer.WithFormatter(cfg))
if err != nil {
return err
}
if err := iter.Do(func(row *spanner.Row) error {
return w.WriteRow(row)
}); err != nil {
return err
}
return w.Flush()Writer package details (GCV export options, RowIterator vs WriteGCVs): writer/README.md.
go-sql-spanner apps often decode query
rows into []spanner.GenericColumnValue (for example via proto decode options) and
export with spanvalue writers: scan → GCV slice →
writer.WriteGCVs.
Column names: database/sql does not surface Spanner
*spannerpb.ResultSetMetadata; register columns with
writer.WithColumnNames
from your scan metadata (or rows.Columns() plus any unnamed-field policy).
When the app already holds *spannerpb.ResultSetMetadata (for example proto decode),
writer.WithMetadata
is appropriate. For display headers outside the writer, use
spanvalue.ColumnNames
on the same field list with the same
spanvalue.UnnamedFieldNamer
as writer.WithUnnamedFieldNamer.
Duplicate aliases: explicit duplicate field names from Spanner (for example SELECT 1 AS a, 2 AS a) are preserved in ColumnNames output (["a", "a"]). Only UnnamedFieldNamer collisions return errors. CSV/TSV headers and JSONL field names follow the same resolved names; see writer/README.md.
CSV / JSONL: register schema and formatting at construction, stream rows, then
Flush (CSV may emit a header on zero-row SELECT; JSONL Flush is a no-op).
When the app already holds *spannerpb.ResultSetMetadata (for example from proto
decode), pass metadata, formatter, and namer together with
writer.DelimitedGCVExportOptions
or writer.JSONLGCVExportOptions
(nil arguments are skipped):
w, err := writer.NewCSVWriter(out, writer.DelimitedGCVExportOptions(
metadata,
spanvalue.SimpleFormatConfig(),
spanvalue.IndexedUnnamedFieldNamer,
)...)
if err != nil {
return err
}
defer rows.Close()
for rows.Next() {
var gcvs []spanner.GenericColumnValue
// decode the scanned row into gcvs
if err := w.WriteGCVs(gcvs); err != nil {
return err
}
}
if err := rows.Err(); err != nil {
return err
}
return w.Flush()If you only have column names from database/sql (no ResultSetMetadata), use
separate With* options instead:
namer := spanvalue.IndexedUnnamedFieldNamer
names := []string{"id", "name"} // same names passed to WithColumnNames
w, err := writer.NewCSVWriter(
out,
writer.WithColumnNames(names),
writer.WithFormatter(spanvalue.SimpleFormatConfig()),
writer.WithUnnamedFieldNamer(namer),
)
if err != nil {
return err
}
defer rows.Close()
for rows.Next() {
var gcvs []spanner.GenericColumnValue
// decode the scanned row into gcvs
if err := w.WriteGCVs(gcvs); err != nil {
return err
}
}
if err := rows.Err(); err != nil {
return err
}
return w.Flush()Native Spanner client: for *spanner.RowIterator, use
writer.WriteRowIterator
or writer.RunRowIterator
instead of building GCV slices per row.
Metadata pseudo-rows, NextResultSet progression, and stats-only result sets stay
in the application (for example spannersh).
Delimited output uses SimpleFormatConfig
by default. Build cells with gcvctor.EnumValue
and gcvctor.ProtoValue,
then WriteGCVs (see TestDelimitedWriterWriteGCVsEnumProto in the writer package).
For display paths where protobuf descriptors are available, prepend opt-in
protofmt plugins to a cloned
formatter. These plugins render PROTO values as protobuf text and ENUM values
as names; they are display-oriented and do not replace descriptor-free SQL
literal output such as FormatProtoAsCast / FormatEnumAsCast. Descriptor
loading and compilation stay in the application. If you enable multiline
prototext, nested ARRAY/STRUCT cells and delimited-output fields can contain
embedded newlines.
Delimited, JSONL, and SQL encodings differ after
spanvalue formats each column; see writer/README.md. For
value-oriented paths, use
writer.RowData, writer.FormatDelimitedRow, writer.FormatJSONLRow, or
writer.FormatJSONLRowSeq
directly. Pass the JSON field-name policy explicitly, for example:
line, err := writer.FormatJSONLRow(
spanvalue.JSONFormatConfig(),
row,
spanvalue.IndexedUnnamedFieldNamer,
)CSV output:
func writeCSV(out io.Writer, rows []*spanner.Row) error {
w, err := writer.NewCSVWriter(out)
if err != nil {
return err
}
for _, row := range rows {
if err := w.WriteRow(row); err != nil {
return err
}
}
return w.Flush()
}Quoted TSV uses the same CSV-style writer with a tab delimiter (encoding/csv
quoting: embedded tabs, quotes, and newlines in a field are escaped). For CSV output,
NewCSVWriter is a thin helper for NewDelimitedWriter(out, writer.Comma). Pass
writer.Comma when using the generic delimited constructor for CSV output.
Delimiters must be non-zero valid runes other than ", \r, \n, or
utf8.RuneError.
func writeTSV(out io.Writer, rows []*spanner.Row) error {
w, err := writer.NewDelimitedWriter(out, '\t')
if err != nil {
return err
}
for _, row := range rows {
if err := w.WriteRow(row); err != nil {
return err
}
}
return w.Flush()
}Some CLIs expose a legacy TAB format that joins pre-formatted column strings
with \t and does not apply CSV-style quoting. That is not what
NewDelimitedWriter(out, '\t') emits. To keep raw tab-separated output while
still using spanvalue formatters, implement writer.Writer
(or writer.RowIteratorWriter
when streaming via writer.WriteRowIterator): format each column, strings.Join(fields, "\t"),
then write the line.
JSONL output:
func writeJSONL(out io.Writer, rows []*spanner.Row) error {
w, err := writer.NewJSONLWriter(out)
if err != nil {
return err
}
for _, row := range rows {
if err := w.WriteRow(row); err != nil {
return err
}
}
return w.Flush()
}SQL INSERT output uses Spanner GoogleSQL quoting by default. Use
writer.WithSQLInsertKind for INSERT OR IGNORE or INSERT OR UPDATE; see
INSERT DML syntax.
writer.WithSQLDialect controls identifier quoting and insert-kind validation,
not value literal formatting. For PostgreSQL-dialect value literals, pass a
PostgreSQL-aware formatter with writer.WithFormatter (for example
spanpg.PostgreSQLLiteralFormatConfig)
together with writer.WithSQLDialect.
func writeInserts(out io.Writer, table string, rows []*spanner.Row) error {
w, err := writer.NewSQLInsertWriter(out, table)
if err != nil {
return err
}
for _, row := range rows {
if err := w.WriteRow(row); err != nil {
return err
}
}
return w.Flush()
}Integration tests that exercise the Spanner client with PostgreSQL dialect (TypeAnnotation on query params and row metadata) are maintained in github.com/apstndb/spanpg (integration/pgtypeannotation), not in this repository.