gdal vector sql

Added in version 3.11.

Apply SQL statement(s) to a dataset.

Synopsis

Usage: gdal vector sql [OPTIONS] <INPUT> [<OUTPUT>]

Apply SQL statement(s) to a dataset.

Positional arguments:
  -i, --dataset, --input <INPUT>                       Input vector datasets [required] [not available in pipelines]
  -o, --output <OUTPUT>                                Output vector dataset [not available in pipelines]

Common Options:
  -h, --help                                           Display help message and exit
  --json-usage                                         Display usage as JSON document and exit
  --config <KEY>=<VALUE>                               Configuration option [may be repeated]
  -q, --quiet                                          Quiet mode (no progress bar or warning message) [not available in pipelines]

Options:
  -f, --of, --format, --output-format <OUTPUT-FORMAT>  Output format ("GDALG" allowed) [not available in pipelines]
  --co, --creation-option <KEY>=<VALUE>                Creation option [may be repeated] [not available in pipelines]
  --lco, --layer-creation-option <KEY>=<VALUE>         Layer creation option [may be repeated] [not available in pipelines]
  --overwrite                                          Whether overwriting existing output dataset is allowed [not available in pipelines]
  --update                                             Whether to open existing dataset in update mode [not available in pipelines]
  --overwrite-layer                                    Whether overwriting existing output layer is allowed [not available in pipelines]
  --append                                             Whether appending to existing layer is allowed [not available in pipelines]
                                                       Mutually exclusive with --upsert
  --skip-errors                                        Skip errors when writing features [not available in pipelines]
  --sql <statement>|@<filename>                        SQL statement(s) [may be repeated] [required]
  --output-layer <OUTPUT-LAYER>                        Output layer name(s) [may be repeated]
  --dialect <DIALECT>                                  SQL dialect (e.g. OGRSQL, SQLITE)

Advanced Options:
  --if, --input-format <INPUT-FORMAT>                  Input formats [may be repeated] [not available in pipelines]
  --oo, --open-option <KEY>=<VALUE>                    Open options [may be repeated] [not available in pipelines]
  --output-oo, --output-open-option <KEY>=<VALUE>      Output open options [may be repeated] [not available in pipelines]
  --upsert                                             Upsert features (implies 'append') [not available in pipelines]
                                                       Mutually exclusive with --append

Description

gdal vector sql returns one or several layers evaluated from SQL statements.

Starting with GDAL 3.12, when using --update, and without an output dataset specified, this can be used to execute statements that modify the input dataset, such as UPDATE, DELETE, etc.

GDALG output (on-the-fly / streamed dataset)

This program supports serializing the command line as a JSON file using the GDALG output format. The resulting file can then be opened as a vector dataset using the GDALG: GDAL Streamed Algorithm driver, and apply the specified pipeline in a on-the-fly / streamed way.

Program-Specific Options

--output-layer <OUTPUT-LAYER>

Output SQL layer name(s). If not specified, a generic layer name such as "SELECT" may be generated.

Must be specified as many times as there are SQL statements, either as several --output-layer arguments, or a single one with the layer names combined with comma.

--quiet

Added in version 3.12.

Silence potential information messages.

--sql <sql_statement>|@<filename>

SQL statement to execute that returns a table/layer (typically a SELECT statement).

Can be repeated to generated multiple output layers (repeating --sql <value> for each output layer)

--dialect <DIALECT>

SQL dialect.

By default the native SQL of an RDBMS is used when using gdal vector sql. If using sql as a step of gdal vector pipeline, this is only true if the step preceding sql is read, otherwise the OGRSQL dialect is used.

If a datasource does not support SQL natively, the default is to use the OGRSQL dialect, which can also be specified with any data source.

The SQL SQLite dialect dialect can be chosen with the SQLITE and INDIRECT_SQLITE dialect values, and this can be used with any data source. Overriding the default dialect may be beneficial because the capabilities of the SQL dialects vary.

Supported dialects can be checked with gdal --format. For example:

$ gdal --format "PostgreSQL"
[...]
Supported SQL dialects: NATIVE OGRSQL SQLITE
[...]

$ gdal --format "ESRI Shapefile"
[...]
Supported SQL dialects: OGRSQL SQLITE
[...]

Standard Options

Details

--append

Whether appending features to existing layer(s) is allowed. This also creates the output dataset if it does not exist yet.

--co, --creation-option <NAME>=<VALUE>

Many formats have one or more optional dataset creation options that can be used to control particulars about the file created. For instance, the GeoPackage driver supports creation options to control the version.

May be repeated.

The dataset creation options available vary by format driver, and some simple formats have no creation options at all. A list of options supported for a format can be listed with the --formats command line option but the documentation for the format is the definitive source of information on driver creation options. See Vector drivers format specific documentation for legal creation options for each format.

Note that dataset creation options are different from layer creation options.

--if, --input-format <format>

Format/driver name to be attempted to open the input file(s). It is generally not necessary to specify it, but it can be used to skip automatic driver detection, when it fails to select the appropriate driver. This option can be repeated several times to specify several candidate drivers. Note that it does not force those drivers to open the dataset. In particular, some drivers have requirements on file extensions.

May be repeated.

--lco, --layer-creation-option <NAME>=<VALUE>

Many formats have one or more optional layer creation options that can be used to control particulars about the layer created. For instance, the GeoPackage driver supports layer creation options to control the feature identifier or geometry column name, setting the identifier or description, etc.

May be repeated.

The layer creation options available vary by format driver, and some simple formats have no layer creation options at all. A list of options supported for a format can be listed with the --formats command line option but the documentation for the format is the definitive source of information on driver creation options. See Vector drivers format specific documentation for legal creation options for each format.

Note that layer creation options are different from dataset creation options.

--oo, --open-option <NAME>=<VALUE>

Dataset open option (format specific).

May be repeated.

-f, --of, --format, --output-format <OUTPUT-FORMAT>

Which output vector format to use. Allowed values may be given by gdal --formats | grep vector | grep rw | sort

--output-open-option, --output-oo <NAME>=<VALUE>

Added in version 3.12.

Dataset open option for output dataset (format specific).

May be repeated.

--overwrite

Allow program to overwrite existing target file or dataset. Otherwise, by default, gdal errors out if the target file or dataset already exists.

--overwrite-layer

Whether overwriting the existing output vector layer is allowed.

--skip-errors

Added in version 3.12.

Whether failures to write feature(s) should be ignored. Note that this option sets the size of the transaction unit to one feature at a time, which may cause severe slowdown when inserting into databases.

--update

Whether to open an existing output dataset in update mode.

--upsert

Added in version 3.12.

Variant of --append where the OGRLayer::UpsertFeature() operation is used to insert or update features instead of appending with OGRLayer::CreateFeature().

This is currently implemented only in a few drivers: GPKG -- GeoPackage vector, Elasticsearch: Geographically Encoded Objects for Elasticsearch and MongoDBv3 (drivers that implement upsert expose the GDAL_DCAP_UPSERT capability).

The upsert operation uses the FID of the input feature, when it is set (and the FID column name is not the empty string), as the key to update existing features. It is crucial to make sure that the FID in the source and target layers are consistent.

For the GPKG driver, it is also possible to upsert features whose FID is unset or non-significant (the --unset-fid option of gdal vector edit can be used to ignore the FID from the source feature), when there is a UNIQUE column that is not the integer primary key.

-q, --quiet

Suppress progress bar and some warning messages.

Return status code

The program returns status code 0 in case of success, and non-zero in case of error (non-blocking errors emitted as warnings are considered as a successful execution).

Examples

Example 1: Generate a GeoPackage file with a layer sorted by descending population

$ gdal vector sql in.gpkg out.gpkg --output-layer country_sorted_by_pop --sql="SELECT * FROM country ORDER BY pop DESC"

Example 2: Generate a GeoPackage file with 2 SQL result layers

$ gdal vector sql in.gpkg out.gpkg --output-layer=beginning,end --sql="SELECT * FROM my_layer LIMIT 100" --sql="SELECT * FROM my_layer OFFSET 100000 LIMIT 100"

Example 3: Modify in-place a GeoPackage dataset

$ gdal vector sql --update my.gpkg --sql "DELETE FROM countries WHERE pop > 1e6"

Example 4: Add a new field to an existing layer of a GeoPackage

$ gdal vector sql --update my.gpkg --sql "ALTER TABLE countries ADD COLUMN abbrev STRING(10)"

Example 5: Append to an existing layer of a GeoPackage file

$ gdal vector pipeline read europe.gpkg ! \
                       sql --sql "SELECT * FROM country WHERE pop > 1e6" ! \
                       write --append --output-layer-name=world world.gpkg