`gdal raster materialize`

Added in version 3.12.

Materialize a piped dataset on disk to increase the efficiency of the following steps

Description

The materialize operation is for use only in a pipeline, and writes the incoming dataset to a dataset on disk, before passing that materialized dataset to following steps. This can be used when materializing a dataset helps for performance compared to on-demand evaluation of previous steps.

By default, a tiled compressed GeoTIFF file is used for the materialized dataset, created in the directory pointed by the CPL_TMPDIR configuration option or in the current directory if not specified, and is deleted when the pipeline finishes. The user can also select another format, or specify an explicit filename, in which case the materialized dataset is not deleted when the pipeline finishes.

Note

VRT output is not compatible with materialize

Synopsis

* materialize [OPTIONS]
-----------------------

Materialize a piped dataset on disk to increase the efficiency of the following steps.

Options:
  -o, --output <OUTPUT>                                Materialized dataset name (created by algorithm)
  -f, --of, --format, --output-format <OUTPUT-FORMAT>  Output format
  --co, --creation-option <KEY>=<VALUE>                Creation option [may be repeated]
  --overwrite                                          Whether overwriting existing output dataset is allowed

Program-Specific Options

--output <OUTPUT>: Optional dataset name. When specified, it is not removed at the end of the process.

Standard Options

Details

--co, --creation-option <NAME>=<VALUE>

Many formats have one or more optional creation options that can be used to control particulars about the file created. For instance, the GeoTIFF driver supports creation options to control compression, and whether the file should be tiled.

May be repeated.

The creation options available vary by format driver, and some simple formats have no creation options at all. A list of options supported for a format can be listed with the --formats command line option but the documentation for the format is the definitive source of information on driver creation options. See Raster drivers format specific documentation for legal creation options for each format.

-f, --of, --format, --output-format <OUTPUT-FORMAT>: Which output raster format to use. Allowed values may be given by gdal --formats | grep raster | grep rw | sort

--overwrite: Allow program to overwrite existing target file or dataset. Otherwise, by default, gdal errors out if the target file or dataset already exists.

Return status code

The program returns status code 0 in case of success, and non-zero in case of error (non-blocking errors emitted as warnings are considered as a successful execution).

Examples

Example 1: Reproject a GeoTIFF file to CRS EPSG:32632 ("WGS 84 / UTM zone 32N"), materialize it to a temporary file and compute its contour lines

$ gdal pipeline ! read in.tif ! reproject --output-crs=EPSG:32632 ! \
                ! materialize ! contour --interval=10 ! write out.gpkg --overwrite

Example 2: How to use materialize for Cloud Optimized GeoTIFF (COG)

Usually when you want load COG data, you are not loading the whole data but instead on certain region and using lower resolution for fast analysis. Therefore, after using command read, it is not supposed to be materialize right away because it mean it will download the whole thing into the memory/disk. Instead, you could use command reproject and use --bbox to limit the region of interest and set --size or --resolution to use lower resolution overview.

$ gdal pipeline ! read /vsicurl/https://some.storage.com/mydata/landcover.tif \
                ! reproject -r mode -d EPSG:4326 --bbox=112,2,116,4.5 --bbox-crs=EPSG:4326 --size=3000,3000 \
                ! materialize \
                ! zonal-stats --zones=/vsicurl/https://some.storage.com/mydata/adm_level4.fgb --stat=values \
                ! write out.geojson

gdal raster materialize