= GDAL/OGR 3.1.4 Release Notes =

The 3.1.4 release is a bug fix release.

== Build ==

* Add compatiblity with Jasper 2.0.21 (#3012)

* Support CharLS 2.1 on Debian as well (#3083)

* fix build in AVC driver in VSIL_STRICT_ENFORCE mode

* mkgdaldist.sh: remove doc/.doxygen_up_to_date from generated tarballs (#3032)

== Port ==

* CPLIsFilenameRelative(): treat "scheme://.." filenames as absolute

* VSIS3FSHandler::SetFileMetadata(): fix memleak.

* CPLJSONDocument::LoadUrl(): avoid potential memory leak

* VSI_CACHE: do not trust unreliable file size from underlying layer (#3006)

* /vsicurl_streaming/: do not trust unreliable file size (#3006)

* /vsicurl_streaming/: avoid potential libcurl bug (at least as of 7.47) in multi-threading (#3006)

== GDAL Algorithms ==

* Warper: emit error message instead of assertion if cutline is not a (multi)polygon (#3037)

* TPS transformer : fix handling of duplicated GCPs (#2917)

== GDAL utilities ==

* gdal2tiles: fix regression of #2808 when tmaxz != nativezoom (#2932)

* gdal2tiles.py: make sure configuration options specified with --config are passed to worker processes (#2950)

* gdal2tiles: fix --profile=raster on a non-georeferenced image (#2998)

* gdalcompare: fix floor division in Python 3

* gdalbuildvrt: fix -srcnodata / -vrtnodata handling in -separate mode (#2978)

== GDAL drivers ==

Multiple drivers:

* JPEG2000 drivers: extend signature for JPEG2000 codestream to avoid false positive detections.

AAIGRID driver:

* Improve search for float data. (#2972)

COG driver:

* fix crash when source dataset is non-Byte/non-UInt16 with a color table (#2946)

GPKG driver:

* fix when writing a tile with uniform negative values or values > 65535 (when nodata is set) in PNG tiles (#3044)

GRIB driver:

* fix retrieval of nodata value for GRIB1 products (GDAL 3.1 regression, #2962)

* correctly report PDS template number for messages with subgrids (#3004)

GTiff driver:

* LERC codec: do not write TIFFTAG_LERC_PARAMETERS several times as it cause spurious directory rewrites, and breaks for example COG creation

* support hidden SHIFT_ORIGIN_IN_MINUS_180_PLUS_180=YES open option used by GDALOpenVerticalShiftGrid()

* Internal libtiff: fix out-of-bounds write triggered by GDAL when repeatedly reading a corrupt strip. Fixes ossfuzz#25493

* Internal libtiff: tif_fax3.h: fixes for ossfuzz#25552 and ossfuzz#26201

JP2OpenJPEG:

* fix reading overviews on Sentinel2 PVI files (343x343 size, with 8x8 tiles) (#2984)

PCRaster driver:

* fix Create() mode by propagating eAccess = GA_Update (#2948)

VRT driver:

* VRTDataset::IRasterIO(): allow source overviews to be used when non-nearest resampling is used, and the VRT bands don't expose overviews (#2911)

* fix VRTRasterBand nodata handling when creating implicit overviews (#2920)

== OGR core ==

* fix memory leak in OGRGeometry::Crosses

* OGR SQL: support constructs 'A AND B AND C ... AND N' with many successive AND (#2989)

== OGR drivers ==

CSV driver:

* do not try to read .csvt if CSV filename has no extension (#3006)

ESRIJSON driver:

* fix GetFeatureCount() and GetExtent()

FlatGeobuf driver:

* metadata extensions backport (#2976)

GML driver:

* avoid 'Destination buffer too small' error to be emitted on /vsicurl_streaming/ URLs with filters coming from the WFS driver

LIBKML driver:

* do not advertize RandomWrite capability (unless on a update layer, when datasource is created with UPDATE_TARGETHREF creation option) (QIGS #39087)

NAS driver:

* do not try to write a .gfs file when NAS_GFS_TEMPLATE is specified

OAPIF driver:

* fix memory leak when reading schema from .xsd

OCI driver:

* fix server 12.2 version detection

ODS and XLSX drivers:

* do not create files with Zip64 extension, to avoid compatibility issue with LibreOffice Calc

PGeo driver:

* fix libPath where /usr/local/lib64 or /usr/lib would not have been tried

SEGY driver:

* fix potential double free

WFS driver:

* avoid /vsicurl_streaming/ URL to be truncated in case of big filter (but the server might reject it)

= GDAL/OGR 3.1.3 Release Notes =

The 3.1.3 release is a bug fix release.

== Build ==

* configure: fix detection of libtiff and libjpeg on mingw (#2881)

* configure: fix linking order for Informix libraries

* Unix: GNUmakefile: make 'all' target an alias of the default one to avoid potential double build of OGR objects (#2777)

* Unix: fix detection of minor version number of Poppler with the new YY.MM.X numbering scheme (#2823)

* gdallinearsystem.cpp: add missing include to fix compilation issue on Slackware 14.2 (#2883)

* fix build with Jasper 2.0.19 (#2844)

== Port ==

* CPLHTTPFetch(): unset CURLOPT_POST after request has been issued (in case of re-use of a connection and doing a POST and then a GET)

* CPLHTTPFetch(): reset CURLOPT_HTTPHEADER after request

* VSI Plugin: add caching option (#2901)

== Algorithms ==

* warping: ComputeSourceWindow(): modify extra source pixel computation

* polygonize: make sure not to use dummy geotransform

== GDAL utilities ==

* gdal2tiles: Fix layer generation with raster profile and nonnative zoom (#2799)

* gdal2tiles: fix generation of tiles at high zoom levels when input is small (#2896)

* gdal_merge: fix rounding of source coordinates, when they are very close to an integer, which would otherwise result in a one-pixel shift

== GDAL drivers ==

BAG driver:

* avoid crash on non-standard dataset

E00grid driver:

* avoid recursive call in _GetNextSourceChar(). Fixes ossfuzz#25161

ENVI driver:

* add support for writing south-up / rotation=180 datasets

FITS driver:

* initialize default geotransform

GPKG driver:

* more robust and simple logic to build overviews and compute overview factor (#2858) (#2860)

* on reading of gridded coverage data with PNG tiles, select -FLT_MAX as the nodata value (#2857)

GeoTIFF driver:

* fix wrong direction for half-pixel shift with GCPs and PixelIsPoint convention (https://github.com/opengeospatial/OGC-API-Sprint-August-2020/issues/18)

* SRS reader: interpret infinite value in GeogInvFlatteningGeoKey as 0 (fixes PROJ#2317)

* Internal libgeotiff: avoid look up of user-defined GeogGeodeticDatumGeoKey that cause PROJ warnings (fixes libgeotiff#41)

Grass driver:

* fix reading GRASS groups (#2876)

HDF4 driver:

* multidim: fix issue when reading transposed array, and duplicate attribute names (#2848)

HDF5 driver:

* multidim: fix performance issue when reading from sliced array

ISCE driver:

* avoid crashing division by zero on corrupted datasets. Fixes ossfuzz #24252

netCDF driver:

* multidim: fix performance issue when reading from sliced array

* fix setting offset and scale in CreateCopy()

RasterLite2 driver:

* remove support for CharLS compression since it is removed from upstream librasterlite2

SAFE driver:

* deal correctly with WV swaths (#2843)

TSX driver:

* fix issue with reading dataset in .zip file on Windows (#2814)

VICAR driver:

* avoid potential null-dereference on corrupted dataset. Fixes ossfuzz #24254

VRT driver:

* round src/dst coordinates to integer within 1e-3 margin

== OGRSpatialReference ==

* Make OSRGetPROJSearchPaths() return the value set by OSRSetPROJSearchPaths()

== OGR utilities ==

* ogrinfo/ogr2ogr: fix issues with -sql @filename where SQL comments are not at start of line (#2811)

== OGR drivers ==

DGNv8/DWG driver:

* ogrteigha.cpp: use correct printf formatter (fix 44ce5be4)

FlatGeobuf driver:

* fix illegal use of std::vector (#2773)

* make GetExtent() work on feature write (#2874)

GeoJSON driver:

* fix opening of file starting with {geometry:{coordinates (fixes #2787)

* RFC7946 writer: fix processing of geometry that covers the whole world (#2833)

GML driver:

* writer: correctly format OFTDate and OFTDateTime fields (#2897)

GPKG driver:

* add a DATETIME_FORMAT=WITH_TZ/UTC dataset creation option (defaults to WITH_TZ) to specify how to deal with non-UTC datetime values (#2898)

MITAB driver:

* fix reading and writing of non-metre linear units

MVT driver:

* writing: fix crashes in multi-threading mode (#2764)

OAPIF driver:

* avoid re-adding user query parameters if they are found in URLs returned by the API (relates to #2873)

ODBC driver:

* Fix ODBC driver fails to correctly fallback to alternative Access ODBC driver name

* Fix DSN string construction for Windows Access ODBC driver, template candidate preference order (#2878)

PGeo/ODBC/Geomedia/Walk drivers:

* Quote DBQ value in driver template to avoid issues opening MDB paths with spaces

* Correctly handle datetime fields provided by the mdbtools ODBC driver

PGeo driver:

* Fixes to automatic ODBC driver installation (#2838)

Shapefile driver:

* when several candidate SRS are found with confidence >= 90%, take the one from EPSG (contributes to fixes QGIS#32255)

== CSharp bindings ==

* Adding typemaps C# for wrapper_GDALWarpDestDS and wrapper_GDALWarpDestName (#2621)

* Expose Dataset.GetSpatialRef (#2620)

* Expose GetCRSInfoListFromDatabase (#1665)

= GDAL/OGR 3.1.2 Release Notes =

The 3.1.2 release is a bug fix release.

== Build ==

Windows:

* nmake.opt: add hint about adding ole32.lib to PROJ_LIBRARY for PROJ 7.1 when static linking (#2743)

* nmake.opt: fix ECW hints (#2748)

== GDAL core ==

* DumpJPK2CodeStream(): fix reported offset of EOC marker when PSOT = 0 (#2724)

* Overview generation: fix progress percentage when using USE_RRD=YES (#2722)

== GDAL utilities ==

* gdal2tiles: fix issue in nativezoom computation with --profile=raster on a raster whose size is below the tile size

* gdal2tiles: fix --xyz with -p raster, fix KML generation with --xyz (fixes #2463) and update OpenLayers export to OpenLayers 6.3.1 (#2732)

* validate_gpkg.py: add expected column type for rtree with SQLite 3.31.1

== GDAL drivers ==

ISIS3 driver:

* make sure that in-line label size is at least 65536 bytes (fixes #2741)

PAux driver:

* avoid ingesting large binary unrelated files (found when investigating #2722)

== OGRSpatialReference ==

* Fix crashing bug with PROJ context and OSRCleanup(), as triggered by PostGIS raster and GMT (#2744)

== OGR drivers ==

CouchDB driver:

* ogrcouchdblayer.cpp: fix compiler warning

DXF driver:

* ogrdxf_leader.cpp: suppress false positive warning

FileGDB and OpenFileGDB drivers:

* qualify DateTime values with UTC timezone when <IsTimeInUTC>true</IsTimeInUTC> is present in layer metadata (#2719)

GeoJSON driver:

* fix opening of file starting with {"coordinates" (#2720)

* ogrgeojsonreader.cpp: fix compiler warning

GML driver:

* when encountering XML issue, defer emission of error message until we return a NULL feature, so as to avoid to confuse ogr2ogr (#2730)

== SWIG Bindings ==

Python bindings:

* gdal.Info(): fix handling of -json in options (#2749)

= GDAL/OGR 3.1.1 Release Notes =

The 3.1.1 release is a bug fix release.

== Build ==

Unix:

* configure: fix CharLS 2.1 detection on case insensitive filesystems (#2710)

* GNUmakefile: split long line to avoid 32,000 character limit on MSYS2

Windows:

* nmake.opt: mention shell32.lib in PROJ_LIBRARY (#2488)

* nmake.opt: revert unwanted change regarding OCI_LIB / OCI_INCLUDE, and also allow to override them (#2533)

* EXR: add missing Windows build support

* generate_vcxproj.bat: add vs2019 compatibility (#2676)

* nmake.opt: add a vc142 reference for ECW

== Doc ==

* generate man pages for gdalmdiminfo, gdalmdimtranslate and gdal_viewshed (#2538)

* Various fixes

== Misc ==

* Driver metadata: fix XML errors in option declarations and add testing (#2656)

== Port ==

* CPLStrtodDelim(): recognize '1.#SNAN' as a NaN value

* /vsiaz/: fix OpenDir()/NextDirEntry() that returned S_IFDIR for regular files

* /vsimem/: fix normalization of slashes in filenames to avoid potential infinite loop in VSIDirGeneric::NextDirEntry()

== GDAL algorithms ==

* Warper/transformer: avoid error about invalid latitude when warping a dataset in Geographic CRS whose north/south lat is > 90deg (#2535)

* Warper: fix average resampling that lead to very wrong results in some circumstances (#2665) (3.1.0 regression) (#2667)

* Viewshed: adjust computation of observer position (#2599)

== GDAL utilities ==

* gdaldem: ignore 'nv' entry in color file if there is no nodata value in input file

* gdal2tiles: make general cmd line switches like --formats work without exception (#2522)

* gdal_calc.py: raise exception in case of I/O error. (QGIS #36867)

* gdalwarp: fix crash if warping a dataset without source or target CRS when -ct is specified (#2675)

* gdal_rasterzie: fix an error string regarding -ts parameter (#2698)

== GDAL core ==

* GDALDataset::IRasterIO(): Fixes wrong IO of subpixel shifted window (#2507)

* GDALMDArray::GetScale() and GetOffset(): avoid nullptr dereference

== GDAL drivers ==

BAG driver:

* fix for big endian arch

COG driver:

* avoid crash if passing an invalid (warp) resampling option

* add tolerance when computing tile numbers, so as to make -co TILING_SCHEME= idempotent on a file already properly tiled

ECW driver:

* fix related to network files with SDK >= 5.5 (#2652)

GPKG driver:

* fix GDAL 3.0 regression regarding some update scenarios (#2325)

* increase limitation of number of tables to 10000, and make it configurable through OGR_TABLE_LIMIT config option as for vector tables

GRIB driver:

* avoid rejecting valid product due to over-zealous security check

GTiff driver:

* fix reading/writing GEO_METADATA TIFF tag on big-endian

* fix importing WGS_1984_Web_Mercator / ESRI:102113 (#2560)

* Internal libtiff: TWebPSetupEncode(): fix logic problem in test that checks input is 8bit unsigned data

* fix potential crash when generating degenerate 1x1 overviews

HDF5 driver:

* multidim API: fixes for big endian host

HFA driver:

* do not report TOWGS84 when reading SRS with WGS84, NAD27 or NAD83 datums (unless OSR_STRIP_TOWGS84 config option is set to NO) (QGIS #36837)

INGR driver:

* avoid invalid cast on corrupted data. (ossfuzz #22351)

LCP driver:

* add extension checking in Identify() (#2641)

PDS driver:

* take into account FIRST_STANDARD_PARALLEL for Mercator projection (#2490)

TileDB driver:

* use returned length to truncate string for tiledb metadata

VICAR driver:

* fix for Basic compression and non-Byte type on big endian host

== OGR core ==

* ogr_geometry.h: export OGRWktOptions class (#2576)

* swq_op_general.cpp: Fix type conversion issues in some arithmetic expressions

* OGR SQL: fix buffer overflow in BuildParseInfo for SQL query when joining multiple tables that each have implicit FID columns.

== OGRSpatialReference ==

* Make GetAuthorityCode('PROJCS') work on a WKT1 COMPD_CS with a VERT_DATUM type = 2002 (Ellipsoid height)

* Avoid warnings and fix crash in GetProjTLSContextHolder() (#2691)

== OGR utilities ==

* ogr2ogr: add missing line feed in usage message (#2548)

* ogr2ogr: make -f PostgreSQL work when using PG:service= syntax

== OGR drivers ==

DXF driver:

* avoid potential big stack calls when eliminated 999 comment lines (ossfuzz #22668)

Elastisearch driver:

* make OVERWRITE_INDEX=YES work properly by re-creating the index afterwards

GML driver:

* fix typo in VFR GFS files BonitovaneDilRizeniId -> BonitovanyDilRizeniId (#2682)

GMLAS driver:

* fix so as to get same unique ids on big-endian arch

GPKG driver:

* hide also view "geometry_columns" (#2518)

* fix wrong RTree _update3 trigger on existing files (QGIS #36935)

GTM driver:

* writing: fix timezone handling issues (#2696)

MITAB driver:

* fix reading and writing of Transverse Mercator projections based on KKJ

* .tab: fix writing empty/null Time fields (#2612)

MVT driver:

* revise logic to attribute FID when reading directories (#2566)

netCDF driver:

* fix for big-endian

* GrowDim(): fix issue with non-ASCII filename on Windows

ODS driver:

* avoid potential deep call stack in formula evaluation (ossfuzz #22237)

PDF driver:

* write correctly attribute object dictionary when there is no field to write, and read back correctly broken files we generated before (#2624)

S57 driver:

* apply update to DSID_EDTN field (#2498)

Shapefile driver:

* SHPRestoreSHX: fix for (64 bit) big endian

SXF driver:

* fixes for big-endian

XLSX driver:

* fix numeric precision issue when reading datetime that could lead to an error of 1 second (#2683)

== SWIG bindings ==

All:

* {OGRDriver}.CopyDataSource(): check that source dataset is not NULL

* validate range of resample_alg algorithm (#2591)

Python bindings:

* Fix Python2 install to be synchronous and report all errors (#2515)

= GDAL/OGR 3.1.0 Release Notes =

== In a nutshell... ==

* Implement RFC 75: support for multidimensional arrays in MEM, VRT, netCDF, HDF4, HDF5 and GRIB drivers. Read/write for MEM and netCDF. Read/only for others. Add gdalmdiminfo and gdalmdimtranslate utilities.

* Implement RFC76: add capability of writing vector drivers in Python

* New GDAL drivers:

- COG: write-only, for Cloud Optimized GeoTIFF

- EXR: read/write driver, relying on OpenEXR library

- ISG: read-only, for geoid models of the International Service for the Geoid

- RDB: read-only, for RIEGL Database .mpx RDB 2 files (#1538) (needs proprietary SDK)

* New OGR drivers:

- FlatGeoBuf: read-support and creation (#1742)

- MapML: read/write driver for experimental web spec

* Improved drivers:

- OAPIF driver (renamed from WFS3): updated to OGC API - Features 1.0 core spec

- GTiff: improve performance of internal overview creation

- GTiff: GeoTIFF 1.1 support

- Shapefile driver: add read/creation/update support for .shz and .shp.zip

- netCDF vector: read/write support for CF-1.8 Encoded Geometries (#1287)

- VICAR: multiple improvements and write support (#1855)

- DDS: add read support

* Other improvements:

- gdalwarp: accept output drivers with only CreateCopy() capabilities

- gdal_viewshed: new utility for viewshed algorithm

* Remove GFT driver now that the online service no longer exists (#2050)

* New Sphinx-based documentation

* Multiple security related fixes (ossfuzz)

* Continued code linting (cppcheck, CoverityScan, etc.)

* Compatibility with GDAL 3.0:

- C and C++ API: backward compatible changes

- C ABI: backward compatible changes

- C++ ABI: modified

- Functional changes: see MIGRATION_GUIDE.TXT

== Build ==

== GDAL/OGR 3.1.0 - General Changes ==

Build(Unix):

* use pkg-config for libxml2 detection (#2173)

* fix detection of libpq in a non-standard place (#1542)

* do not use absolute path in linking command. Helps Mac OS and cygwin builds (#2075)

* Enable Bash completions and control installation

* GDALmake.opt.in: silence datarootdir warning

* Doc: allow user full control over installation directory

* fix JVM detection for HDFS support on MacOS (#2313)

* Remove #define HOST_FILLORDER from cpl_config.h (#2345)

* Added search for proj library in lib64 directory.

* configure: strip -L/usr/lib and similar from netCDF, MySQL, GEOS and SFCGAL lib path (#2395)

* configure: remove useless -lproj from --with-spatialite dectection

Build(Windows):

* parametrize number of CPUs for parallel builds with CPU_COUNT variable (#1922)

* add HDF5_H5_IS_DLL variable to switch the scenario when HDF5 is built as a DLL (#1931)

* add POSTFIX that defaults to _d for GDAL .dll, .lib and .pdb for DEBUG builds (#1901)

* Fix issues with thread_local and C++ objects that don't work well with DLL on Windows

All:

* Support Poppler 0.82, 0.83, 0.85

== GDAL 3.1.0 - Overview of Changes ==

== Algorithms ==

* Warper: add sum resampling method (#1437)

* Average resampling (warp and overview/translate): use weighted average for border source pixels

* GDALReprojectImage(): properly take into account source/target alpha bands

* GDALCreateReprojectionTransformerEx(): do not emit error if reverse transformation fails, and fix crash when trying to use null reverse transformation

* Warper: fix GDAL 2.3 regression in a situation with source nodata value, multiple bands and nearest resampling where the logic to detect which source pixels are nodata was inverted (#1656)

* GWKAverageOrModeThread(): reject invalid source pixels for average/q1/q3/mode/min/max resampling (#2365)

* Multithreaded warper: make sure a transformer object is used by the thread which created it (#1989). This workarounds a PROJ bug also fixed per https://github.com/OSGeo/PROJ/pull/1726

* Contour: fix SegmentMerger list iterator skipping and out of bounds error (#1670)

* Contour: fix (over) precision issue when comparing pixel value to NoData on Float32 rasters (#1987)

* Contour: add sanity checks for interval based contouring, in case the dataset contains extreme values regarding the settings, which would lead to a lot of memory allocations / too large computation time

* TPS warper: enhance precision without armadillo support (#1809)

* RPC warper: fix issue when source image has a geotransform (#2460)

* GDALRasterizeGeometries(): fix potential integer overflow / memory allocation failure, depending on GDAL_CACHEMAX and raster dimensions (#2261)

* Rasterize: speed optimization for geometry collections (#2369)

* GDALContourGenerate(): propagate raster acquisition error (#2410)

== Port ==

* /vsitar/: support >100 character file names (#1559)

* /vsitar/: accept space as end of field terminator

* /vsigz/: fix seeking within .gz/.tgz files larger than 2 GB (#2315)

* /vsicurl (and derived filesystems): fix concurrency issue with multithreaded reads (#1244)

* /vsicurl/: avoid downloading one extra block when the end offset is just at a chunk boundary

* /vsicurl/: fix CPL_VSIL_CURL_ALLOWED_EXTENSIONS with query string (#1614)

* /vsicurl/: allow 'Connection timed out' CURL errors as candidate for HTTP retry

* /vsicurl/: GetFileSize(): when HEAD request does not return Content-Length header, retry with GET

* /vsis3/: for a long living file handle, refresh credentials coming from EC2/AIM (#1593)

* /vsis3/: invalidate cached non-existing file is AWS_ config options are changed in the meantime (#2294)

* /vsis3/ /vsigs/ /vsiaz/: implement Rename() first doing a copy of the original file and then deleting it

* /vsis3/ and similar: add a NUM_THREADS option to Sync() for parallelized copy

* AWS: Fix error in loading ~/.aws/config file (#2203)

* VSISync(): when copying from /vsis3/ to /vsis3/ (or /vsigs/->/vsigs/, /vsiaz/->/vsiaz/), use CopyObject() method to use server side copy

* VSISync(): make file copying from /vsis3/ actually use /vsis3_streaming/ to reduce number of GET requests

* VSISync(): add a CHUNK_SIZE option to Sync() to split large objects and get parallelization of their download and upload

* Add VSIUnlinkBatch() for batch deletion of files, and add optimized /vsis3/ implementation

* Add efficient VSIRmdirRecursive() implementation for /vsis3/

* Add VSIGetFileMetadata()/VSISetFileMetadata() and implement them to get/set HTTP headers, and AWS S3 object tagging

* /vsis3/ and other network filesystems: avoid useless network requests when we already got a directory listing (#2429)

* /vsiswift/: V3 authentication method, handling auth token expiration

* /vsimem/: make Rename() error if destination file is not in /vsimem/

* /vsizip/ writing: in ZIP64 mode, also advertise 45 as the version in the central directory (avoids a warning from 'zip' utility)

* Add CPLCanRecode function and use it in MITAB, Shape and SXF drivers to decide when to advertize UTF-8 capability

* CPLConfigOptionSetter: only reset thread-locale value, not global one

* CPLJSONObject::GetType(): return Long when the value doesn't fit on a int32

* CPLJSON: distinguish Null type from Unknown/invalid type

* CPLEscapeString(): escape double-quote for CPLES_URL

* Add a CPLCondTimedWait()

* cpl_safemaths.hpp: safe + and * for GUInt64

* Add CPLJSonStreamingWriter class

* Add cpl_error_internal.h with logic with error accumulator

* Add VSIOverwriteFile()

* Add CPLLaunderForFilename()

* cpl_error.h: add a CPLDebugOnly() macro that expands to CPLDebug() only for DEBUG builds

* QuietDelete: support expliciting the drivers to use

== Core ==

* Block cache: fix corruption on multithreaded write on datasets (#2011)

* GDALInvGeoTransform(): make it work with scale and rotation/skew coefficients of small absolute value (#1615)

* GDALCopyWholeRasterGetSwathSize(): fix potential int overflows for big values of GDAL_SWATH_SIZE or GDAL_CACHEMAX

* PAMRasterBand: add presence flag for Offset and SetScale so that GetOffset()/GetScale() properly reports if they are defined in PAM

* GDALDefaultOverviews::BuildOverviews(): fix typo in detection of 1x1 overview (#1730)

* Fix precision loss at GDALResampleChunkC32R with complex data

* Fix precision loss at GDALComputeBandStats with complex data

* Proxy pool: Load band block sizes if not provided at creation. Fixes floating point exception on copy overviews from PRF dataset to destination dataset.

* GTiff and PAM: allow serializing WKT2 for SRS using non-WKT1 compatible projections such as Vertical Perspective (#1856)

* RasterIO(): fix non-neareset resampling over nodata blocks (#1941)

* Overview dataset (-oo OVERVIEW_LEVEL): expose mask if the source dataset has a mask with overviews

* GDALUnrolledCopy<GByte,2,1>: fix SSE2-only implementation (when SSSE3 is not available, on older AMD CPUs)

* PAM: support reading GCPs from ESRI GeodataXform in .aux.xml files

* Make it possible to call BuildOverviews() on a dataset returned by GDALBuildVRT()

* GDALDataset::SetProjection(): re-allow the use of PROJ4 strings (#2034)

== Utilities ==

* GDALInfo(): fix axis order issue in lon,lat corner coordinates, in particular when reading from a .aux.xml with a ProjectedCRS (#2195)

* gdal_translate: Make 'gdal_translate foo.tif foo.tif.ovr -outsize 50% 50% -of GTiff' work

* gdal_translate: clamp/round source nodata value when not compatible of the target data type. Was already done when using -a_nodata, but not with implicit copy (#2105)

* gdalwarp: accept output drivers with only CreateCopy() capabilities

* gdalwarp: adjust nodata values, passed with -srcnodata/-dstnodata, and close to FLT_MAX to exactly it (#1724)

* gdalwarp: fix wrong axis order when using source/target CRS being a geographic3D CRS such as EPSG:4979, and with vertical shift grid application (#1561, GDAL 3.0 regression)

* gdal_contour: remove explicit width/precision=12/3 of the elev field (#1487)

* gdal_contour: turn on quiet mode if output dataset is standard output (refs #2057)

* gdaldem: avoid potential integer overflow in color-relief mode (#2354)

* gdal_calc.py: fixed NaN-streaking in output images when the --allBands option is given to tiled images

* gdal_polygonize.py: fix outputing to geojson without explicit -f switch (#1533)

* gdalcompare.py: take into account mask bands

* gdal_retile.py: add resume option (#1703)

* gdallocationinfo: emit verbose error when dataset canot be opened (#1684)

* gdallocationinfo and gdaltransform: print a hint when values are expected from the command line and stdin is an interactive terminal (refs #1684)

* Python utilities: fix GetOutputDriverFor() when multiple drivers found (#1719)

* gdal2tiles.py: add remaining resample methods

* gdal2tiles.py: add option for setting the tile size (#2097)

* gdal2tiles.py: add --xyz option to generate tiles in the OSM Slippy Map standard (#2098)

* gdal2tiles.py: show warning when running against non-Byte input (#1956)

* gdal2tiles.py: update cache calculation (#2020)

* gdal2tiles.py: check that min zoom <= max zoom (#2161)

* gdal2tiles.py: ignore nodata values that are not in the range of the band data type

* gdal2tiles.py: fix hang when --s_srs specified but image lacks georeferencing

* gdal_translate / gdalwarp / ogrct: allow dealing with non-WKT1 representable SRS (#1856)

* gdal_edit.py: add a -units switch

* gdal_edit.py: add -a_ulurll switch

* gdal_fillnodata.py/GDALFillNodata: fix crash when smooth_iterations is used, and with some progress functions such as the one used by Python (#1184)

* Python scripts and samples: use python3 for shebang (#2168)

== Sample scripts ==

* Add tile_extent_from_raster.py: sample script to generate the extent of each raster tile in a overview as a vector layer

* Add gdal_remove_towgs84.py script

== GDAL drivers ==

Multiple drivers:

* GTiff, GPKG, MBTiles, PostgisRaster drivers: share lock of overview dataset with parent dataset (#1488)

* HDF5 and netCDF: fix crash when reading attributes of type string of variable length with NULL values

* CTable2/LOSLAS/NTv1/NTv2: document in metadata that positive longitude shift values are towards west

* Revise raster drivers GDAL_DMD_HELPTOPIC

* JP2KAK and JP2OPENJPEG: fix to read images whose origin is not (0,0)

* Strip TOWGS84 when datum is known, in GTiff, Spatialite and GPKG drivers

BAG driver:

* modify way georeferencing is read (particularly pixel sizes and origin) (#1728)

* Backward compatibility for metadata reading for BAG < 1.5 (#2428)

BSB driver:

* Report PLY coordinates as a WKT POLYGON in a BSB_CUTLINE metadata item

BYN driver:

* fix nodata value for Int32 encoded products

DAAS driver:

* accept 4-band RGBA PNG response even when a single band is requested

DIMAP driver:

* don't look inside Dataset_Components if Raster_Data is present

* avoid reparsing xml if already a product dimap

* ignore missing strip xml file

DDS driver:

* add read support

DTED driver:

* Add DTED_APPLY_PIXEL_IS_POINT environment switch (#2384)

* emit a CE_Failure instead of a CE_Warning in case of checksum verification failure (#2410)

* support E96 as well as MSL for COMPD_CS (#2422)

EEDA driver:

* fix startTime / endTime comparisons that were incomplete in #1506

FITS driver:

* fix memory leaks

ENVI driver:

* add read support for reading GCPs (#1528), and fix off-by-one offset on line,pixel on reading GCP

* fix potential use of invalid pointer on some unusual std::string implementations (#1527)

* preserve 'byte order' on update (#1796)

GPKG driver:

* support opening subdataset of /vsicurl/ files (#2153)

GTiff driver:

* improve performance of internal overview creation (#1442)

* in COPY_SRC_OVERVIEWS=YES, interleave mask with imagery, and add leader/trailer to strile

* optimize read of cloud-optimized geotiffs

* do not generate a TIFFTAG_GDAL_METADATA with color interpretation information for JPEG YCbCr compression

* make -co NUM_THREADS produce reproducible output

* make overview blocksize defaults to same as full-resolution

* move defered tile/strip offset/bytecount loading to libtiff

* make WEBP_LEVEL to be honored in Create() mode (fixes #1594)

* PushMetadataToPam(): early exit when PAM is disabled, to avoid error messages

* remove support for libtiff 3.X

* set a LAYOUT=COG metadata item in the IMAGE_STRUCTURE metadata domain of the dataset when the hidden feature declarations typical of a COG file are found

* fix memory leak with -co APPEND_SUBDATASET=YES

* fix error message for NBITS != 16 and Float32

* add explicit error message when trying to create a too big tiled TIFF file (refs #1786)

* ensure GDAL PROJ context is used for all libgeotiff functions (requires internal libgeotiff / libgeotiff 1.6)

* make sure that GetMetadataDomainList() doesn't return EXIF when there's no EXIF metadata

* GTIFGetOGISDefn: avoid querying UOM length info when it is KvUserDefined to avoid an error to be emitted

* on CRS reading, avoid unsetting of EPSG code when ProjLinearUnitsGeoKey = Linear_Foot_US_Survey and the CRS also uses that unit (#2290)

* do not write in GeoTIFF keys non-standard projections

* do not write TOWGS84 that come from EPSG codes, unless GTIFF_WRITE_TOWGS84=YES is explicitly set

* GTiff writing: workaround PROJ 6.3.0 bug when writing a EPSG:4937 ETRS89 Geog3D CRS

* GTiff writing: do not write by default EPSG:3857/WebMercator as a ESRI PE string. Fixes 3.0 regression

* avoid crash on single-component file with Whitepoint and PrimaryChromaticities tags

* libtiff: BigTIFF creation: write TileByteCounts/StripByteCounts tag with SHORT or LONG when possible

* Internal libgeotiff: set UOMLength from GeogLinearUnits (for geocentic CRS) (#1596)

* Internal libgeotiff: upgrade to libgeotiff 1.6.0dev to support OGC GeoTIFF 1.1

* Internal libtiff: resync with internal libtiff (post 4.1.0)

GRIB driver:

* do not do erroneous K->C unit conversion for derived forecasts whose content is not a temperature, but a derived quantity, such as spread

* avoid erroneous K->C conversion for Dew point depression

* update GRIB tables to degrib 2.24

* add missing entries in MeteoAtmoChem table

* add more values from Table 4.5 in Surface type table

* add support for template 4.48 Optical Properties of Aerosol

* add support for rotated lat-lon grids

* avoid quadratic performance on GRIB2 datasets using subgrids within a single GRIB message (#2288)

GXF driver:

* avoid closing the file pointer before being sure this is a GXF driver, otherwise this can prevent the opening of some raw format files (#1521)

HDF4 driver:

* fix retrieval of non-string swath and grid attributes

* fix GR support for non-Byte rasters

* remove broken attribure reading

* fixes related to color table

HDF5 driver:

* avoid error report of the HDF5 library when _FillValue attribute is missing

* Make GH5_FetchAttribute(CPLString) more robust and able to cope with variable-length string

* type detection: only detect complex data type if the component names start with r/R and i/I (refinement of #359)

* add a way to open datasets split over several files using the 'family' driver

* fix reading single dimension dataset (#2180)

HFA driver:

* fix writing of compressed file when a RLE run count is in the [0x4000,0x8000[ range or [0x400000, 0x800000[ (#2150)

IGNFHeightASCIIGrid driver:

* fix to read RASPM2018.mnt grid

IRIS driver:

* make identification more restrictive to avoid false-positive identification of raw binary formats such as ENVI (#1961)

ISIS3 driver:

* extract band name from BandBin group, and wavelength/bandwidth (#1853)

* preserve label in PAM .aux.xml when copying to other formats (#1854)

* add support for PointPerspective projection (#1856)

* add support for Oblique Cylindrical projection (#1856)

JP2ECW driver:

* add support for ECWJP2 SDK 5.5

JP2OpenJPEG driver:

* Fix multi-threading race condition (#1847)

* fix reading overviews, when tiled API is used, and the dimensions of the full resolution image are not a multiple of 2^numresolutions (#1860)

* fix to return the proper number of bytes read when we read more than 2 GB at once (fixes https://github.com/uclouvain/openjpeg/issues/1151)

JP2KAK driver:

* fix issue with multi-threaded reads

* NMAKE: Allow users to tweak/append extras to KAKINC (#1584)

JPEG driver:

* fix further calls to RasterIO after reading full image at full resolution (#1947)

* tune sanity check for multiple-scan (QGIS #35957)

* in case of multiscan image and implicit overviews, limit memory consumption

JPEGLS driver:

* add support for CharLS 2.1

KEA driver:

* return error when deleting metadata item rather than crashing (#1681)

* Backport thread safety fixes and nullptr tests from standalone driver (#2372)

LOSLAS driver:

* add support for .geo geoid models

MEM driver:

* Set access mode to the one required in Open()

MRF driver:

* fix relative file name detection

* relax TIFF tile format rules on read

* Accept any known projection encoding

* use PNG library for swapping

* fixes caching MRF issue

MrSID driver:

* add MRSID_PLATFORM to frmts/mrsid/nmake.opt

netCDF driver:

* improve performance when reading chunked netCDF 4 bottom-up files (read-only)

* correctly parse grid_mapping attribute in expanded form

* allow "radian" value for the X/Y axis units

* very partial workaround for an issue in libnetcdf 4.6.3 and 4.7.0 with unlimited dimensions (refs https://github.com/Unidata/netcdf-c/pull/1442)

* NCDFIsUserDefinedType(): make it work for types in subgroups

* Disregard valid range if min > max (#1811)

* for byte (signed/unsigned) variables, do not report a nodata value if there's no explicit _FillValue/missing_value as recommended by the netCDF specs. And for other data types use nc_inq_var_fill() to get the default value

* do not report nodata if NOFILL is set

* fix bottom-up identification with negative scale_factor for y axis (#1383)

* fix issue when opening /vsicurl/http[s]://example.com/foo.nc filenames (#2412)

* Fix support of WKT1_GDAL with netCDF rotated pole formulation

NGW driver:

* Add support for additional raster types and QGIS styles.

* Add more server side attribute filters.

NITF driver:

* skip bad UDID data (#1578)

NTv2 driver:

* add support for the Canadian NAD83(CRSR)v7 / NAD83v70VG.gvb velocity grid

OZI driver:

* fix axis order issue with georeferencing (3.0 regression)

PDF driver:

* update to use newer versions of pdfium

PDS3 driver:

* report the label in a json:PDS metadata domain

* fix support of Oblique Cylindrical (#1856)

* ix opening of datasets with BSQ organization (or single band), where one band is larger than 2 GB (2.3 regression)

* nasakeywordhandler: fixes to be able to read some labels with metadata items whose value is a list on several lines

* add a GDAL_TRY_PDS3_WITH_VICAR configuration option that can be set to YES so that PDS3 datasets that contain a VICAR label are opened by the VICAR driver

PDS4 driver:

* update CART schema to 1D00_1933 and PDS to 1C00 (#1851)

* fix reading side of Mercator and Orthographic

* add a CREATE_LABEL_ONLY=YES creation option, and create a <Header> element (#1832)

* add write support for LAEA projection

PNG driver:

* Update internal libpng to 1.6.37

PNM driver:

* emit warning if creating file with non-standard extension

RasterLite2 driver:

* fix build against latest librasterlite2, and require it

RMF driver:

* Add support for PZ-90.11 and GSK-2011 coordinate systems

* Read vertical datum info

* Add translation vertical CS ID to dataset's spatial reference

RS2 driver:

* add half-pixel shift to reported GCP line and column numbers (#1666)

Sentinel2 driver:

* Add support for exposing Level-2A AOT, WVP, SCL, CLD, SNW data in SAFE_COMPACT format (#2404)

TileDB driver:

* add capability to define co-registered attributes per band

* set row-major reads and removed adviseread (#1479)

* added support for raster band metadata

* TILEDB_LIBS added for windows build

* partial updates to existing arrays

* support reading tiles in update mode (#2185)

* use array metadata to store xml (#2103)

* redirect vsis3 calls to tiledb s3 direct calls

* Flush cache in block order for global writes (#2391)

TSX driver:

* add PAZ support

USGSDEM driver:

* fix reading FEMA generated Lidar datasets whose header is 918 bytes large

VICAR driver:

* Support FORMAT=HALF, DOUB and COMP

* Support big-endian order for integer & floating point values

* Support VAX floating-point order

* Support BIP and BIL organizations

* Ignore binary label records (NBL), and properly skip binary prefixes (NBB)

* fix reading of EOL labels with non BSQ organizations, and possible confusion with LBLSIZE of EOL overwriting the main one

* improvements in label reading, and report label in a json:VICAR metadata domain

* read binary prefixes as OGR layer

* add read support for BASIC and BASIC2 compression methods

* add write support

* remove obsolete END-OF-DATASET_LABEL, CONVERSION_DETAILS and PIXEL-SHIFT-BUG metadata items that dates back to 1.X era

VRT driver:

* add 'vrt://{gdal_dataset}?bands=num1,...,numN' syntax as a convenient way of creating a on-the-fly VRT with a subset of bands

* VRT warp: do not fail if a block has no corresponding source pixels (#1985)

* VRT Python: also probe libpython3.Xm.so.1.0 (#1660)

* VRT Python: add Python 3.8 compatibility

* avoid erroneous pixel request do be done with KernelFilteredSource

* VRTRawRasterBand: add GetVirtualMemAuto interface to enable mmap io

* fix requesting a downsampling version of the mask band of a source that has masks and overviews

* VRT pansharpening: fix crash when NoData is set and input multispectral bands are separate files (#2328)

* fix IGetDataCoverageStatus() that can cause issue for the use case of https://lists.osgeo.org/pipermail/gdal-dev/2020-April/051889.html

* gdalvrt.xsd: add 'dataAxisToSRSAxisMapping' attribute for GCPList element (#2378)

WCS driver:

* pass user-supplied progress function to curl HTTP request when using DirectRasterIO

WMS driver:

* IReadBlock(): limit number of tiles downloaded at once

* Use curl_multi_wait instead of socket API (#1789)

* AGS: Remove unused parameters from url

== OGR 3.1.0 - Overview of Changes ==

== Core ==

* Support API-level precision/round for geometry WKT (#1797)

* OGR SQL: make LIKE behave in a case sensitive way from now

* OGR SQL: support constructs 'A OR B OR C ... OR N' with many successive OR

* SQL SQLite: do not propagate 'IS / IS NOT value' constructs to OGR SQL

* SQL SQLite: add support for ST_MakeValid() using OGRGeometry::MakeValid() if not exposed by Spatialite already

* Rename swq.h->ogr_swq.h and install it, mark SQL query parse API with

CPL_UNSTABLE_API macro (#1925)

* SQLite dialect: fix issue when using JOIN on a layer without fast filter count capability

* OGRSimpleCurve::getPoints() with XYZM: fix wrong stride used for M array

* OGRSimpleCurve: fix reversePoints() and addSubLineString() to take into account M dimension

* Add OGR_G_CreateGeometryFromEsriJson() and map it to SWIG

* OGRLinearRing::isPointOnRingBoundary(): fix incomplete test that could falsely return true if the point was aligned with a segment, but not between the nodes. Impact correct reconstruction of holes in shapefile driver

* OGRGeometryFactory::ForceTo(): fix crash when forcing a MultiPolygon Z/M/ZM to a CompoundCurve (#2120)

* OGRGeometryFactory::forceToMultiLineString(): fix for a GeometryCollection of LineStringZ/M/ZM (#1873)

* OGRGeometryRebuildCurves(): only try to reconstruct curved geometry if one of the input geometries has really a non-linear portion

* curveFromLineString(): make sure to exactly close the compound curve if the input curve is itself closed

* GDALDataset::CopyLayer(): fix crash when using DST_SRSWKT option. And also set SetAxisMappingStrategy(OAMS_TRADITIONAL_GIS_ORDER)

* Add OGR_G_RemoveLowerDimensionSubGeoms()

== OGRSpatialReference ==

* Revise how SRS methods deal with TOWGS84. Add OSR_ADD_TOWGS84_ON_IMPORT_FROM_EPSG, OSR_ADD_TOWGS84_ON_EXPORT_TO_PROJ4 and OSR_ADD_TOWGS84_ON_EXPORT_TO_WKT1 configuration options. See https://github.com/OSGeo/gdal/commit/cc02dc4397c7ec43ec4c4b842e5faabe16c54904 for details

* SetFromUserInput(): add capability to import PROJJSON

* add ExportToPROJJSON() (PROJ >= 6.2)

* GetAxis(): make it work with a compound CRS (#1604)

* Coordinate transformation: fix invalid output with some input coordinates in EPSG:4326 to EPSG:3857 transformation (3.0 regression)

* Coordinate transformation: Coordinate transformation: add a OGR_CT_OP_SELECTION=PROJ/BEST_ACCURACY/FIRST_MATCHING config option. Defaults to PROJ for PROJ >= 6.3

* importFromWkt(): emit a CPLError() in case of import failure (#1623)

* Add OSRGetAxesCount() to C API and SWIG bindings

* Add OSRPromoteTo3D() and map it to SWIG (PROJ >= 7) (#1852)

* importFromESRI(): acept COMPD_CS (#1881)

* add an internal cache for importFromEPSG() and importFromWkt(). Helps performance for MapServer PROJ6 migration

* Add support for Vertical Perspective projection (#1856)

* Add a OSRGetPROJSearchPaths() function and a SWIG osr.GetPROJVersionMicro()

* Fix use-after-free issue when destroying a OGRSpatialReference object in a thread when another thread has created it but has been destroy in-between

* Coordinate transformation: make it work with hacky WKT1 rotated pole from netCDF driver

* Add OGR_GeomTransformer_XXXX API that wraps OGRGeometryFactory::transformWithOptions() and expose it to SWIG (fixes #1911)

* Fix PROJ usage across fork() calls (#2221)

* OGRCoordinateTransformation: correctly deal when transforming CRS that includes +lon_wrap= or similar qualifiers

* Add OSRIsDerivedGeographic()

* EPSGTreatsAsNorthingEasting(): fix it to properly deal with Polar projected CRS with northing,easting order

== Utilities ==

* ogrinfo: add a -nogeomtype switch

* ogrinfo: fix to output WKT2 SRS by default. Was done correctly for several geometry column, but not single one

* ogr2ogr: emit better error message when using -f VRT

* ogr2ogr: improve performance of -explodecollections on collections with big number of parts

* ogr2ogr: avoid non-relevant warning 'Input datasource uses random layer reading, but output datasource does not support random layer writing' when converting one single layer

* ogr2ogr: allow to combine -nlt CONVERT_TO_LINEAR and -nlt PROMOTE_TO_MULTI (#2338)

* ogr2ogr: add a -makevalid switch (requires GEOS 3.8 or later) and expose it in Python as makeValid=True options of gdal.VectorTranslate()

== OGR drivers ==

Multiple drivers:

* GML/WFS: add support for Date, Time and DateTime fields

* GeoJSONSeq & TopoJSON: avoid false positive detection and errors on unrelated http[s]:// filenames

AmigoCloud driver:

* Modify amigocloud URL endpoint.

AVCE00 driver:

* fix alternance from sequential to per-FID reading

CAD driver:

* Fix read ellipse and arc (#1886)

DXF driver:

* Re-add some header and trailer elements that caused compatibility issues with recent Autocad versions (#1213)

* Generate correct HATCH boundary elliptical arc segments for certain start/end angles

* Skip "Embedded Object" sections in 2018 version DXFs