2018-08-10 (GCS 1.6.8, BQ 0.10.9)
Changelog
Cloud Storage connector:
-
Support parallel execution of GCS batch requests.
Number of threads to execute batch requests configurable via property:
fs.gs.batch.threads (default: 0)
If
fs.gs.batch.threads
value is set to 0 then batch requests will be executed sequentially by caller thread. -
Do not send batch request when performing operations (rename, delete, copy) on 1 object.
-
Add configuration properties to control batching of copy operations separately from other operations:
fs.gs.copy.max.requests.per.batch (default: 30) fs.gs.copy.batch.threads (default: 0)
-
Fix
RejectedExecutionException
during parallel execution of GCS batch requests. -
Change default values for GCS batch/directory operations properties:
fs.gs.copy.with.rewrite.enable (default: false -> true) fs.gs.copy.max.requests.per.batch (default: 30 -> 1) fs.gs.copy.batch.threads (default: 0 -> 50) fs.gs.max.requests.per.batch (default: 30 -> 25) fs.gs.batch.threads (default: 0 -> 25)
BigQuery connector:
- POM updates for GCS connector 1.6.8.