Skip to content

2018-09-20 (GCS 1.9.7, BQ 0.13.7)

Compare
Choose a tag to compare
@medb medb released this 20 Sep 19:36
· 872 commits to master since this release

Changelog

Cloud Storage connector:

  1. Add an option to provide credentials directly in Hadoop Configuration, without having to place a file on every node, or associating service accounts with GCE VMs.

    fs.gs.auth.service.account.private.key.id
    fs.gs.auth.service.account.private.key
    
  2. Add an option to specify max bytes rewritten per rewrite request when fs.gs.copy.with.rewrite.enable is set to true:

    fs.gs.rewrite.max.bytes.per.call (default: 536870912)
    

    Even though GCS does not require this parameter for rewrite requests, rewrite requests are flaky without it.

BigQuery connector:

  1. POM updates for GCS connector 1.9.7.