Skip to content

3.0.0
Compare
Choose a tag to compare
@dataproc-robot dataproc-robot released this 20 Dec 10:08
· 9 commits to branch-3.0.x since this release
v3.0.0
667bf17
  1. Remove Hadoop 2.x support.
  2. Update all dependencies to the latest versions.
  3. Add support for downscoped tokens in AccessTokenProvider.
  4. Implement FileSystem.openFile to take advantage of the FileStatus if
    passed.
  5. Remove an obsolete AuthorizationHandler and related properties: 
    fs.gs.authorization.handler.impl
fs.gs.authorization.handler.properties.<AUTHORIZATION_HANDLER_PROPERTY>
  6. Remove support for Apache HTTP transport and related property: 
    fs.gs.http.transport.type
  7. Support GCS fine-grained action in AuthorizationHandlers.
  8. Decrease log level for hflush rate limit log message.
  9. Remove Cooperative Locking support for directory operations and related
    properties: 
    fs.gs.cooperative.locking.enable
fs.gs.cooperative.locking.expiration.timeout.ms
fs.gs.cooperative.locking.max.concurrent.operations
  10. Migrate authentication to com.google.auth.Credentials and remove obsolete
    properties: 
    fs.gs.auth.service.account.email
fs.gs.auth.service.account.keyfile
fs.gs.auth.service.account.private.key
fs.gs.auth.service.account.private.key.id
  11. Refactor authentication configuration to use an explicit fs.gs.auth.type
    enum property, instead of relying on inference of the authentication type
    based on the set configuration properties, and remove obsolete properties: 
    fs.gs.auth.null.enable
fs.gs.auth.service.account.enable
  12. Add support for a new USER_CREDENTIALS authentication type that retrieves
    a refresh token using the authorisation code grant flow configured via the
    following properties: 
    fs.gs.auth.client.id
fs.gs.auth.client.secret
fs.gs.auth.refresh.token
  13. Merge all output stream types functionality in the default output stream
    that behaves similarly to the FLUSHABLE_COMPOSITE stream, and remove
    obsolete fs.gs.outputstream.type property.
  14. Set default value for fs.gs.list.max.items.per.call property to 5000.
  15. Set socket read timeout (fs.gs.http.read-timeout) as early as possible on
    new sockets returned from the custom SSLSocketFactory. This guarantees the
    timeout is enforced during TLS handshakes when using Conscrypt as the
    security provider.
  16. The Google Cloud Storage Connector now can be used as a
    Hadoop Credential Provider.
  17. Added dependency on the Cloud Storage Client Library
    (google-cloud-storage).
  18. Rename fs.gs.rewrite.max.bytes.per.call property to
    fs.gs.rewrite.max.chunk.size.
  19. Remove support of the deprecated fs.gs.io.buffersize.write property.
  20. Add support for size suffixes (k, m, g, etc) in values of size-related
    properties: 
    fs.gs.inputstream.inplace.seek.limit
fs.gs.inputstream.min.range.request.size
fs.gs.outputstream.buffer.size
fs.gs.outputstream.pipe.buffer.size
fs.gs.outputstream.upload.cache.size
fs.gs.outputstream.upload.chunk.size
fs.gs.rewrite.max.chunk.size
  21. Remove .ms suffix from names and add support for time suffixes (ms, s,
    m, etc) in values of time-related properties: 
    fs.gs.http.connect-timeout
fs.gs.http.read-timeout
fs.gs.max.wait.for.empty.object.creation
fs.gs.outputstream.sync.min.interval
fs.gs.performance.cache.max.entry.age
  22. Change default values of properties: 
    fs.gs.http.connect-timeout (default: 20s -> 5s)
fs.gs.http.read-timeout (default: 20s -> 5s)
fs.gs.outputstream.upload.chunk.size (default: 64m -> 24m)
  23. Upgrade Hadoop to 3.3.5.
  24. Upgrade java-storage to 2.25.0
  25. Add support for WORKLOAD_IDENTITY_FEDERATION_CREDENTIAL_CONFIG_FILE authentication type that retrieves a refresh token using workload identity federation configuraiton defined in: fs.gs.auth.workload.identity.federation.credential.config.file
Assets 5