Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Slow uploads with HTTP/2 #25297

Open
haarp opened this issue Jan 23, 2021 · 6 comments
Open

Slow uploads with HTTP/2 #25297

haarp opened this issue Jan 23, 2021 · 6 comments
Labels
1. to develop bug performance 🚀

Comments

Assignees
No one assigned
Labels
1. to develop bug performance 🚀
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
@aledr @haarp @SimplyCorbett @DoktorHeisenberg @szaimen @Juuuuulie10
@haarp
Copy link

@haarp haarp commented Jan 23, 2021 •
edited
Loading

How to use GitHub

  • Please use the 👍 reaction to show that you are affected by the same issue.
  • Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
  • Subscribe to receive notifications on status change and new comments.

Summary

When HTTP/2 is enabled on the webserver, uploads are very slow, topping out at slightly over 1MByte/s, despite both client and server offering vastly more bandwidth. Once HTTP2 was disabled, upload speeds were improved by a factor of over x10.

All PHP/DB optimizations are in place, and more. The CPU load on server and client remains low, IO load on the DB and storage remains low. Issue is observed during large file uploads, so DB overhead isn't the issue.

This issue was observed by multiple people, e.g. here, here and here. It seems to affect both nginx and Apache.

Steps to reproduce

  1. Enable HTTP/2 on webserver
  2. Upload using a WebDAV client - this test used the Windows 10 client via a mapped network drive
  3. Get uploadin'

Expected behaviour

>10 Mbyte/s

Actual behaviour

1-2 Mbyte/s

Server configuration

Operating system: Debian Linux 10
Web server: Apache 2.4.38-3+deb10u4
Database: MariaDB 10.3.27-0+deb10u1
PHP version: PHP FPM-CGI 7.3.19-1~deb10u1
Nextcloud version: 19.0.7
Updated from an older Nextcloud/ownCloud or fresh install: Updated
Where did you install Nextcloud from: Included updater

Signing status:

Signing status 
No errors have been found.

List of activated apps:

App list 
Enabled:
  - accessibility: 1.5.0
  - activity: 2.12.1
  - calendar: 2.0.4
  - camerarawpreviews: 0.7.10
  - checksum: 0.4.5
  - cloud_federation_api: 1.2.0
  - comments: 1.9.0
  - contacts: 3.4.3
  - dav: 1.15.0
  - duplicatefinder: 0.0.4
  - federatedfilesharing: 1.9.0
  - federation: 1.9.0
  - files: 1.14.0
  - files_3d: 0.3.3
  - files_rightclick: 0.16.0
  - files_sharing: 1.11.0
  - files_trashbin: 1.9.0
  - files_versions: 1.12.0
  - logreader: 2.4.0
  - lookup_server_connector: 1.7.0
  - metadata: 0.12.0
  - nextcloud_announcements: 1.8.0
  - notes: 3.6.4
  - notifications: 2.7.0
  - oauth2: 1.7.0
  - password_policy: 1.9.1
  - photos: 1.1.0
  - privacy: 1.3.0
  - provisioning_api: 1.9.0
  - recommendations: 0.7.0
  - serverinfo: 1.9.0
  - settings: 1.1.0
  - sharebymail: 1.9.0
  - support: 1.2.1
  - survey_client: 1.7.0
  - systemtags: 1.9.0
  - text: 3.0.1
  - theming: 1.10.0
  - twofactor_backupcodes: 1.8.0
  - updatenotification: 1.9.0
  - viewer: 1.3.0
  - workflowengine: 2.1.0
Disabled:
  - admin_audit
  - contactsinteraction
  - encryption
  - files_external
  - files_pdfviewer
  - files_videoplayer
  - firstrunwizard
  - ransomware_protection
  - user_ldap

Nextcloud configuration:

Config report 
{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "overwritewebroot": "\/",
        "htaccess.RewriteBase": "\/",
        "dbtype": "mysql",
        "version": "19.0.7.1",
        "knowledgebaseenabled": false,
        "apps_paths": [
            {
                "path": "\/var\/www\/nextcloud\/apps",
                "url": "\/apps",
                "writable": true
            }
        ],
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "loglevel": 2,
        "mail_smtpmode": "sendmail",
        "theme": "",
        "maintenance": false,
        "trashbin_retention_obligation": "auto",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "session_lifetime": 86400,
        "mysql.utf8mb4": true,
        "updater.release.channel": "stable",
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "preview_max_filesize_image": 100,
        "app_install_overwrite": [
            "checksum"
        ]
    }
}

Are you using external storage, if yes which one: no
Are you using encryption: no
Are you using an external user-backend, if yes which one: no

Client configuration

Browser: -
Client software: Windows 10 WebClient service (Microsoft-WebDAV-MiniRedir/10.0.18363)
Operating system: Windows 10

Logs

Web server error log

Web server error log 
no error logs during upload

Nextcloud log (data/nextcloud.log)

Nextcloud log 
no nextcloud log during upload

Browser log

Browser log 
not using a browser
@aledr
Copy link

@aledr aledr commented Jun 2, 2021

I see no improvement in disabling h2 and h2c.
1.8 MB/sec maximum.

Apache 2.4.43
PHP 7.4.19
Nextcloud 21.0.2

Loading

@szaimen
Copy link
Contributor

@szaimen szaimen commented Jul 7, 2021

Is this Issue still valid in NC21.0.3? If not, please close this issue. Thanks! :)

Loading

@haarp
Copy link
Author

@haarp haarp commented Jul 8, 2021

Unfortunately I can't test currently. Anybody else?

Loading

@SimplyCorbett
Copy link

@SimplyCorbett SimplyCorbett commented Jul 17, 2021

Yes this is still an issue in nextcloud 22. 2MB/s with http2 40MB/s without http2.

Loading

@Juuuuulie10
Copy link

@Juuuuulie10 Juuuuulie10 commented Aug 31, 2021

4MB/s with http2
55MB/s without http2

Loading

@DoktorHeisenberg
Copy link

@DoktorHeisenberg DoktorHeisenberg commented Sep 20, 2021

When disabling http2, I'm getting three times the upload speeds:
2MB/s with http2
6,2 MB/s without http2

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment