Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption (CVE-2023-24536) #59153

Closed
neild opened this issue Mar 20, 2023 · 19 comments
Assignees
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker Security
Milestone

Comments

@neild
Copy link
Contributor

neild commented Mar 20, 2023

Multipart form parsing can consume large amounts of CPU and memory when processing form inputs containing very large numbers of parts. This stems from several causes:

mime/multipart.Reader.ReadForm limits the total memory a parsed multipart form can consume. ReadForm could undercount the amount of memory consumed, leading it to accept larger inputs than intended.
Limiting total memory does not account for increased pressure on the garbage collector from large numbers of small allocations in forms with many parts.
ReadForm could allocate a large number of short-lived buffers, further increasing pressure on the garbage collector.
The combination of these factors can permit an attacker to cause an program that parses multipart forms to consume large amounts of CPU and memory, potentially resulting in a denial of service. This affects programs that use mime/multipart.Reader.ReadForm, as well as form parsing in the net/http package with the Request methods FormFile, FormValue, ParseMultipartForm, and PostFormValue.

ReadForm now does a better job of estimating the memory consumption of parsed forms, and performs many fewer short-lived allocations.

In addition, mime/multipart.Reader now imposes the following limits on the size of parsed forms:

Forms parsed with ReadForm may contain no more than 1000 parts. This limit may be adjusted with the environment variable GODEBUG=multipartmaxparts=.
Form parts parsed with NextPart and NextRawPart may contain no more than 10,000 header fields. In addition, forms parsed with ReadForm may contain no more than 10,000 header fields across all parts. This limit may be adjusted with the environment variable GODEBUG=multipartmaxheaders=.
Thanks to Jakob Ackermann (@das7pad) for discovering this issue.

This is CVE-2023-24536 and Go issue https://go.dev/issue/59153.

/cc @golang/security and @golang/release

@neild neild added Security NeedsFix The path to resolution is known, but the work has not been done. labels Mar 20, 2023
@neild neild self-assigned this Mar 20, 2023
@dmitshur dmitshur added this to the Go1.21 milestone Mar 20, 2023
@julieqiu
Copy link
Member

@gopherbot please open backport issues.

@gopherbot
Copy link
Contributor

Backport issue(s) opened: #59269 (for 1.19), #59270 (for 1.20).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/481979 mentions this issue: [release-branch.go1.19] mime/multipart: limit parsed mime message sizes

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/481977 mentions this issue: [release-branch.go1.19] mime/multipart: avoid excessive copy buffer allocations in ReadForm

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/481978 mentions this issue: [release-branch.go1.19] net/textproto, mime/multipart: improve accounting of non-file data

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/481985 mentions this issue: [release-branch.go1.19] mime/multipart: limit parsed mime message sizes

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/481983 mentions this issue: [release-branch.go1.19] mime/multipart: avoid excessive copy buffer allocations in ReadForm

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/481984 mentions this issue: [release-branch.go1.19] net/textproto, mime/multipart: improve accounting of non-file data

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/482077 mentions this issue: mime/multipart: limit parsed mime message sizes

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/482075 mentions this issue: mime/multipart: avoid excessive copy buffer allocations in ReadForm

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/482076 mentions this issue: net/textproto, mime/multipart: improve accounting of non-file data

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/481989 mentions this issue: [release-branch.go1.20] mime/multipart: avoid excessive copy buffer allocations in ReadForm

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/481990 mentions this issue: [release-branch.go1.20] net/textproto, mime/multipart: improve accounting of non-file data

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/481991 mentions this issue: [release-branch.go1.20] mime/multipart: limit parsed mime message sizes

gopherbot pushed a commit that referenced this issue Apr 4, 2023
…llocations in ReadForm

When copying form data to disk with io.Copy,
allocate only one copy buffer and reuse it rather than
creating two buffers per file (one from io.multiReader.WriteTo,
and a second one from os.File.ReadFrom).

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59269

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802453
Run-TryBot: Damien Neil <[email protected]>
Reviewed-by: Julie Qiu <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802395
Run-TryBot: Roland Shoemaker <[email protected]>
Reviewed-by: Damien Neil <[email protected]>
Change-Id: Ie405470c92abffed3356913b37d813e982c96c8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/481983
Run-TryBot: Michael Knyszek <[email protected]>
TryBot-Result: Gopher Robot <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
Reviewed-by: Matthew Dempsky <[email protected]>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
…ting of non-file data

For requests containing large numbers of small parts,
memory consumption of a parsed form could be about 250%
over the estimated size.

When considering the size of parsed forms, account for the size of
FileHeader structs and increase the estimate of memory consumed by
map entries.

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59269

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802454
Run-TryBot: Damien Neil <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
Reviewed-by: Julie Qiu <[email protected]>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802396
Run-TryBot: Roland Shoemaker <[email protected]>
Reviewed-by: Damien Neil <[email protected]>
Change-Id: I31bc50e9346b4eee6fbe51a18c3c57230cc066db
Reviewed-on: https://go-review.googlesource.com/c/go/+/481984
Reviewed-by: Matthew Dempsky <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
TryBot-Result: Gopher Robot <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
The parsed forms of MIME headers and multipart forms can consume
substantially more memory than the size of the input data.
A malicious input containing a very large number of headers or
form parts can cause excessively large memory allocations.

Set limits on the size of MIME data:

Reader.NextPart and Reader.NextRawPart limit the the number
of headers in a part to 10000.

Reader.ReadForm limits the total number of headers in all
FileHeaders to 10000.

Both of these limits may be set with with
GODEBUG=multipartmaxheaders=<values>.

Reader.ReadForm limits the number of parts in a form to 1000.
This limit may be set with GODEBUG=multipartmaxparts=<value>.

Thanks for Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59269

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802455
Run-TryBot: Damien Neil <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
Reviewed-by: Julie Qiu <[email protected]>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1801087
Reviewed-by: Damien Neil <[email protected]>
Run-TryBot: Roland Shoemaker <[email protected]>
Change-Id: If134890d75f0d95c681d67234daf191ba08e6424
Reviewed-on: https://go-review.googlesource.com/c/go/+/481985
Run-TryBot: Michael Knyszek <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
TryBot-Result: Gopher Robot <[email protected]>
Reviewed-by: Matthew Dempsky <[email protected]>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
…llocations in ReadForm

When copying form data to disk with io.Copy,
allocate only one copy buffer and reuse it rather than
creating two buffers per file (one from io.multiReader.WriteTo,
and a second one from os.File.ReadFrom).

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59270

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802453
Run-TryBot: Damien Neil <[email protected]>
Reviewed-by: Julie Qiu <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
Change-Id: I44ef17c4b4964cdac2858317275594194801fee3
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802398
Run-TryBot: Roland Shoemaker <[email protected]>
Reviewed-on: https://go-review.googlesource.com/c/go/+/481989
Auto-Submit: Michael Knyszek <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
TryBot-Result: Gopher Robot <[email protected]>
Reviewed-by: Matthew Dempsky <[email protected]>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
…ting of non-file data

For requests containing large numbers of small parts,
memory consumption of a parsed form could be about 250%
over the estimated size.

When considering the size of parsed forms, account for the size of
FileHeader structs and increase the estimate of memory consumed by
map entries.

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59270

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802454
Run-TryBot: Damien Neil <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
Reviewed-by: Julie Qiu <[email protected]>
Change-Id: I9753aa1f8a1b1479c160f870def3b7081b6847ac
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802399
TryBot-Result: Security TryBots <[email protected]>
Run-TryBot: Roland Shoemaker <[email protected]>
Reviewed-on: https://go-review.googlesource.com/c/go/+/481990
TryBot-Bypass: Michael Knyszek <[email protected]>
Reviewed-by: Matthew Dempsky <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
The parsed forms of MIME headers and multipart forms can consume
substantially more memory than the size of the input data.
A malicious input containing a very large number of headers or
form parts can cause excessively large memory allocations.

Set limits on the size of MIME data:

Reader.NextPart and Reader.NextRawPart limit the the number
of headers in a part to 10000.

Reader.ReadForm limits the total number of headers in all
FileHeaders to 10000.

Both of these limits may be set with with
GODEBUG=multipartmaxheaders=<values>.

Reader.ReadForm limits the number of parts in a form to 1000.
This limit may be set with GODEBUG=multipartmaxparts=<value>.

Thanks for Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59270

Change-Id: I36ddceead7f8292c327286fd8694e6113d3b4977
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802455
Run-TryBot: Damien Neil <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
Reviewed-by: Julie Qiu <[email protected]>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802608
Run-TryBot: Roland Shoemaker <[email protected]>
Reviewed-on: https://go-review.googlesource.com/c/go/+/481991
Run-TryBot: Michael Knyszek <[email protected]>
Reviewed-by: Matthew Dempsky <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
TryBot-Bypass: Michael Knyszek <[email protected]>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
When copying form data to disk with io.Copy,
allocate only one copy buffer and reuse it rather than
creating two buffers per file (one from io.multiReader.WriteTo,
and a second one from os.File.ReadFrom).

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802453
Run-TryBot: Damien Neil <[email protected]>
Reviewed-by: Julie Qiu <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
Change-Id: I732bd2e1e7467918cac8ab9d65d089272ba4656f
Reviewed-on: https://go-review.googlesource.com/c/go/+/482075
Auto-Submit: Michael Knyszek <[email protected]>
Reviewed-by: Matthew Dempsky <[email protected]>
TryBot-Bypass: Michael Knyszek <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
For requests containing large numbers of small parts,
memory consumption of a parsed form could be about 250%
over the estimated size.

When considering the size of parsed forms, account for the size of
FileHeader structs and increase the estimate of memory consumed by
map entries.

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802454
Run-TryBot: Damien Neil <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
Reviewed-by: Julie Qiu <[email protected]>
Change-Id: I9620758495ed77c09ca6dc5db4b723c29f3baad8
Reviewed-on: https://go-review.googlesource.com/c/go/+/482076
TryBot-Result: Gopher Robot <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
Reviewed-by: Matthew Dempsky <[email protected]>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
The parsed forms of MIME headers and multipart forms can consume
substantially more memory than the size of the input data.
A malicious input containing a very large number of headers or
form parts can cause excessively large memory allocations.

Set limits on the size of MIME data:

Reader.NextPart and Reader.NextRawPart limit the the number
of headers in a part to 10000.

Reader.ReadForm limits the total number of headers in all
FileHeaders to 10000.

Both of these limits may be set with with
GODEBUG=multipartmaxheaders=<values>.

Reader.ReadForm limits the number of parts in a form to 1000.
This limit may be set with GODEBUG=multipartmaxparts=<value>.

Thanks for Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802455
Run-TryBot: Damien Neil <[email protected]>
Reviewed-by: Roland Shoemaker <[email protected]>
Reviewed-by: Julie Qiu <[email protected]>
Change-Id: I08dd297bd75724aade4b0bd6a7d19aeca5bbf99f
Reviewed-on: https://go-review.googlesource.com/c/go/+/482077
Run-TryBot: Michael Knyszek <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
TryBot-Result: Gopher Robot <[email protected]>
Reviewed-by: Matthew Dempsky <[email protected]>
@mknyszek mknyszek changed the title security: fix CVE-2023-24536 net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption (CVE-2023-24536) Apr 4, 2023
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/482238 mentions this issue: html/template,mime/multipart: document new GODEBUG settings

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/482535 mentions this issue: [release-branch.go1.19] html/template,mime/multipart: document new GODEBUG settings

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/482555 mentions this issue: [release-branch.go1.20] html/template,mime/multipart: document new GODEBUG settings

gopherbot pushed a commit that referenced this issue Apr 5, 2023
This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.

Updates #59153.
Updates #59234.

Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482238
Auto-Submit: Michael Knyszek <[email protected]>
Reviewed-by: Ian Lance Taylor <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
Reviewed-by: Damien Neil <[email protected]>
TryBot-Result: Gopher Robot <[email protected]>
@mknyszek
Copy link
Contributor

mknyszek commented Apr 5, 2023

I believe this has been resolved. (The CL just didn't have "Fixes" in the commit message.)

@neild please feel free to reopen if I misunderstood.

@mknyszek mknyszek closed this as completed Apr 5, 2023
gopherbot pushed a commit that referenced this issue Apr 5, 2023
…DEBUG settings

This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.

Updates #59153.
For #59270.
Updates #59234.
For #59272.

Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482555
TryBot-Result: Gopher Robot <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
Reviewed-by: Damien Neil <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
gopherbot pushed a commit that referenced this issue Apr 5, 2023
…DEBUG settings

This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.

Updates #59153.
For #59269.
Updates #59234.
For #59271.

Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482535
TryBot-Result: Gopher Robot <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
Reviewed-by: Damien Neil <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
@bisonliao
Copy link

I use go 1.15,how should I do ? upgrade my go to 1.21? It will be a horrible mission. Can I just upgrade net package and mime package? thank you.

bradfitz pushed a commit to tailscale/go that referenced this issue May 25, 2023
…DEBUG settings

This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.

Updates golang#59153.
For golang#59270.
Updates golang#59234.
For golang#59272.

Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482555
TryBot-Result: Gopher Robot <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
Reviewed-by: Damien Neil <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
bradfitz pushed a commit to tailscale/go that referenced this issue May 25, 2023
…DEBUG settings

This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.

Updates golang#59153.
For golang#59270.
Updates golang#59234.
For golang#59272.

Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482555
TryBot-Result: Gopher Robot <[email protected]>
Auto-Submit: Michael Knyszek <[email protected]>
Reviewed-by: Damien Neil <[email protected]>
Run-TryBot: Michael Knyszek <[email protected]>
@golang golang locked and limited conversation to collaborators Apr 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker Security
Projects
None yet
Development

No branches or pull requests

6 participants