Skip to content

Commit

Permalink
Remove legacy-samesite Web Platform Tests
Browse files Browse the repository at this point in the history
SameSite-Lax-by-default and SameSite=None-requires-Secure have been
standardized ([1], [2]) and launched in Chromium [3] as well as
Firefox [4]. The WPTs testing "legacy" behavior are no longer needed,
and they are also failing on wpt.fyi [5].

This change removes the legacy-samesite virtual test suite, as well as
the ?legacy-samesite variant of the tests in wpt/cookie/samesite/. The
test for SameSite=None-requires-Secure also loses its "tentative"
designation.

Chromium will soon no longer support the configuration used in the
virtual test suite, so there would be no way to run these tests anyway.

[1] httpwg/http-extensions#1325
[2] httpwg/http-extensions#1323
[3] https://crrev.com/c/2231445
[4] https://hacks.mozilla.org/2020/08/changes-to-samesite-cookie-behavior/
[5] https://wpt.fyi/results/cookies/samesite?run_id=5099687737556992&run_id=5641857632567296&run_id=5678463840157696&run_id=5638004375814144

Bug: 961439, 1211388
Change-Id: Idb3c835908bcd61dde3593b4fb4f9349e738031f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3047679
Reviewed-by: Steven Bingler <[email protected]>
Commit-Queue: Lily Chen <[email protected]>
Cr-Commit-Position: refs/heads/master@{#905289}
NOKEYCHECK=True
GitOrigin-RevId: a11b13cf0fb388ee424f03814c955052856331ad
  • Loading branch information
chlily1 authored and Copybara-Service committed Jul 26, 2021
1 parent 5fa517c commit 357a438
Show file tree
Hide file tree
Showing 21 changed files with 32 additions and 198 deletions.
46 changes: 0 additions & 46 deletions blink/web_tests/NeverFixTests
Expand Up @@ -1724,52 +1724,6 @@ crbug.com/1085129 virtual/force-eager/external/wpt/measure-memory/idlharness.win
crbug.com/1207146 virtual/force-eager/external/wpt/measure-memory/service-worker.https.any.serviceworker.html [ Skip ]
crbug.com/1207146 virtual/force-eager/external/wpt/measure-memory/shared-worker.https.any.sharedworker.html [ Skip ]

# Legacy SameSite cookie tests do not apply when non-legacy behavior is in effect.
crbug.com/961439 external/wpt/cookies/samesite/fetch.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/form-get-blank-reload.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/form-get-blank.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/form-post-blank-reload.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/form-post-blank.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/iframe-reload.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/iframe.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/img.https.html?legacy-samesite [ Skip ]
crbug.com/1161591 external/wpt/cookies/samesite/multiple-samesite-attributes.https.html?legacy-samesite [ Skip ]
crbug.com/1161591 virtual/schemeful-same-site/external/wpt/cookies/samesite/multiple-samesite-attributes.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/setcookie-lax.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/setcookie-navigation.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/window-open-reload.https.html?legacy-samesite [ Skip ]
crbug.com/961439 external/wpt/cookies/samesite/window-open.https.html?legacy-samesite [ Skip ]

# Unskip the above tests for virtual/legacy-samesite.
virtual/legacy-samesite/external/wpt/cookies/samesite/fetch.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/form-get-blank-reload.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/form-get-blank.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/form-post-blank-reload.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/form-post-blank.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/iframe-reload.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/iframe.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/img.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/multiple-samesite-attributes.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/setcookie-lax.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/setcookie-navigation.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/window-open-reload.https.html?legacy-samesite [ Pass ]
virtual/legacy-samesite/external/wpt/cookies/samesite/window-open.https.html?legacy-samesite [ Pass ]

# Non-legacy SameSite cookie tests do not apply when legacy SameSite behavior is in effect.
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/fetch.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/form-get-blank-reload.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/form-get-blank.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/form-post-blank-reload.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/form-post-blank.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/iframe-reload.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/iframe.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/img.https.html [ Skip ]
crbug.com/1161591 virtual/legacy-samesite/external/wpt/cookies/samesite/multiple-samesite-attributes.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/setcookie-lax.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/setcookie-navigation.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/window-open-reload.https.html [ Skip ]
crbug.com/961439 virtual/legacy-samesite/external/wpt/cookies/samesite/window-open.https.html [ Skip ]

# AppCache tests that need to run only in a virtual test suite
# But have to live under non-virtual http/ to get http/ behavior
crbug.com/582750 http/tests/appcache/* [ Skip ]
Expand Down
5 changes: 0 additions & 5 deletions blink/web_tests/TestExpectations
Expand Up @@ -4582,9 +4582,6 @@ crbug.com/827231 [ Win ] external/wpt/cookie-store/change_eventhandler_for_docum
crbug.com/843945 external/wpt/cookies/samesite/form-get-blank-reload.https.html [ Failure ]
crbug.com/843945 external/wpt/cookies/samesite/form-post-blank-reload.https.html [ Failure ]
crbug.com/843945 external/wpt/cookies/samesite/window-open-reload.https.html [ Failure ]
crbug.com/961745 virtual/legacy-samesite/external/wpt/cookies/samesite/form-get-blank-reload.https.html?legacy-samesite [ Failure ]
crbug.com/961745 virtual/legacy-samesite/external/wpt/cookies/samesite/form-post-blank-reload.https.html?legacy-samesite [ Failure ]
crbug.com/961745 virtual/legacy-samesite/external/wpt/cookies/samesite/window-open-reload.https.html?legacy-samesite [ Failure ]

# When SameSiteByDefaultCookies is enabled (which is the case by default since
# it's one of the experimental-web-platform-features, the Lax-allow-unsafe
Expand All @@ -4595,8 +4592,6 @@ crbug.com/990439 http/tests/cookies/same-site/popup-cross-site-post.https.html [
# Temporarily disable some tests that produce non-deterministic outputs.
crbug.com/1074441 external/wpt/cookies/samesite/iframe-reload.https.html [ Skip ]
crbug.com/1074441 external/wpt/cookies/samesite/iframe.https.html [ Skip ]
crbug.com/1074441 virtual/legacy-samesite/external/wpt/cookies/samesite/iframe-reload.https.html?legacy-samesite [ Skip ]
crbug.com/1074441 virtual/legacy-samesite/external/wpt/cookies/samesite/iframe.https.html?legacy-samesite [ Skip ]
# Flaky Windows-only content_shell crash
crbug.com/1162205 [ Win ] virtual/schemeful-same-site/external/wpt/cookies/attributes/path-redirect.html [ Crash Pass ]
crbug.com/1162205 [ Win ] external/wpt/cookies/attributes/path-redirect.html [ Crash Pass ]
Expand Down
6 changes: 0 additions & 6 deletions blink/web_tests/VirtualTestSuites
Expand Up @@ -692,12 +692,6 @@
"bases": ["fast/forms/color-scheme/select"],
"args": ["--enable-features=UseCommonSelectPopup"]
},
{
"prefix": "legacy-samesite",
"bases": ["external/wpt/cookies/samesite"],
"args": ["--disable-features=SameSiteByDefaultCookies",
"--disable-features=CookiesWithoutSameSiteMustBeSecure"]
},
{
"prefix": "forced-high-contrast-colors",
"bases": ["external/wpt/forced-colors-mode",
Expand Down
8 changes: 0 additions & 8 deletions blink/web_tests/android/WebviewWPTExpectations
Expand Up @@ -707,25 +707,17 @@ crbug.com/1050754 external/wpt/cookies/samesite-none-secure/cookies-without-same
crbug.com/1050754 external/wpt/cookies/samesite/about-blank-subresource.https.html [ Failure Timeout ]
crbug.com/1050754 external/wpt/cookies/samesite/about-blank-toplevel.https.html [ Failure Pass Timeout ]
crbug.com/1050754 external/wpt/cookies/samesite/fetch.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/fetch.https.html?legacy-samesite [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/form-get-blank-reload.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/form-get-blank-reload.https.html?legacy-samesite [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/form-post-blank-reload.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/form-post-blank-reload.https.html?legacy-samesite [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/form-post-blank.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/iframe-reload.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/iframe-reload.https.html?legacy-samesite [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/iframe.document.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/iframe.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/iframe.https.html?legacy-samesite [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/img.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/img.https.html?legacy-samesite [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/sandbox-iframe-nested.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/sandbox-iframe-subresource.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/setcookie-lax.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/setcookie-lax.https.html?legacy-samesite [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/window-open-reload.https.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/samesite/window-open-reload.https.html?legacy-samesite [ Failure ]
crbug.com/1050754 external/wpt/cookies/schemeful-same-site/schemeful-iframe-subresource.tentative.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/schemeful-same-site/schemeful-navigation.tentative.html [ Failure ]
crbug.com/1050754 external/wpt/cookies/schemeful-same-site/schemeful-subresource.tentative.html [ Failure ]
Expand Down
Expand Up @@ -167,32 +167,9 @@ async function resetSameSiteCookies(origin, value) {
}
}

// Given an |expectedStatus| and |expectedValue|, assert the |cookies| contains the
// proper set of cookie names and values, according to the legacy behavior where
// unspecified SameSite attribute defaults to SameSite=None behavior.
function verifySameSiteCookieStateLegacy(expectedStatus, expectedValue, cookies, domCookieStatus) {
assert_equals(cookies["samesite_none"], expectedValue, "SameSite=None cookies are always sent.");
assert_equals(cookies["samesite_unspecified"], expectedValue, "Unspecified-SameSite cookies are always sent.");
if (expectedStatus == SameSiteStatus.CROSS_SITE) {
assert_not_equals(cookies["samesite_strict"], expectedValue, "SameSite=Strict cookies are not sent with cross-site requests.");
assert_not_equals(cookies["samesite_lax"], expectedValue, "SameSite=Lax cookies are not sent with cross-site requests.");
} else if (expectedStatus == SameSiteStatus.LAX) {
assert_not_equals(cookies["samesite_strict"], expectedValue, "SameSite=Strict cookies are not sent with lax requests.");
assert_equals(cookies["samesite_lax"], expectedValue, "SameSite=Lax cookies are sent with lax requests.");
} else if (expectedStatus == SameSiteStatus.STRICT) {
assert_equals(cookies["samesite_strict"], expectedValue, "SameSite=Strict cookies are sent with strict requests.");
assert_equals(cookies["samesite_lax"], expectedValue, "SameSite=Lax cookies are sent with strict requests.");
}

if (cookies["domcookies"]) {
verifyDocumentCookieLegacy(domCookieStatus, expectedValue, cookies["domcookies"]);
}
}

// Same as above except this expects samesite_unspecified to act the same as
// samesite_lax (which is the behavior expected when SameSiteByDefault is
// enabled).
function verifySameSiteCookieStateWithSameSiteByDefault(expectedStatus, expectedValue, cookies, domCookieStatus) {
// Given an |expectedStatus| and |expectedValue|, assert the |cookies| contains
// the proper set of cookie names and values. Expects SameSite-Lax-by-default.
function verifySameSiteCookieState(expectedStatus, expectedValue, cookies, domCookieStatus) {
assert_equals(cookies["samesite_none"], expectedValue, "SameSite=None cookies are always sent.");
if (expectedStatus == SameSiteStatus.CROSS_SITE) {
assert_not_equals(cookies["samesite_strict"], expectedValue, "SameSite=Strict cookies are not sent with cross-site requests.");
Expand All @@ -209,32 +186,11 @@ function verifySameSiteCookieStateWithSameSiteByDefault(expectedStatus, expected
}

if (cookies["domcookies"]) {
verifyDocumentCookieWithSameSiteByDefault(domCookieStatus, expectedValue, cookies["domcookies"]);
}
}

function verifyDocumentCookieLegacy(expectedStatus, expectedValue, domcookies) {
const cookies = domcookies.split(";")
.map(cookie => cookie.trim().split("="))
.reduce((obj, cookie) => {
obj[cookie[0]] = cookie[1];
return obj;
}, {});

if (expectedStatus == DomSameSiteStatus.SAME_SITE) {
assert_equals(cookies["samesite_none"], expectedValue, "SameSite=None cookies are always included in document.cookie.");
assert_equals(cookies["samesite_unspecified"], expectedValue, "Unspecified-SameSite cookies are always included in document.cookie.");
assert_equals(cookies["samesite_strict"], expectedValue, "SameSite=Strict cookies are always included in document.cookie.");
assert_equals(cookies["samesite_lax"], expectedValue, "SameSite=Lax cookies are always included in document.cookie.");
} else if (expectedStatus == DomSameSiteStatus.CROSS_SITE) {
assert_equals(cookies["samesite_none"], expectedValue, "SameSite=None cookies are always included in document.cookie.");
assert_equals(cookies["samesite_unspecified"], expectedValue, "Unspecified-SameSite cookies are always included in document.cookie.");
assert_not_equals(cookies["samesite_strict"], expectedValue, "SameSite=Strict cookies are not included in document.cookie when cross-site.");
assert_not_equals(cookies["samesite_lax"], expectedValue, "SameSite=Lax cookies are not included in document.cookie when cross-site.");
verifyDocumentCookieSameSite(domCookieStatus, expectedValue, cookies['domcookies']);
}
}

function verifyDocumentCookieWithSameSiteByDefault(expectedStatus, expectedValue, domcookies) {
function verifyDocumentCookieSameSite(expectedStatus, expectedValue, domcookies) {
const cookies = domcookies.split(";")
.map(cookie => cookie.trim().split("="))
.reduce((obj, cookie) => {
Expand All @@ -255,16 +211,6 @@ function verifyDocumentCookieWithSameSiteByDefault(expectedStatus, expectedValue
}
}

function isLegacySameSite() {
return location.search === "?legacy-samesite";
}

// Get the proper verifier based on the test's variant type.
function getSameSiteVerifier() {
return isLegacySameSite() ?
verifySameSiteCookieStateLegacy : verifySameSiteCookieStateWithSameSiteByDefault;
}

//
// LeaveSecureCookiesAlone-specific test helpers:
//
Expand Down
@@ -1,8 +1,6 @@
<!DOCTYPE html>
<meta charset="utf-8"/>
<meta name="timeout" content="long">
<meta name="variant" content="">
<meta name="variant" content="?legacy-samesite">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/cookies/resources/cookie-helper.sub.js"></script>
Expand All @@ -15,7 +13,7 @@
return credFetch(target + "/cookies/resources/list.py")

.then(r => r.json())
.then(cookies => getSameSiteVerifier()(expectedStatus, value, cookies, DomSameSiteStatus.SAME_SITE));
.then(cookies => verifySameSiteCookieState(expectedStatus, value, cookies, DomSameSiteStatus.SAME_SITE));
});
}, title);
}
Expand Down
@@ -1,7 +1,5 @@
<!DOCTYPE html>
<meta charset="utf-8"/>
<meta name="variant" content="">
<meta name="variant" content="?legacy-samesite">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/cookies/resources/cookie-helper.sub.js"></script>
Expand Down Expand Up @@ -30,7 +28,7 @@
var reloaded = false;
var msgHandler = e => {
try {
getSameSiteVerifier()(expectedStatus, value, e.data, DomSameSiteStatus.SAME_SITE);
verifySameSiteCookieState(expectedStatus, value, e.data, DomSameSiteStatus.SAME_SITE);
} catch (e) {
reject(e);
}
Expand Down
@@ -1,8 +1,6 @@
<!DOCTYPE html>
<meta charset="utf-8"/>
<meta name="timeout" content="long">
<meta name="variant" content="">
<meta name="variant" content="?legacy-samesite">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/cookies/resources/cookie-helper.sub.js"></script>
Expand Down Expand Up @@ -33,7 +31,7 @@
window.removeEventListener("message", msgHandler);
e.source.close();
try {
getSameSiteVerifier()(expectedStatus, value, e.data, DomSameSiteStatus.SAME_SITE);
verifySameSiteCookieState(expectedStatus, value, e.data, DomSameSiteStatus.SAME_SITE);
resolve("Popup received the cookie.");
} catch (e) {
reject(e);
Expand Down
@@ -1,7 +1,5 @@
<!DOCTYPE html>
<meta charset="utf-8"/>
<meta name="variant" content="">
<meta name="variant" content="?legacy-samesite">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/cookies/resources/cookie-helper.sub.js"></script>
Expand All @@ -20,7 +18,7 @@
var reloaded = false;
var msgHandler = e => {
try {
getSameSiteVerifier()(expectedStatus, value, e.data, DomSameSiteStatus.SAME_SITE);
verifySameSiteCookieState(expectedStatus, value, e.data, DomSameSiteStatus.SAME_SITE);
} catch (e) {
reject(e);
}
Expand Down
@@ -1,8 +1,6 @@
<!DOCTYPE html>
<meta charset="utf-8"/>
<meta name="timeout" content="long">
<meta name="variant" content="">
<meta name="variant" content="?legacy-samesite">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/cookies/resources/cookie-helper.sub.js"></script>
Expand All @@ -22,7 +20,7 @@
window.removeEventListener("message", msgHandler);
e.source.close();
try {
getSameSiteVerifier()(expectedStatus, value, e.data, DomSameSiteStatus.SAME_SITE);
verifySameSiteCookieState(expectedStatus, value, e.data, DomSameSiteStatus.SAME_SITE);
resolve("Popup received the cookie.");
} catch (e) {
reject(e);
Expand Down
@@ -1,8 +1,6 @@
<!DOCTYPE html>
<meta charset="utf-8"/>
<meta name="timeout" content="long">
<meta name="variant" content="">
<meta name="variant" content="?legacy-samesite">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/cookies/resources/cookie-helper.sub.js"></script>
Expand All @@ -21,7 +19,7 @@
var reloaded = false;
var msgHandler = e => {
try {
getSameSiteVerifier()(expectedStatus, value, e.data, expectedDomStatus);
verifySameSiteCookieState(expectedStatus, value, e.data, expectedDomStatus);
} catch (e) {
reject(e);
}
Expand Down
@@ -1,8 +1,6 @@
<!DOCTYPE html>
<meta charset="utf-8"/>
<meta name="timeout" content="long">
<meta name="variant" content="">
<meta name="variant" content="?legacy-samesite">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/cookies/resources/cookie-helper.sub.js"></script>
Expand All @@ -24,7 +22,7 @@
document.body.removeChild(iframe);
window.removeEventListener("message", msgHandler);
try {
getSameSiteVerifier()(expectedStatus, value, e.data, expectedDomStatus);
verifySameSiteCookieState(expectedStatus, value, e.data, expectedDomStatus);
resolve();
} catch(e) {
reject(e);
Expand Down

0 comments on commit 357a438

Please sign in to comment.