Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Cocoa] Upstream support for system context retrieval through UIIntelligenceSupport #29843

Merged
merged 1 commit into from
Jun 15, 2024
Merged

[Cocoa] Upstream support for system context retrieval through UIIntelligenceSupport #29843

merged 1 commit into from
Jun 15, 2024

Conversation

whsieh
Copy link
Member

@whsieh whsieh commented Jun 14, 2024 •
edited by webkit-early-warning-system

66195e9

[Cocoa] Upstream support for system context retrieval through UIIntelligenceSupport
https://bugs.webkit.org/show_bug.cgi?id=275510

Reviewed by Aditya Keerthi and Megan Gardner.

Upstream support for system context retrieval through the UIIntelligenceSupport framework on iOS 18
and macOS Sequoia. See below for more comments.

* LayoutTests/fast/text-extraction/basic-text-extraction.html:
* LayoutTests/fast/text-extraction/text-extraction-when-scrolled.html:

Add an explicit `textExtractionEnabled=true` to enable use of the text extraction SPI.

* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WTF/wtf/PlatformHave.h:

Add a new compile-time flag: `HAVE(UIINTELLIGENCESUPPORT_FRAMEWORK)`. Use this flag to enable the
runtime flag `TextExtractionEnabled` by default on supported platforms.

* Source/WebKit/Configurations/WebKitSwift.xcconfig:

Remove `WKWebView+TextExtraction.swift` as an excluded source, now that the file is no longer in
WebKitAdditions.

* Source/WebKit/DerivedSources-input.xcfilelist:
* Source/WebKit/DerivedSources.make:
* Source/WebKit/UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::didCommitLoadForFrame):

Remove an unnecessary compile-time flag, `ENABLE(TEXT_EXTRACTION)`. This was only used to avoid
eagerly loading `libWebKitSwift` on platforms where support for UIIntelligenceSupport is missing;
it's a bit cleaner to just turn the runtime feature off by default when the above compile-time flag,
`HAVE(UIINTELLIGENCESUPPORT_FRAMEWORK)`, is off.

* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebKitSwift/TextExtraction/WKWebView+TextExtraction.swift: Added.

Move this source file to `WebKitSwift/`, from WebKitAdditions.

(createEditable(for:)):
(createElementContent(for:)):
(createIntelligenceElement(_:)):
(WKWebView._intelligenceBaseClass):
(WKWebView._intelligenceCollectContent(in:collector:)):
(WKWebView._intelligenceCollectRemoteContent(in:remoteContextWrapper:)):
(WKWebView._requestTextExtraction(in:completionHandler:)):

Upstream support code for converting `WKTextExtractionItem` → `IntelligenceElement`.

* Tools/WebKitTestRunner/TestOptions.cpp:
(WTR::TestOptions::defaults):
(WTR::TestOptions::keyTypeMapping):
* Tools/WebKitTestRunner/TestOptions.h:
(WTR::TestOptions::textExtractionEnabled const):
* Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm:
(WTR::TestController::cocoaResetStateToConsistentValues):

Reintroduce a test option to enable TextExtraction, since enablement is now properly gated on
`HAVE(UIINTELLIGENCESUPPORT_FRAMEWORK)`. Prior to this upstreaming, it was unconditionally (and
unnecessarily) enabled everywhere.

Canonical link: https://commits.webkit.org/280051@main

c8750d9

Misc iOS, tvOS & watchOS macOS Linux Windows
✅ 🧪 style ✅ 🛠 ios ✅ 🛠 mac ✅ 🛠 wpe ✅ 🛠 wincairo
✅ 🧪 bindings ✅ 🛠 ios-sim ✅ 🛠 mac-AS-debug ✅ 🧪 wpe-wk2 ✅ 🧪 wincairo-tests
✅ 🧪 webkitperl ❌ 🧪 ios-wk2 ✅ 🧪 api-mac ✅ 🧪 api-wpe
✅ 🧪 ios-wk2-wpt ✅ 🧪 mac-wk1 ✅ 🛠 wpe-cairo
✅ 🛠 🧪 jsc ✅ 🧪 api-ios ✅ 🧪 mac-wk2 ✅ 🛠 gtk
✅ 🛠 🧪 jsc-arm64 ✅ 🛠 tv ✅ 🧪 mac-AS-debug-wk2 ✅ 🧪 gtk-wk2
✅ 🛠 tv-sim ✅ 🧪 mac-wk2-stress ✅ 🧪 api-gtk
✅ 🛠 🧪 merge ✅ 🛠 watch   🛠 jsc-armv7
✅ 🛠 watch-sim   🧪 jsc-armv7-tests

@whsieh whsieh self-assigned this Jun 14, 2024
@whsieh whsieh added the Platform Portability improvements and other general platform improvements not driven directly by site bugs. label Jun 14, 2024
@webkit-early-warning-system
Copy link
Collaborator

EWS run on previous version of this PR (hash 7e788bb)

Misc iOS, tvOS & watchOS macOS Linux Windows
✅ 🧪 style ✅ 🛠 ios ✅ 🛠 mac ❌ 🛠 wpe   🛠 wincairo
✅ 🛠 ios-sim ✅ 🛠 mac-AS-debug ❌ 🧪 wpe-wk2   🧪 wincairo-tests
✅ 🧪 webkitperl   🧪 ios-wk2   🧪 api-mac ❌ 🧪 api-wpe
✅ 🧪 ios-wk2-wpt   🧪 mac-wk1   🛠 wpe-cairo
  🛠 🧪 jsc ⏳ 🧪 api-ios   🧪 mac-wk2 ❌ 🛠 gtk
✅ 🛠 🧪 jsc-arm64 ✅ 🛠 tv   🧪 mac-AS-debug-wk2 ❌ 🧪 gtk-wk2
  🛠 tv-sim   🧪 mac-wk2-stress ❌ 🧪 api-gtk
  🛠 watch   🛠 jsc-armv7
  🛠 watch-sim   🧪 jsc-armv7-tests

@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Jun 14, 2024
@whsieh whsieh removed the merging-blocked Applied to prevent a change from being merged label Jun 14, 2024
@webkit-early-warning-system
Copy link
Collaborator

EWS run on previous version of this PR (hash 99e6bd8)

Misc iOS, tvOS & watchOS macOS Linux Windows
✅ 🧪 style ✅ 🛠 ios ✅ 🛠 mac ✅ 🛠 wpe ✅ 🛠 wincairo
✅ 🛠 ios-sim ✅ 🛠 mac-AS-debug   🧪 wpe-wk2   🧪 wincairo-tests
✅ 🧪 webkitperl ❌ 🧪 ios-wk2 ✅ 🧪 api-mac ✅ 🧪 api-wpe
✅ 🧪 ios-wk2-wpt ✅ 🧪 mac-wk1 ✅ 🛠 wpe-cairo
✅ 🛠 🧪 jsc ⏳ 🧪 api-ios ❌ 🧪 mac-wk2 ✅ 🛠 gtk
✅ 🛠 🧪 jsc-arm64 ✅ 🛠 tv   🧪 mac-AS-debug-wk2   🧪 gtk-wk2
  🛠 tv-sim ✅ 🧪 mac-wk2-stress ⏳ 🧪 api-gtk
✅ 🛠 watch   🛠 jsc-armv7
  🛠 watch-sim   🧪 jsc-armv7-tests

@whsieh
Copy link
Member Author

whsieh commented Jun 15, 2024 •
edited

Oh, looks like I've inadvertently disabled text extraction when running layout tests on downlevel OSes. I'll need to restore that part (by reintroducing a TestOption to enable text extraction APIs).

Thanks for the reviews — will fix these tests and reupload..

@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Jun 15, 2024
@whsieh whsieh removed the merging-blocked Applied to prevent a change from being merged label Jun 15, 2024
@webkit-early-warning-system
Copy link
Collaborator

EWS run on current version of this PR (hash c8750d9)

Misc iOS, tvOS & watchOS macOS Linux Windows
✅ 🧪 style ✅ 🛠 ios ✅ 🛠 mac ✅ 🛠 wpe ✅ 🛠 wincairo
✅ 🧪 bindings ✅ 🛠 ios-sim ✅ 🛠 mac-AS-debug ✅ 🧪 wpe-wk2 ✅ 🧪 wincairo-tests
✅ 🧪 webkitperl ❌ 🧪 ios-wk2 ✅ 🧪 api-mac ✅ 🧪 api-wpe
✅ 🧪 ios-wk2-wpt ✅ 🧪 mac-wk1 ✅ 🛠 wpe-cairo
✅ 🛠 🧪 jsc ✅ 🧪 api-ios ✅ 🧪 mac-wk2 ✅ 🛠 gtk
✅ 🛠 🧪 jsc-arm64 ✅ 🛠 tv ✅ 🧪 mac-AS-debug-wk2 ✅ 🧪 gtk-wk2
✅ 🛠 tv-sim ✅ 🧪 mac-wk2-stress ✅ 🧪 api-gtk
✅ 🛠 🧪 merge ✅ 🛠 watch   🛠 jsc-armv7
✅ 🛠 watch-sim   🧪 jsc-armv7-tests

@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Jun 15, 2024
@whsieh whsieh added merge-queue Applied to send a pull request to merge-queue and removed merging-blocked Applied to prevent a change from being merged labels Jun 15, 2024
@whsieh
[Cocoa] Upstream support for system context retrieval through UIIntel…
66195e9 
…ligenceSupport

https://bugs.webkit.org/show_bug.cgi?id=275510

Reviewed by Aditya Keerthi and Megan Gardner.

Upstream support for system context retrieval through the UIIntelligenceSupport framework on iOS 18
and macOS Sequoia. See below for more comments.

* LayoutTests/fast/text-extraction/basic-text-extraction.html:
* LayoutTests/fast/text-extraction/text-extraction-when-scrolled.html:

Add an explicit `textExtractionEnabled=true` to enable use of the text extraction SPI.

* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WTF/wtf/PlatformHave.h:

Add a new compile-time flag: `HAVE(UIINTELLIGENCESUPPORT_FRAMEWORK)`. Use this flag to enable the
runtime flag `TextExtractionEnabled` by default on supported platforms.

* Source/WebKit/Configurations/WebKitSwift.xcconfig:

Remove `WKWebView+TextExtraction.swift` as an excluded source, now that the file is no longer in
WebKitAdditions.

* Source/WebKit/DerivedSources-input.xcfilelist:
* Source/WebKit/DerivedSources.make:
* Source/WebKit/UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::didCommitLoadForFrame):

Remove an unnecessary compile-time flag, `ENABLE(TEXT_EXTRACTION)`. This was only used to avoid
eagerly loading `libWebKitSwift` on platforms where support for UIIntelligenceSupport is missing;
it's a bit cleaner to just turn the runtime feature off by default when the above compile-time flag,
`HAVE(UIINTELLIGENCESUPPORT_FRAMEWORK)`, is off.

* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebKitSwift/TextExtraction/WKWebView+TextExtraction.swift: Added.

Move this source file to `WebKitSwift/`, from WebKitAdditions.

(createEditable(for:)):
(createElementContent(for:)):
(createIntelligenceElement(_:)):
(WKWebView._intelligenceBaseClass):
(WKWebView._intelligenceCollectContent(in:collector:)):
(WKWebView._intelligenceCollectRemoteContent(in:remoteContextWrapper:)):
(WKWebView._requestTextExtraction(in:completionHandler:)):

Upstream support code for converting `WKTextExtractionItem` → `IntelligenceElement`.

* Tools/WebKitTestRunner/TestOptions.cpp:
(WTR::TestOptions::defaults):
(WTR::TestOptions::keyTypeMapping):
* Tools/WebKitTestRunner/TestOptions.h:
(WTR::TestOptions::textExtractionEnabled const):
* Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm:
(WTR::TestController::cocoaResetStateToConsistentValues):

Reintroduce a test option to enable TextExtraction, since enablement is now properly gated on
`HAVE(UIINTELLIGENCESUPPORT_FRAMEWORK)`. Prior to this upstreaming, it was unconditionally (and
unnecessarily) enabled everywhere.

Canonical link: https://commits.webkit.org/280051@main
@webkit-commit-queue
Copy link
Collaborator

Committed 280051@main (66195e9): https://commits.webkit.org/280051@main

Reviewed commits have been landed. Closing PR #29843 and removing active labels.

@webkit-commit-queue webkit-commit-queue merged commit 66195e9 into WebKit:main Jun 15, 2024
@webkit-commit-queue webkit-commit-queue removed the merge-queue Applied to send a pull request to merge-queue label Jun 15, 2024
@whsieh whsieh deleted the eng/275510 branch June 15, 2024 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pxlcoder pxlcoder pxlcoder approved these changes

@megangardner megangardner megangardner approved these changes

@emw-apple emw-apple Awaiting requested review from emw-apple emw-apple is a code owner

@cdumez cdumez Awaiting requested review from cdumez cdumez is a code owner

@hortont424 hortont424 Awaiting requested review from hortont424

@rr-codes rr-codes Awaiting requested review from rr-codes

@aprotyas aprotyas Awaiting requested review from aprotyas

@JonWBedard JonWBedard Awaiting requested review from JonWBedard JonWBedard is a code owner

Assignees

@whsieh whsieh

Labels
Platform Portability improvements and other general platform improvements not driven directly by site bugs.
Projects
None yet
Milestone
No milestone
6 participants
@whsieh @webkit-early-warning-system @webkit-commit-queue @pxlcoder @megangardner @webkit-ews-buildbot