Skip to content

Latest commit

 

History

History
66 lines (37 loc) · 13.8 KB

PA_implementation_overview.md

File metadata and controls

66 lines (37 loc) · 13.8 KB

Protected Audience Implementation Overview

This document is intended to give a rough overview of the Protected Audience implementation in Chrome, with a focus towards performance-relevant design decisions, and timeouts.  The purpose is to both explain how these work, and help consumers figure out if different timeouts/timeout behavior would be useful.

This document starts out discussing singleton auctions, without any nested component auctions.  There is a section on component auctions at the end.

Phase 1 Load Interest Groups

Immediately after runAdAuction() is invoked, all interest groups, k-anon data, and win history will be loaded from the SQLite database, wrapped by an in-memory cache to avoid duplicate lookups.  No scripts will be loaded until after this is complete, for all interestGroupBuyers participating in an auction.  Interest groups are loaded on a per-buyer basis, and remain grouped by buyer for the entire auction.  Interest groups that cannot participate in the auction (due to, e.g., no biddingLogicURL) are removed.  If there are no interest groups that can bid in the auction, and no additional bids, the auction completes without a winner, bypassing future stages.

For each buyer origin, Chrome determines a tentative priority amongst all interest groups for that buyer - using each interest group’s fixed priority, or priority vector, if present.  Chrome then sorts the interest groups based on priority, removing any with priority vectors and negative priorities.  Interest groups with matching priorities are grouped by joining origin (to improve performance in case executionMode is group-by-origin).  Groups with matching joining origins and priority are randomly shuffled.  For each buyer origin, Chrome checks if any interest group participating in the auction has enableBiddingSignalsPrioritization set.  If not, Chrome applies the auction’s per-buyer interest group limit (perBuyerGroupLimits) to the buyer’s interest groups.  Either way, the interest groups are now in a priority order that will affect the order in which other resources will be fetched, and scripts will be run in.  None of this affects additional bids.

Phase 2 Bidding and Scoring Phase

Starting Script Executors

Once all interest groups are loaded, Chrome starts loading the seller script, and all bidder scripts participating in an auction.  To load a script, Chrome first checks if there is already a matching script “executor” (Same script origin, same JavaScript URL, WASM URL, trusted signals URL, same frame - frame match is required by the DevTools hooks. "Frame" here means the frame runAdAuction() was called in.  It may be for a different auction / component auction, however).  If so, the executor is reused.  Note that different executors do not currently share any network fetches or JavaScript contexts, although they do share the same global HTTP cache, so it’s generally a good idea to try and use the same URLs for all interest groups, to maximize executor reuse.  Also, interest groups that share an executor are guaranteed to be run in the priority order they were sorted into (this may change if trusted signals fetches are split up, though).  Interest groups that don’t share an executor are run on the basis of whichever has everything it needs to run first, runs first.

Otherwise, Chrome requests a buyer or seller process scoped to the associated buyer/seller origin.  Buyers and sellers do not share processes, even if the origin is the same, though a single buyer or seller origin will globally use a single process shared by all of its executors. There are separate Chrome-wide limits on the number of buyer processes (10) and seller processes (3).  These numbers are not optimized, and subject to change.  On mobile, Chrome will generally reuse the single renderer process with one thread for bidders and one for sellers, instead of using more secure, isolated processes, due to resource constraints.  Within a process, all Protected Audience JavaScript operations use a single thread.  Once a process is assigned, Chrome creates an executor for the script, which starts loading the JavaScript and WASM URLs.  Because bidders share processes based on origin, this process means that all of a single buyer origin’s interest groups are assigned an executor (newly created or reused) at once.

Processes/executors for individual bidders are requested in the order they appear in interestGroupBuyers, and new processes are assigned in a FIFO order.  So unless there’s a pre-existing bidder process for a particular origin, bids are generally generated based on the order buyers appear in the interestGroupBuyers.  There may be some potential advantages/disadvantages based on order (e.g., if Promises take a while to resolve, the cumulative timeout starts after executor assignment for bidders that get assigned an executor before the Promises resolve), so it’s recommended that buyer order be randomized.

When an executor has been assigned to a buying origin’s scripts, if there are no pending Promises in the auction config (more on that a bit below), the cumulative timeout for that bidder (perBuyerCumulativeTimeouts) starts.  The cumulative timeout is wall clocked time from the point on - it never pauses.  It’s possible that the bidder process is still being started when the executor is assigned.  In the case of multiple auctions run in a single frame, it’s also possible the bidder’s executor process has already finished starting up, and has already loaded the relevant JavaScript/WASM URLs, and is ready to go.

Trusted Signals Fetches / Reprioritization

Once an interest group has an executor, Chrome sends the executor information about all the interest groups that can share the executor, and starts the trusted signals fetch for that executor.

If enableBiddingSignalsPrioritization was set for any of a buyer’s interest groups, Chrome then waits for all of that buyer’s trusted signals fetches to complete, and recalculates the priority of all interest groups with enableBiddingSignalsPrioritization set, and sorts them again, removing interest groups with negative priorities after the multiplication.  Chrome then applies the auction config’s interest group limit for the bidder (perBuyerGroupLimits), if there is one.  Only after this can bidding start.

If enableBiddingSignalsPrioritization was not true for any of a bidder’s interest group, this extra wait-for-all-signals-and-then-reprioritize step is skipped.

Wait for Promise Resolution

Before generateBid() may be invoked, the Chrome waits for all Promises that are members of an auction’s config to be resolved, including those that do not pass anything to generateBid() (like, e.g., additionalBids).  Chrome also delays starting the per-buyer cumulative timeouts until all of an auction configs' Promises have been resolved (in addition to that buyer’s scripts having been assigned executors).

Generate Bids

Once all Promises have been resolved, and an interest group’s trusted bidding signals have been fetched (if necessary), and reprioritization has been done (again, if necessary), generateBid() is finally invoked.  If the returned ad is not k-anonymous, it’s immediately invoked a second time, restricting the passed in ads to k-anonymous ones.  The perBuyerTimeout, which defaults to 50 milliseconds, is applied independently to each call to generateBid().  Note that it includes running global scripts, in addition to running generateBid() itself.

Once all generateBid() calls that share an executor are complete, Chrome immediately releases the executor handle for that particular generateBid() call to free up resources.

Reaching Per Buyer Cumulative Timeout

Once the Per Buyer Cumulative Timeout (perBuyerCumulativeTimeouts) is reached, all pending work for a bidder that has yet to fully complete and return a result to the auction is dropped - that includes any currently running generateBid() call, generateBid() calls that have completed but whose bids have yet to reach the browser process (which shouldn’t be common), as well as interest groups at any earlier stage.

So this timeout potentially includes executor process startup time, time to load bidding scripts and trusted bidding signals, time to wait for all interest groups to get bidding signals (if enableBiddingSignalsPrioritization is true for any of a bidder’s interest group), and, of course, time to generate a bid.  It does not include time waiting for Promises passed into the auction config to be resolved, time to load interest groups, or time to call scoreAd() for the bids a buyer generates.

Note that if multiple auctions share a bidder process, regardless of whether they share executors or not, they share a JavaScript thread, so there is potential contention for a shared resources, while both of their cumulative timeouts run down.  On the other hand, the bidder that starts second may be able to reuse an already created executor, so could avoid having the time to start a process and fetch the bidding script counting against the cumulative timeout.

Scoring Bids

Once the seller executor has been loaded, of which there is always only one per aucton config, and Promises resolved, bids are passed to scoreAd() as soon as they’re available, on a first-come-first-served basis.  If any additionalBids are passed in to an auction, they’ll likely be scored first.  Since ties are resolved randomly, and there’s currently no timeouts that span multiple scoreAd() calls, the order in which bids are scored currently does not matter.  Each scoreAd() call independently respects the scoreAd() timeout (sellerTimeout), which includes the time to run the JavaScript file as well as the time to run scoreAd().

Trusted seller signals fetches are started every 10 milliseconds, when there’s a bid a seller executor is waiting to score.  Once the final bid for an auction has been generated, the final signals fetch is immediately issued.  As with bidders, seller executors can be shared between auctions in the same frame running concurrently, which also allows for for shared seller signals fetches.

Once all bids are scored, the result is immediately returned to the JavaScript Promise.

Phase 3 - Reporting Scripts

On auction completion, Chrome immediately reuses the seller executor to call reportResult(), and then reloads the winning bidder’s executor to call reportWin(), if it’s not still in memory.  The winning bidder executor is not kept in memory to avoid consuming bidder process quota, and to potentially avoid causing deadlock (e..g, if there are 10 auctions, all keeping different currently top-scoring bidders around, but that still need to load more bidders before they know the final winner, no auction could proceed).

Nothing is reported to the server, and the win is not recorded, until the winning ad is navigated to, or deprecatedURNToURL() is invoked, which may happen while still running reporting scripts.  If the ad loads before a reporting script completes, and tries to send a report that depends on reportWin() / reportResult(), and that function hasn’t completed, the report is queued until the script has finished running. Reporting scripts have a default timeout of 50 milliseconds.  In general, the run duration of these should not be web visible, though if the frame the auction ran in is destroyed before they complete, any currently in-progress work will be aborted.  This is considered a bug that needs to be fixed.  Running reporting scripts can affect the performance of running auctions, since processes used by reporting count towards the global Protected Audience process quotas, and use the same JavaScript threads as bidder/seller scripts from the same origin.

Component Auctions

In the case of top-level auction with multiple nested component auctions, the auctions are largely run independently, except that the interest group loading phases for all auctions must complete before any auction enters the generate bid phase. Each component auction waits only on its own Promises to resolve before generating bids. The top-level auction only requests an executor/process for itself once all component auctions have received one, to avoid deadlock.  The top-level auction scores bids as soon as it receives winning bids from each component auction, has its executor loaded, and all Promises for all auction configs have been resolved, including for component auctions.

Each component auction unloads the seller executor once all bids have been scored, again to avoid deadlock.  So the reporting phase may have to reload both the winning component seller's script, and the winning bidder's script.

Timeouts are only scoped to the auction associated with the auction config that specifies them.  The top-level auctions perBuyer[Cumulative]Timeouts are ignored, and its sellerTimeout only applies to its own scoreAd() calls, not to those of component auctions.

As hinted in the bid generation section, if two component auctions share a bidder, that bidder will use the same process in both auctions, reuse executors, and share a JavaScript execution thread.  Worklets are run on a first-come, first-served basis (though if the interest groups of a single buyer need multiple different executors, interest groups with a ready executor and loaded signals can jump the queue), and component auctions start generateBid() calls in the order they appear in the top-level auction config, so it is recommended that the order component auction config’s are listed be randomized, if cumulative buyer timeouts are in use, since they run concurrently, and, on a per-bidder basis, the earlier an auction’s bidders queue up to run generateBid() are more likely they are to fall within the timeout.