Extend `setExperimentActive` to support enrollment IDs
Categories
(Toolkit :: Telemetry, enhancement, P2)
Tracking
()
| Tracking | Status | |
|---|---|---|
| firefox70 | --- | fixed |
People
(Reporter: mythmon, Assigned: chutten)
References
Details
Attachments
(2 files)
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
2.66 KB,
text/plain
|
teon
:
data-review+
|
Details |
In order to help correlate enrollments, unenrollments and other events, and help distinguish between cloned profiles, Normandy would like to associate each call to setExperimentActive with a random, unique id that is generated when a user first enrolls in an experiment. This data should then be accessible by analysts in some way.
One way this could be done is extending the setExperimentActive method to allow for another option, enrollmentId. Normandy could then generate that data, persist it, and pass it to setExperimentActive when needed.
|
Assignee | |
Comment 1•5 years ago
|
||
Is this needed independently of (earlier than?) the broader project in bug 1549760?
|
Reporter | |
Comment 2•5 years ago
|
||
This is independent from 1549760. It's a separate request for more useful Telemetry from Normandy that happened to be at the same time as bug 1549760.
|
|
Assignee | |
Updated•5 years ago
|
|
|
Assignee | |
Comment 3•5 years ago
|
||
This id, to be generated at runtime by the client, will be used to distinguish
when a client enrolls multiple times in the same id/branch/type.
|
|
Assignee | |
Comment 4•5 years ago
|
||
:mythmon, would you mind writing up the Data Collection Review for this? I'm not 100% clear on how this is intended to be used.
Alternatively, I suppose we could attach the Data Review to bug 1555178 where the IDs will actually be generated and submitted.
|
|
Assignee | |
Comment 5•5 years ago
|
||
Hey Sunah, does this look like a useful shape for this to be in for Data Engineering? Anything else needing to be done?
Looks good to me! I just filed a dependent bug for the data engineering side.
|
|
Reporter | |
Comment 7•5 years ago
|
||
|
|
Reporter | |
Updated•5 years ago
|
|
|
Assignee | |
Updated•5 years ago
|
|
|
Assignee | |
Updated•5 years ago
|
|
||
Comment 8•5 years ago
|
||
just need a bit of clarification:
- for the inclusion of the
enrollment_id, will this be added as a scalar to the telemetry collection? - and for the
enrollment_id, this is an existing id, but will just be included in the environment?
|
|
Reporter | |
Comment 9•5 years ago
|
||
enrollment_id will be newly generated for each enrollment, and won't be re-used beyond a single experiment. I don't know how exactly this will be represented in the telemetry ping, for that I'd direct you towards :chutten.
We also plan to include the enrollment ID in other pings related to a users enrollment in an experiment, such as Telemetry Events.
|
|
Assignee | |
Comment 10•5 years ago
|
||
It'll be present in the Environment of every ping that contains an Environment. Documentation can be found here: https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html
(( It'll be included in the example and explained in the experiments subsection. It will also have a section detailing its usage in the API documentation here: https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/experiments.html#the-js-api ))
|
|
||
Comment 11•5 years ago
|
||
Comment on attachment 9077814 [details] data-collection-review-1555172.md DATA COLLECTION REVIEW RESPONSE: Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate? Yes, it will be present in the Environment of every ping that contains an Environment. Documentation can be found here: https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html Is there a control mechanism that allows the user to turn the data collection on and off? Yes. This collection is Telemetry so can be controlled through Firefox's Preferences. If the request is for permanent data collection, is there someone who will monitor the data over time? Yes, :mythmon is responsible. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under? Category 2, Interaction. Is the data collection request for default-on or default-off? Default on for all channels with Normandy enabled. Does the instrumentation include the addition of any new identifiers? Yes, enrollment_id. The ID will expire after the experiment is completed. Is the data collection covered by the existing Firefox privacy notice? Yes. Does there need to be a check-in in the future to determine whether to renew the data? No. This collection is permanent. --- Result: datareview
|
||
Comment 12•5 years ago
|
||
Pushed by chutten@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/acd1ffd459f7 Allow annotating experiments with Enrollment Ids. r=janerik,mythmon
|
|
||
Updated•5 years ago
|
|
||
Comment 13•5 years ago
|
||
| bugherder | ||
|
||
Comment 14•5 years ago
|
||
Alicia, I noticed that this collection contains a new persistent ("life of an experiment") random identifier, and one possible parsing of the data collection review form is "escalate to legal if the data includes new identifiers"—I don't have any concerns but wanted to put your eyes on it. AIUI, Bug 1555176 will actually start reporting this identifier.
|
||
Comment 15•5 years ago
|
||
Hi Tim,
Thanks for the escalation. Trust & Security review = approved; Cat 2 interaction data. Bug 1555176 has also been updated accordingly with Trust signoff.
Description
•