Simplified Chinese Kanji font appears instead of Japanese Kanji font on some websites depending on Accept-Language
Categories
(Core :: Layout: Text and Fonts, defect)
Tracking
()
Tracking | Status | |
---|---|---|
relnote-firefox | --- | 125+ |
firefox-esr115 | --- | unaffected |
firefox125 | --- | fixed |
firefox126 | --- | fixed |
firefox127 | --- | fixed |
People
(Reporter: yelwm46907, Assigned: jfkthame)
References
(Regression)
Details
(Keywords: regression)
Attachments
(6 files, 1 obsolete file)
19.73 KB,
text/plain
|
Details | |
374.03 KB,
image/png
|
Details | |
372.01 KB,
image/png
|
Details | |
48 bytes,
text/x-phabricator-request
|
Details | Review | |
48 bytes,
text/x-phabricator-request
|
phab-bot
:
approval-mozilla-beta+
|
Details | Review |
48 bytes,
text/x-phabricator-request
|
phab-bot
:
approval-mozilla-release+
|
Details | Review |
User Agent: Mozilla/5.0 (Linux; Android 14; Mobile; rv:127.0) Gecko/127.0 Firefox/127.0
Steps to reproduce:
Nightly and Beta versions of Android Firefox display simplified Chinese kanji fonts instead of Japanese kanji fonts on some sites (github, bugzilla, chatgpt, etc.).
This did not happen in previous versions or in other browsers.
I tried resetting "intl.accept_languages" in about:config and the problem went away until I quit those apps.
Comment 1•1 month ago
|
||
Zack, thanks for reporting this bug! Can you please share your Firefox's about:support
information in this bug?
What is your Android system locale? What was the value of "intl.accept_languages" before you reset it?
(In reply to Chris Peterson [:cpeterson] from comment #1)
Zack, thanks for reporting this bug! Can you please share your Firefox's
about:support
information in this bug?What is your Android system locale? What was the value of "intl.accept_languages" before you reset it?
My Android system locale is Japan. The value of "intl.accept_languages" before resetting is "ja-JP,en-US,zh-Hans-CN,zh-Hant-TW,zh-Hant-HK,zh-CN,zh-TW,zh-HK".
Comment 4•1 month ago
|
||
The bug has a release status flag that shows some version of Firefox is affected, thus it will be considered confirmed.
Comment 5•1 month ago
|
||
:boek next week is the last week of beta for Fx126. We have little time to investigate this.
Could this be triaged and prioritized for investigation?
Updated•1 month ago
|
Comment 6•1 month ago
|
||
My Android system locale is Japanese too. intl.accept_languages was ja-JP here, I tried adding zh-CN, but, none of the sites you mentioned (chatgpt, bugzilla, github) are shown to me in Japanese or Chinese. They're all in English...
Comment 7•1 month ago
|
||
:zack I see you have this addon for translating pages: TWP - Translate Web Pages
. Would you mind disabling or uninstalling this addon and checking again? The settings for this addon has language selectors for translating webpages in it's settings and I'm wondering if this is actually causing your problems. Additionally, there is a mobile version of this addon called TWP - Translate for Mobile
that may work better but I haven't tested.
Sorry, my standard system locale is Japanese, but other languages were also set. All my Android languages are Japanese, English, Simplified Chinese (Mainland China), Traditional Chinese (Taiwan) and Traditional Chinese (Hong Kong).
Kanji has slightly different character forms for the same character code kanji depending on the country and region (you may not know this difference unless you use Kanji on a regular basis).
This bug is that Kanji characters in other regions are displayed even though the language is set to Japanese.
This is a trivial problem, but as I always use Kanji, I feel quite uncomfortable.
I have uninstalled TWP and initialized Firefox, but this bug still occurs.
Comment 10•1 month ago
|
||
Can you give a link to a specific page that shows Japanese written with Chinese characters? A screenshot could be useful too.
Comment 11•1 month ago
|
||
FWIW, after adding simplified Chinese via the Android settings, I now have intl.accept_languages set to ja-JP,zh-Hans-CN,zh-CN
, and e.g. Google or Wikipedia are still displayed in proper Japanese.
Comment 12•1 month ago
|
||
So does chatGPT if I talk to it in Japanese.
Reporter | ||
Comment 13•1 month ago
|
||
この画像はintl.accept_languagesをリセットする前のchatgptです。
Reporter | ||
Comment 14•1 month ago
|
||
And this is after the reset.
Reporter | ||
Comment 15•1 month ago
|
||
Ahhh, Sorry, I wrote the sentence before I translated it
"この画像はintl.accept_languagesをリセットする前のchatgptです。" -> "This image shows chatgpt before resetting intl.accept_languages."
Reporter | ||
Comment 16•1 month ago
|
||
I don't get this bug with google or wikipedia on my device.
Comment 17•1 month ago
|
||
Okay, I'm definitely able to reproduce, and it's even reproducible with Android in English
My STR:
- Go to about:config and ensure
intl.accept_languages
is reset. - Open, Android settings app, select "System", then "Languages & input", then "Languages"
- Set things up so that you have the following languages:
- English (United States)
- 日本語 (日本)
- 简体中文(中国)
- In Firefox, go to https://chat.openai.com
- As a prompt, type 変化、設置、街角、絵画、抱く、花火
I think the most visible difference to people not accustomed to Chinese/Japanese is in 抱く
If you go back to the Android settings and remove 简体中文(中国), returning to Firefox resets the display (without reloading the page or anything) and switches to the right font.
Comment 18•1 month ago
|
||
I can also confirm this doesn't affect Chrome.
Comment 19•1 month ago
|
||
I can also reproduce with Firefox 125.2.0 (Build #2016016039)
Comment 20•1 month ago
|
||
In that case, this bug isn't a regression in Firefox 126, though perhaps it's a regression in 125 or earlier.
Assignee | ||
Comment 21•1 month ago
|
||
I wonder if this is related to the (fairly recent) implementation of Android font visibility restrictions (anti-fingerprinting) in bug 1826412. Does turning off tracking protection affect the behavior?
Comment 22•1 month ago
|
||
(In reply to Jonathan Kew [:jfkthame] from comment #21)
Does turning off tracking protection affect the behavior?
Assuming I turned off the right thing, no.
Reporter | ||
Comment 23•1 month ago
|
||
The cause may be that Firefox has set only ja-JP as the locale code for Japanese in accept-language.
Firefox for Windows sets ja for Japanese locale code.
Other mobile browsers set both ja and ja-JP.
Adding ja to the beginning of the intl.accept_languages value produces the same result as resetting it.
Assignee | ||
Comment 24•1 month ago
|
||
(In reply to zack from comment #23)
The cause may be that Firefox has set only ja-JP as the locale code for Japanese in accept-language.
Oh, interesting -- yes, I suspect that could be the issue; the font preferences may not recognize that. If that's the issue, it should be straightforward to fix, I think. I'm out of the office today but will try to check on this when I'm back, if no-one else does it in the meantime.
Updated•1 month ago
|
Comment 25•1 month ago
|
||
14:50.30 INFO: No more integration revisions, bisection finished.
14:50.30 INFO: Last good revision: c55a609359a20d23e6534c7c188007d07bb1877b
14:50.30 INFO: First bad revision: 0f63ab5624960b1aa464bb2e3bd76c6cd196f186
14:50.30 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=c55a609359a20d23e6534c7c188007d07bb1877b&tochange=0f63ab5624960b1aa464bb2e3bd76c6cd196f186
Updated•1 month ago
|
Updated•1 month ago
|
Comment 26•1 month ago
|
||
Makoto, this bug is a regression from your Accept-Language change in bug 1847701 in Fx 125. Is this a website/webcompat bug? Or should we change how we format the locale tags in Accept-Language again?
Updated•1 month ago
|
Assignee | ||
Comment 27•1 month ago
|
||
I suspect (but have not yet confirmed) that this is happening because the GetFontPrefLangFor
function called here only recognizes ja
, not ja-JP
, and we should fix that to make it smarter.
Comment 28•1 month ago
|
||
And ja is not the only locale with the problem:
https://searchfox.org/mozilla-central/source/gfx/thebes/gfxFontPrefLangList.h#1
Comment 29•1 month ago
|
||
It is worth noting that the claim in bug 1847701's commit message that Accept-language is "language-region format like Desktop" is actually not true for Japanese:
https://hg.mozilla.org/l10n-central/ja/file/eaeda87d1d4ecbe97a51beccde04b09e055a5157/toolkit/chrome/global/intl.properties#l23
Comment 30•1 month ago
|
||
Many locales put both "language-region" and "language", but many skip "language-region".
Comment 31•1 month ago
|
||
Hmm, before landing bug 1847701, accept-language was ja-JP if system locale is Japanese. Chinese was zh-Hans-TW (Taiwan) or zh-Hans-CN (China). But after this change, Chinese becomes zh-TW or zh-CN. It means that font selection in gfx matches with Chinese.
Updated•1 month ago
|
Comment 32•1 month ago
|
||
(Although I filed bug 1889845, we should normalize this value by Gecko side at future)
Comment 33•1 month ago
|
||
(In reply to Makoto Kato [:m_kato] from comment #31)
Hmm, before landing bug 1847701, accept-language was ja-JP if system locale is Japanese. Chinese was zh-Hans-TW (Taiwan) or zh-Hans-CN (China). But after this change, Chinese becomes zh-TW or zh-CN. It means that font selection in gfx matches with Chinese.
In the geckoview example app, it was indeed ja-JP,zh-Hans-CN with my current Android system settings in version 124. And indeed, if I change that to ja-JP,zh-CN manually, I get the same font selection problem. Seems like comment 27 might be spot on, and we never even matched Japanese before.
Comment 34•1 month ago
|
||
That said, we are also currently not using language-region on desktop.
Comment 35•1 month ago
|
||
This bruteforce approach fixed the issue for me: https://hg.mozilla.org/try/rev/f960adc00da5cea1c5dc9bca5a3c7a3e0903033b, so comment 27 is confirmed. Interestingly, logging the GetFontPrefLangFor function, I can see that it regularly returns Others on my desktop, where it's queried for e.g. ja-jp (lowercase, I haven't looked where it comes from, but it's not from accept_languages).
Comment 36•1 month ago
|
||
Original issue is that font selection doesn't match with Japanese when system
locale is Japanese. Before landing bug 1847701, all far east languages don't
always match. But after langing it, Chinese (Simplified and Traditional) is
macthed only.
Gfx doesn't want that Accept-Langues preference has a country code if
languguage is Japanese or Korean.
So we should return language code only for accept-language if language tag is
ja-JP or ko-* (North or South).
Updated•1 month ago
|
Updated•1 month ago
|
Assignee | ||
Comment 37•1 month ago
|
||
In principle, this also affects the other font prefs where we use a bare language subtag without region, if the accept-language list (or other source of lang tags) has the language-region form. These include Arabic, Greek, Hebrew, and Thai, as well as Japanese and Korean. In practice those cases are less likely to be noticed, as fallback behavior will likely still pick an appropriate font; there aren't the same region-dependent variants as with CJK. But we should really fix them all.
Assignee | ||
Comment 38•1 month ago
|
||
Assignee | ||
Comment 39•1 month ago
|
||
The patch in D208588 is a somewhat ugly hack, but should resolve this AFAICS. We could go the whole way and use Locale parsing and then extract the lang subtag, but that seems overkill given the very limited set of font pref lang codes we're dealing with; and it's my hope that we'll eventually be replacing all this with a more principled lang- and script-driven architecture anyway.
Updated•1 month ago
|
Assignee | ||
Comment 41•1 month ago
|
||
Moving this to Core :: Graphics: Text, as it's really an issue in how gfx/thebes handles lang codes in relation to font prefs, and in principle it could affect other platforms as well, if people modify their accept-language
to have <lang-region> codes for cases where gfx only expects the lang.
Comment 42•1 month ago
|
||
Set release status flags based on info from the regressing bug 1847701
Updated•1 month ago
|
Assignee | ||
Comment 43•1 month ago
|
||
Actually, this is really about font selection, rather than rendering; -> Layout.
Comment 44•1 month ago
|
||
Pushed by jkew@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/943302be4233 When looking for font prefs based on a lang tag, try comparing the base lang alone if the whole tag doesn't match. r=lsalzman
Assignee | ||
Comment 45•1 month ago
|
||
Original Revision: https://phabricator.services.mozilla.com/D208588
Updated•1 month ago
|
Assignee | ||
Comment 46•1 month ago
|
||
Original Revision: https://phabricator.services.mozilla.com/D208588
Updated•1 month ago
|
Comment 47•1 month ago
|
||
beta Uplift Approval Request
- User impact if declined: Japanese users may see incorrect (Chinese) font; potentially other similar impacts
- Code covered by automated testing: yes
- Fix verified in Nightly: no
- Needs manual QE test: no
- Steps to reproduce for manual QE testing: n/a
- Risk associated with taking this patch: low
- Explanation of risk level: simple check for additional forms of language code
- String changes made/needed: none
- Is Android affected?: yes
Comment 48•1 month ago
|
||
release Uplift Approval Request
- User impact if declined: Japanese users may see incorrect (Chinese) font; potentially other similar impacts
- Code covered by automated testing: yes
- Fix verified in Nightly: no
- Needs manual QE test: no
- Steps to reproduce for manual QE testing: n/a
- Risk associated with taking this patch: low
- Explanation of risk level: simple check for more lang code forms
- String changes made/needed: none
- Is Android affected?: yes
Comment 49•1 month ago
|
||
bugherder |
Updated•1 month ago
|
Updated•1 month ago
|
Updated•1 month ago
|
Updated•1 month ago
|
Comment 50•1 month ago
|
||
uplift |
https://hg.mozilla.org/releases/mozilla-beta/rev/9127c3adbf51
Updated•1 month ago
|
Comment 51•1 month ago
|
||
uplift |
https://hg.mozilla.org/releases/mozilla-release/rev/53302f85d080
Comment 52•1 month ago
|
||
Just for information:
You can see the differences between the glyphs more easily with the following letters than with the test HTML.
刃 直 骨 海 角 遙 入
Comment 53•1 month ago
|
||
Added to the 125.0.3 relnotes.
Fixed an issue which could cause Japanese users to see incorrect fonts in some situations
Description
•