Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix MoE matmul scale issue #685

Merged
merged 1 commit into from
Jun 5, 2024
Merged

Fix MoE matmul scale issue #685

merged 1 commit into from
Jun 5, 2024

Conversation

RissyRan
Copy link
Collaborator

@RissyRan RissyRan commented Jun 5, 2024 •
edited

Description

Update base_emb_dim to emb_dim so that global_parameter_scale can take effect properly.

Test

End-to-end run with global_parameter_scale=2: test log

@RissyRan RissyRan marked this pull request as ready for review June 5, 2024 15:36
@copybara-service copybara-service bot merged commit 16aeac4 into main Jun 5, 2024
14 checks passed
@copybara-service copybara-service bot deleted the fix_matmul_scale branch June 5, 2024 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gobbleturk gobbleturk gobbleturk approved these changes

@rwitten rwitten Awaiting requested review from rwitten rwitten is a code owner

Assignees

@gobbleturk gobbleturk

Labels
pull ready
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@RissyRan @gobbleturk