Commits

Commits on May 21, 2024

bump and pin versions (#112 )
vchiley
committedMay 21, 2024
fix amp (#111 )
mvpatel2000
committedMay 21, 2024

Commits on May 17, 2024

Add Shared Expert (#109 )
vchiley
committedMay 17, 2024

Commits on May 15, 2024

Fix for ffn_hidden_size of 128, and better error message for incompatible ffn sizes. (#108 )
snarayan21
committedMay 15, 2024

Commits on Mar 26, 2024

Update README.md (#98 )
dakinggg
committedMar 26, 2024

Commits on Feb 20, 2024

Remove turbo (#96 )
dblalock
committedFeb 20, 2024

Commits on Jan 11, 2024

Updt triton pin (#89 )
vchiley
committedJan 11, 2024

Commits on Jan 10, 2024

Fix moe_normalize_expert_weights when top_k=1 (#87 )
152334H
committedJan 10, 2024

Commits on Dec 22, 2023

fix default (#76 )
mvpatel2000
committedDec 22, 2023
refactor (#75 )
j316chuck
committedDec 22, 2023
Mem opt glu bkwd (#72 )

mvpatel2000
and
vchiley
committedDec 22, 2023

Commits on Dec 21, 2023

Refactor dtesnor (#74 )
mvpatel2000
committedDec 21, 2023
Dtensor to all paths (#73 )
mvpatel2000
committedDec 21, 2023
Add cast to tensor for DTensor inputs for groupedmlp (#71 )
eracah
committedDec 21, 2023
add mem optimized grouped glu (#66 )
vchiley
committedDec 21, 2023
change router norm from in-place (#70 )
sashaDoubov
committedDec 21, 2023

Commits on Dec 20, 2023

Skip updating load balancing loss on eval (#69 )
sedrick-keh-tri
committedDec 20, 2023

Commits on Dec 19, 2023

enable custom activation functions (#65 )
vchiley
committedDec 19, 2023

Commits on Dec 15, 2023

Update README.md (#63 )
eltociear
committedDec 15, 2023

Commits on Dec 13, 2023

More customizable norm for expert weights (#60 )
snarayan21
committedDec 13, 2023
enable arg enabled normalization of routing weights (#58 )
vchiley
committedDec 13, 2023

Commits on Dec 11, 2023

Update setup.py to support multiple device capabilities (#56 )
simon-mo
committedDec 11, 2023
Update Megatron-LM scripts and integration for latest Docker container. (#55 )
tgale96
committedDec 11, 2023
Fix README (#54 )
tgale96
committedDec 11, 2023
Update dependencies and package organization. (#52 )
tgale96
committedDec 11, 2023

Commits on Dec 8, 2023

bump to v0.5.0
mvpatel2000
committedDec 8, 2023
Merge pull request #47 from stanford-futuredata/fix-topology-kernel
tgale96
committedDec 8, 2023
Fix bug in topology kernel for ffn_hidden_size>4096.
tgale96
committedDec 8, 2023

Commits on Dec 5, 2023

Merge pull request #43 from bcui19/removing_extra_size
tgale96
committedDec 5, 2023
removing extra size
bcui19
committedDec 5, 2023

Commits on Dec 4, 2023

Merge pull request #41 from vchiley/enable_flat_inputs
tgale96
committedDec 4, 2023

Commits on Dec 1, 2023

enable generic dimentionality for input
vchiley
committedDec 1, 2023
Merge pull request #38 from sashaDoubov/sasha/glu
tgale96
committedDec 1, 2023

Commits on Nov 30, 2023

fix spacing again
sashaDoubov
committedNov 30, 2023
get rid of spacing
sashaDoubov
committedNov 30, 2023