Activity
Stars and mentions · Last 30 days, Oct 5 – Nov 3
Top repositories
alrojo’s repositories ranked by pull request authors, as on trending.
Contributes to
Repositories outside alrojo’s account where alrojo opened or reviewed pull requests in the past 365 days.
Who noticed alrojo
Everyone who starred or forked alrojo’s repositories since the archive began in 2011.
Notable accounts
Ordered by the stars their own repositories have received.
Biggest fans
Accounts that starred the most of these repositories.
Breakout weeks
Each repository’s best week of new stars, with the release published around it.
Achievements
Awards earned by the repositories alrojo owns. Select an award to embed its badge.
No awards yet
Repositories earn awards by reaching the top three in a completed trending day or week.
A repository earns an award for finishing first, second or third in a completed trending day or week, ranked by new stars, new forks or pull request authors. Awards follow a repository through renames and transfers and count toward its current owner. Weeks run Monday to Sunday, UTC.
Mentions
CroissAttn reproduces for stable NVFP4 training: github.com/alrojo/fp4Attn/blob/main/Croissattn_FP… Slight ppl degradation: 3.4765 vs 3.4585 for BF16 for 1B at 254M model. Trains 6% slower. So why use it? Because the final model ends up being 4-bit precision with the best possible ppl.
Incredible momentum in betterment we are seeing. alrojo/fp4Attn
@askalphaxiv This is THE fix for running attention backward pass in FP4 training. We can finally do end to end FP4 training! alrojo/fp4Attn
@petrenko_ai This line of work is incredibly interesting. We are very interested in quantizing RL rollouts. We use the same method here to run the full attention backward pass in FP4. alrojo/fp4Attn I believe exact replay might be the fix here.
🥐 Paper: alrojo/fp4Attn If you work on attention kernels, low-precision training, or RL rollouts, come talk to us!