Activity
Stars and mentions · Last 30 days, Oct 8 – Nov 6
Top repositories
KellerJordan’s repositories ranked by new forks, as on trending.
Contributes to
Repositories outside KellerJordan’s account where KellerJordan opened or reviewed pull requests in the past 365 days.
Who noticed KellerJordan
Everyone who starred or forked KellerJordan’s repositories since the archive began in 2011.
Notable accounts
Ordered by the stars their own repositories have received.
Biggest fans
Accounts that starred the most of these repositories.
Breakout weeks
Each repository’s best week of new stars, with the release published around it.
Achievements
Awards earned by the repositories KellerJordan owns. Select an award to embed its badge.
No awards yet
Repositories earn awards by reaching the top three in a completed trending day or week.
A repository earns an award for finishing first, second or third in a completed trending day or week, ranked by new stars, new forks or pull request authors. Awards follow a repository through renames and transfers and count toward its current owner. Weeks run Monday to Sunday, UTC.
Mentions
@jon_barron Appendix A7 in the blogpost! it's a new kind of momentum we used to set a Muon nanogpt record: modded-nanogpt#369 tldr: instead of tuning an exponential decay \beta for momentum we can specify desiderata (mean lag, recency bias, etc) and directly solve for the optimal momentum coefficients max ent = maximum entropy, which is the optimization objective slop = because i kind of hate this method, i don't understand why it works, and i want every reader to understand that it's a research object that i don't necessarily endorse
@beyarkay this is because you have to include the logs in the PR from the rules of the repo. it's not actual code modded-nanogpt#367
Record PR: modded-nanogpt#367 Blog post: brunborg.com
I like to point to KellerJordan/modded-nanogpt which does what took OpenAI weeks on a specialized computer cluster in 2018-2019, in a manner of minutes and for less than a dollar worth of on-demand GPU compute
GPT-2(small)相当のNanoGPTを高速化するRTAが、今のベストスコアで8xH100で1分切ってるので、やろうと思えば割と自分だけのデータセットで自作LLMするハードルはだいぶ下がった気がする kellerjordan/modded-nanogpt
Geez, GPT-2-Medium can now be trained on 221 seconds... modded-nanogpt#380 (And small in 9.65 seconds)