With my friend @realcromwellian.bsky.social’s help (and his Mac Studio), we got a 126M param NanoGPT model training in ~165s — primarily written in SQL.
github.com/xqlsystems/xarray-sql/blob/main/benchm…
2 likes
Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL github.com/xqlsystems/xarray-sql/blob/6debe2495dc… (news.ycombinator.com)
17 views
Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL
1 point · 1 comment
Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL | Discussion
1 like
Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL
In this demo, we automatically deduce a FlashAttention kernel with ~25 lines of SQL, which just expresses vanilla attention. This is possible due to three portable SQL extensions that we’re building:
- xarray-sql: a library that efficiently represents petabyte scale array datasets in the cloud as tables (relations).
- ddx: a database extension that adds automatic differentiation in the style of JAX to standard SQL, assuming the above array data model.
- einfold: a modular query planner for fast tensor contractions in vanilla SQL, again, assuming the above array data model.
My hope is that the relational model and algebra provide fresh perspectives on how we might do accelerator programming. Here, our thesis is that separating the logical layers in a tensor program from the physical layers is both more correct and more scalable.
If the Bitter Lesson is a central dogma of machine learning, then why not formalize it with relational algebra?
If this interests you and you’d like to tinker with us at the intersection of database theory and ML systems, join us on Discord at xql.systems (link at the bottom). Cheers.
26 points · 1 comment
In other words, you’re free to write the mathematical definition of attention in SQL, and you get the high performance just by adding on my smart query planner:
xqlsystems/einfold
9 likes
New result: Automatic derivation of FlashAttention from vanilla attention in ~25 lines of SQL via three portable extensions.
github.com/xqlsystems/xarray-sql/blob/6debe2495dc…
27 likes · 3 reposts
Well, in this sibling project, I had one implementor make stacked PRs, and the adversary fuzz every feature: github.com/xqlsystems/ddx/blob/main/crates/ddx-co…
That has caught numerous defects in my code and upstream (5 bugs in upstream deps, like DataFusion).
Another review agent checked for max composability.
1 like
Mentions
With my friend @realcromwellian.bsky.social’s help (and his Mac Studio), we got a 126M param NanoGPT model training in ~165s — primarily written in SQL. github.com/xqlsystems/xarray-sql/blob/main/benchm…
Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL github.com/xqlsystems/xarray-sql/blob/6debe2495dc… (news.ycombinator.com)
Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL
Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL | Discussion
Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL In this demo, we automatically deduce a FlashAttention kernel with ~25 lines of SQL, which just expresses vanilla attention. This is possible due to three portable SQL extensions that we’re building: - xarray-sql: a library that efficiently represents petabyte scale array datasets in the cloud as tables (relations). - ddx: a database extension that adds automatic differentiation in the style of JAX to standard SQL, assuming the above array data model. - einfold: a modular query planner for fast tensor contractions in vanilla SQL, again, assuming the above array data model. My hope is that the relational model and algebra provide fresh perspectives on how we might do accelerator programming. Here, our thesis is that separating the logical layers in a tensor program from the physical layers is both more correct and more scalable. If the Bitter Lesson is a central dogma of machine learning, then why not formalize it with relational algebra? If this interests you and you’d like to tinker with us at the intersection of database theory and ML systems, join us on Discord at xql.systems (link at the bottom). Cheers.
In other words, you’re free to write the mathematical definition of attention in SQL, and you get the high performance just by adding on my smart query planner: xqlsystems/einfold
New result: Automatic derivation of FlashAttention from vanilla attention in ~25 lines of SQL via three portable extensions. github.com/xqlsystems/xarray-sql/blob/6debe2495dc…
Well, in this sibling project, I had one implementor make stacked PRs, and the adversary fuzz every feature: github.com/xqlsystems/ddx/blob/main/crates/ddx-co… That has caught numerous defects in my code and upstream (5 bugs in upstream deps, like DataFusion). Another review agent checked for max composability.