Skip to content

Commit

Permalink
Experimental interface for torch ops
Browse files Browse the repository at this point in the history
See SHI-Labs#184

Only supports forward pass for now, due to current limitations of
registering custom ops with torch compared to autograd functions. Some
of those limitations are:

* No stable interface for supporting autocasting to fp16/bf16,
  * Gradient scaling doesn't seem to be supported either, leading to
    training instability.

* Ops cannot indicate that they expect contiguous operands, and need to
  call `.contiguous()` within, and this incurs additional tensor copy
  costs, and brings down throughput (in some cases it's hard to even
  tell the difference between compiled and eager.)
  • Loading branch information
alihassanijr committed Dec 10, 2024
1 parent ee45f9d commit 18ecf32
Show file tree
Hide file tree
Showing 3 changed files with 609 additions and 2 deletions.
Loading

0 comments on commit 18ecf32

Please sign in to comment.