First version of attention fusion

[gramalingam]

First version of attention fusion.

Limitations:

• Targetting only static shapes for now. Dynamic shapes will alter the pattern.
• Targetting only MHA and the new onnx Attention op.

[codecov]

❌ 8 Tests Failed:

Tests completed	Failed	Passed	Skipped
12317	8	12309	2012

View the top 1 failed tests by shortest run time

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0110_test_bitshift_left_uint32

Stack Traces | 0.003s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_bitshift_left_uint32'

The above exception was the direct cause of the following exception:
.nox\test_ort_nightly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_bitshift_left_uint32' (e=No module named 'tests.onnx_backend_test_code.test_bitshift_left_uint32') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_bitshift_left_uint32.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_bitshift_left_uint32.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import UINT32
E   from onnxscript.onnx_opset import opset11
E   
E   @script()
E   def bck_test_bitshift_left_uint32(x: UINT32[3], y: UINT32[3]) -> (UINT32[3]):
E       z = opset11.BitShift(x, y, direction='LEFT')
E       return z

View the full list of 2 ❄️ flaky tests

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0878_test_reduce_mean_do_not_keepdims_random

Flake rate in main: 16.67% (Passed 20 times, Failed 4 times)

Stack Traces | 0.003s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_reduce_mean_do_not_keepdims_random'

The above exception was the direct cause of the following exception:
.nox\test\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_reduce_mean_do_not_keepdims_random' (e=No module named 'tests.onnx_backend_test_code.test_reduce_mean_do_not_keepdims_random') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_reduce_mean_do_not_keepdims_random.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_reduce_mean_do_not_keepdims_random.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT, INT64
E   from onnxscript.onnx_opset import opset18
E   
E   @script()
E   def bck_test_reduce_mean_do_not_keepdims_random(data: FLOAT[3,2,2], axes: INT64[1]) -> (FLOAT[3,2]):
E       reduced = opset18.ReduceMean(data, axes, keepdims=0)
E       return reduced

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0981_test_resize_upsample_sizes_nearest_axes_3_2

Flake rate in main: 10.00% (Passed 27 times, Failed 3 times)

Stack Traces | 0.004s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_resize_upsample_sizes_nearest_axes_3_2'

The above exception was the direct cause of the following exception:
.nox\test\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_resize_upsample_sizes_nearest_axes_3_2' (e=No module named 'tests.onnx_backend_test_code.test_resize_upsample_sizes_nearest_axes_3_2') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_resize_upsample_sizes_nearest_axes_3_2.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_resize_upsample_sizes_nearest_axes_3_2.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT, INT64
E   from onnxscript.onnx_opset import opset19
E   
E   @script()
E   def bck_test_resize_upsample_sizes_nearest_axes_3_2(X: FLOAT[1,1,2,2], sizes: INT64[2]) -> (FLOAT[1,1,7,8]):
E       Y = opset19.Resize(X, None, None, sizes, axes=[3, 2], mode='nearest')
E       return Y

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

[github-advanced-security]

lintrunner found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

Copilot reviewed 6 out of 13 changed files in this pull request and generated no comments.

Files not reviewed (7)

• onnxscript/rewriter/onnxruntime/xformers/cos_sin_cache_test.py: Evaluated as low risk
• onnxscript/rewriter/onnxruntime/xformers/skip_normalization_test.py: Evaluated as low risk
• onnxscript/rewriter/onnxruntime/xformers/rms_normalization_test.py: Evaluated as low risk
• onnxscript/rewriter/onnxruntime/xformers/cos_sin_cache.py: Evaluated as low risk
• onnxscript/rewriter/onnxruntime/xformers/_smollm_1.py: Evaluated as low risk
• onnxscript/rewriter/onnxruntime/xformers/init.py: Evaluated as low risk
• onnxscript/rewriter/onnxruntime/xformers/rotary_embedding_test.py: Evaluated as low risk

[justinchuby]

The mypy checks can be ignored here:

onnxscript/.lintrunner.toml

Lines 51 to 53 in 347aa06

	'onnxscript/rewriter/onnxruntime/transformers/fastgelu.py', # FIXME
	'onnxscript/rewriter/onnxruntime/instance_to_group_normalization.py', # FIXME
	'onnxscript/rewriter/onnxruntime/xformers/_smollm_1layer.py', # onnxscript code