[SYCL][Driver] Support bfloat16 devicelib selection when multiple AOT targets specified #16494

jinge90 · 2024-12-31T13:24:29Z

User can specify multiple AOT targets when building sycl program in followings ways:
1). via -fsycl-targets=intel_gpu_pvc,intel_gpu_acm_g10,....
2). via -fsycl-targets=spir64_gen ... -Xs "-device pvc,dg2...."
3). via -fsycl-targets=spir64_gen..., -Xsycl-target-backend=spir64_gen "-device pvc"
We should select native bfloat16 devicelib when all AOT targets specified support native bfloat16 conversion. Currently, pvc, dg2, bmg devices support native bfloat16.
If user specifies JIT target together with AOT targets which all support native bfloat16 conversion, we still select native bfloat16 devicelib since bfloat16 devicelib is skipped in linking step for JIT target.

Signed-off-by: jinge90 <[email protected]>

…support native bfloat16 conversion

clang/test/Driver/sycl-device-lib-bfloat16.cpp

mdtoguchi · 2025-01-02T15:43:50Z

clang/lib/Driver/ToolChains/SYCL.cpp

+    // 1). clang++ -fsycl -fsycl-targets=spir64_gen -Xs "-device pvc,...,"
+    // 2). clang++ -fsycl -fsycl-targets=intel_gpu_pvc,...
+    // We assume that users will only apply either 1) or 2) and won't mix the
+    // 2 ways in their compiling command.


Given this assumption, can we emit a diagnostic when this s combination occurs?

Hi, @mdtoguchi
The assumption is listed here because compiler has already emitted error message about such mix use. For example, if we mix using like:
clang++ -fsycl -fsycl-targets=spir64_gen,intel_gpu_dg1 -Xs "-device pvc" test.cpp
Compilation will abort and emit:
clang++: error: unsupported option '-device' for target 'intel_gpu_dg1'

Thanks very much.

That error isn't guaranteed. For instance:
clang++ -fsycl -fsycl-targets=spir64_gen,intel_gpu_dg1 -Xsycl-target-backend=spir64_gen "-device pvc" test.cpp
This will compile, and create 2 different device binaries. The use of -Xs <arg> is a more general usage where <arg> is passed to all device compilations, while you can steer specific args using -Xsycl-target-backend=target <arg>

Hi, @mdtoguchi
Thanks for pointing out this, we need to take care of the scenario you mentioned, native bfloat16 devicelib can only be used when all GPU AOT devices support native bfloat16 conversion.

srividya-sundaram · 2025-01-02T18:34:15Z

clang/lib/Driver/ToolChains/SYCL.cpp

+    // 1). clang++ -fsycl -fsycl-targets=spir64_gen -Xs "-device pvc,...,"
+    // 2). clang++ -fsycl -fsycl-targets=intel_gpu_pvc,...
+    // We assume that users will only apply either 1) or 2) and won't mix the
+    // 2 ways in their compiling command.


What about the case where we can specify Intel GPUs via --offload-arch option?

clang++ --offload-new-driver -fsycl --offload-arch=bdw

Hi, @srividya-sundaram
Thanks for pointing out this, this PR didn't consider new offload driver, I will update it.
Thanks very much.

clang/lib/Driver/ToolChains/SYCL.cpp

srividya-sundaram · 2025-01-02T18:45:55Z

Please add some description to the PR in addition to the PR title.

Co-authored-by: Michael Toguchi <[email protected]>

Signed-off-by: jinge90 <[email protected]>

bashbaug · 2025-01-15T19:31:01Z

clang/lib/Driver/ToolChains/SYCL.cpp

+  static llvm::SmallSet<StringRef, 8> GPUArchsWithNBF16{
+      "intel_gpu_pvc", "intel_gpu_acm_g10", "intel_gpu_acm_g11",
+      "intel_gpu_acm_g12", "intel_gpu_bmg_g21"};


This is fine as a short-term fix, but longer-term we should be thinking of a different way to identify architectures with bfloat16 support that doesn't require maintaining a list of devices. For example, is there a way to query the properties of the target device(s) to determine if bfloat16 is supported?

Note that Lunar Lake GPUs also support bfloat16 (I think), and it's not in this list.

Hi, @bashbaug
The bf16 support can be queried in execution time because we can only know the real target the program is running during execution time. In compilation time for AOT mode, compiler driver can only decide according to target platform specified and we have to maintain the bf16 support information in compiler driver source code, otherwise compiler driver won't know whether a target platform supports bf16. The platform we are building the program may be different from the platform the program is going to run on.
Thanks very much.

Could we call into ocloc to do the query? For example, if we're AOT compiling for DG2, we could do something like (this example uses the ocloc command-line interface, but the same is supported for the library interface):

$ ocloc query CL_DEVICE_EXTENSIONS_WITH_VERSION -device dg2 <snip> cl_intel_bfloat16_conversions:1.0.0 <snip>

Because the cl_intel_bfloat16_conversions extension is supported, we can know that DG2 supports SPIR-V bfloat16 conversion instructions.

Hi, @bashbaug
I just tried the ocloc command but found after running "ocloc query CL_DEVICE_EXTENSIONS_WITH_VERSION -device dg2" but found a file named with "CL_DEVICE_EXTENSIONS_WITH_VERSION" is generated in local directory, it seems to be ocloc's behavior, is there any way to get rid of this?
Thanks very much.

clang/lib/Driver/ToolChains/SYCL.cpp

mdtoguchi · 2025-01-16T00:04:02Z

clang/lib/Driver/ToolChains/SYCL.cpp

+      if (Arg *SYCLTarget = Args.getLastArg(options::OPT_fsycl_targets_EQ)) {
+        for (auto TargetsV : SYCLTarget->getValues()) {
+          if (!checkSpirvJIT(StringRef(TargetsV)) &&
+              !StringRef(TargetsV).starts_with("spir64_gen") &&


Comment states 'filter out CPU and FPGA' targets, but GPU is checked here as well, which would seamingly negate the -device check above.

Hi, @mdtoguchi
If we use following command:
clang++ -fsycl -fsycl-targets=spir64,spir64_fpga,spir64_gen,spir64_x86_64 -Xsycl-target-backend=spir64_gen "-device pvc" assert-debug.cpp
Although only GPU toolchain is checked here, the sycl target value got from "-fsycl-targets=" is a list including "spir64", "spir64_fpga", "spir64_gen", "spir64_x86_64" and we only need to care about "spir64_gen" and corresponding "-device XXX" argument, so I filtered "spir64", "spir64_x86_64", "spir64_fpga".
Thanks very much.

mdtoguchi · 2025-01-16T00:05:58Z

clang/test/Driver/sycl-device-lib-bfloat16.cpp

+// RUN:   | FileCheck %s -check-prefix=BFLOAT16-FALLBACK-NONE-FALLBACK
+// RUN: %clangxx -fsycl -fsycl-targets=intel_gpu_pvc,spirv64-unknown-unknown,intel_gpu_dg1 \
+// RUN:   --sysroot=%S/Inputs/SYCL %s -### 2>&1 \
+// RUN:   | FileCheck %s -check-prefix=BFLOAT16-FALLBACK-NONE-FALLBACK


Please add a check for the -fsycl-targets=intel_gpu_pvc,spir64_gen -Xsycl-target-backend=spir64_gen "-device dg1" type of scenario if we are allowing that combination.

Hi, @mdtoguchi
Already added the test. For this scenario (mix use of -fsycl-targets=intel_gpu_xxx,spir64_gen ... -Xsycl-target-backend=spir64_gen "-device YYY"), do we have plan to continue to support it in incoming new offload driver mode? Or do we have plan to unify all the ways to use offloading AOT compilation?
Currently, such combined use will create 2 device binaries and call device link 2 times, selectBfloat functions will be called in each device link process but I can't figure out a reliable way to get to know what the target device architecture is in each device link phase, so have to limit the use of native bfloat16 devicelibs only when all device architectures support native bfloat16 conversion, do you have any good idea to solve this?

Thanks very much.

Co-authored-by: Michael Toguchi <[email protected]>

…native_bf16_aot

Signed-off-by: jinge90 <[email protected]>

jinge90 · 2025-01-21T07:58:35Z

Hi, @mdtoguchi
I am a little confused about the intel_gpu_* targets string, we can use:
clang++ -fsycl -fsycl-targets=intel_gpu_acm_g10 xxx to specify aot compilation for DG2 platform but we also have:
clang++ -fsycl -fsycl-targets=intel_gpu_dg2_g10 xxx for DG2 target. Is there duplicate here? Or is there any difference between intel_gpu_acm_* and intel_gpu_dg2_*

Thanks very much.

Signed-off-by: jinge90 <[email protected]>

jinge90 added 2 commits December 31, 2024 14:59

[SYCL] Select Native Bfloat16 devicelib in AOT

27512df

Signed-off-by: jinge90 <[email protected]>

Merge remote-tracking branch 'upstream/sycl' into link_native_bf16_aot

becaba8

jinge90 requested a review from a team as a code owner December 31, 2024 13:24

jinge90 temporarily deployed to WindowsCILock December 31, 2024 13:24 — with GitHub Actions Inactive

[SYCL][Driver] Select Native bfloat16 when all AOT targets specified …

223f5e0

…support native bfloat16 conversion

jinge90 temporarily deployed to WindowsCILock December 31, 2024 14:04 — with GitHub Actions Inactive

jinge90 requested review from mdtoguchi and bashbaug January 2, 2025 08:47

mdtoguchi reviewed Jan 2, 2025

View reviewed changes

srividya-sundaram reviewed Jan 2, 2025

View reviewed changes

clang/lib/Driver/ToolChains/SYCL.cpp Show resolved Hide resolved

jinge90 and others added 4 commits January 3, 2025 13:39

Update clang/test/Driver/sycl-device-lib-bfloat16.cpp

21fb49e

Co-authored-by: Michael Toguchi <[email protected]>

Update clang/test/Driver/sycl-device-lib-bfloat16.cpp

b1fe3b3

Co-authored-by: Michael Toguchi <[email protected]>

Update clang/test/Driver/sycl-device-lib-bfloat16.cpp

3c8b6d7

Co-authored-by: Michael Toguchi <[email protected]>

Update clang/test/Driver/sycl-device-lib-bfloat16.cpp

d6dd9e2

Co-authored-by: Michael Toguchi <[email protected]>

jinge90 temporarily deployed to WindowsCILock January 3, 2025 05:40 — with GitHub Actions Inactive

jinge90 temporarily deployed to WindowsCILock January 3, 2025 06:20 — with GitHub Actions Inactive

Support mix use of '-device xxx' and fsycl-targets=

45ae7e4

Signed-off-by: jinge90 <[email protected]>

jinge90 had a problem deploying to WindowsCILock January 11, 2025 16:21 — with GitHub Actions Error

Merge remote-tracking branch 'upstream/sycl' into link_native_bf16_aot

2cfd4ab

jinge90 temporarily deployed to WindowsCILock January 11, 2025 16:30 — with GitHub Actions Inactive

jinge90 temporarily deployed to WindowsCILock January 11, 2025 17:07 — with GitHub Actions Inactive

bashbaug approved these changes Jan 15, 2025

View reviewed changes

mdtoguchi reviewed Jan 16, 2025

View reviewed changes

jinge90 and others added 2 commits January 17, 2025 15:59

Merge remote-tracking branch 'upstream/sycl' into link_native_bf16_aot

34e4819

Update clang/lib/Driver/ToolChains/SYCL.cpp

b3b2a38

Co-authored-by: Michael Toguchi <[email protected]>

jinge90 had a problem deploying to WindowsCILock January 17, 2025 08:07 — with GitHub Actions Error

jinge90 temporarily deployed to WindowsCILock January 17, 2025 08:12 — with GitHub Actions Inactive

Merge remote-tracking branch 'origin/link_native_bf16_aot' into link_…

55f0e7c

…native_bf16_aot

jinge90 temporarily deployed to WindowsCILock January 17, 2025 08:48 — with GitHub Actions Inactive

jinge90 temporarily deployed to WindowsCILock January 20, 2025 01:54 — with GitHub Actions Inactive

Merge remote-tracking branch 'upstream/sycl' into link_native_bf16_aot

64ba41b

jinge90 had a problem deploying to WindowsCILock January 20, 2025 02:31 — with GitHub Actions Error

jinge90 temporarily deployed to WindowsCILock January 20, 2025 02:59 — with GitHub Actions Inactive

add test for -Xsycl-target-backend

23dc1de

Signed-off-by: jinge90 <[email protected]>

jinge90 temporarily deployed to WindowsCILock January 20, 2025 03:36 — with GitHub Actions Inactive

jinge90 requested a review from mdtoguchi January 20, 2025 09:10

jinge90 temporarily deployed to WindowsCILock January 21, 2025 07:54 — with GitHub Actions Inactive

add intel_gpu_dg2_*

c049ddd

Signed-off-by: jinge90 <[email protected]>

jinge90 temporarily deployed to WindowsCILock January 21, 2025 08:35 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SYCL][Driver] Support bfloat16 devicelib selection when multiple AOT targets specified #16494

[SYCL][Driver] Support bfloat16 devicelib selection when multiple AOT targets specified #16494

jinge90 commented Dec 31, 2024 •

edited

Loading

mdtoguchi Jan 2, 2025

jinge90 Jan 3, 2025

mdtoguchi Jan 6, 2025

jinge90 Jan 11, 2025

srividya-sundaram Jan 2, 2025 •

edited

Loading

jinge90 Jan 3, 2025

srividya-sundaram commented Jan 2, 2025

bashbaug Jan 15, 2025

jinge90 Jan 17, 2025

bashbaug Jan 22, 2025

jinge90 Jan 22, 2025

mdtoguchi Jan 16, 2025

jinge90 Jan 20, 2025

mdtoguchi Jan 16, 2025

jinge90 Jan 20, 2025

jinge90 commented Jan 21, 2025

[SYCL][Driver] Support bfloat16 devicelib selection when multiple AOT targets specified #16494

Are you sure you want to change the base?

[SYCL][Driver] Support bfloat16 devicelib selection when multiple AOT targets specified #16494

Conversation

jinge90 commented Dec 31, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

srividya-sundaram Jan 2, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

srividya-sundaram commented Jan 2, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jinge90 commented Jan 21, 2025

jinge90 commented Dec 31, 2024 •

edited

Loading

srividya-sundaram Jan 2, 2025 •

edited

Loading