Add ARM FP8 detection#413
Conversation
| #define CPUINFO_ARM_LINUX_FEATURE2_SME_B16B16 UINT64_C(0x0000020000000000) | ||
| #define CPUINFO_ARM_LINUX_FEATURE2_SME_F16F16 UINT64_C(0x0000040000000000) | ||
| #define CPUINFO_ARM_LINUX_FEATURE2_FP8 UINT64_C(0x0008000000000000) | ||
| #define CPUINFO_ARM_LINUX_FEATURE2_F8MM UINT64_C(0x0010000000000000) |
There was a problem hiding this comment.
Here this is HWCAP2_F8FMA eg (1UL << 52).
Should we rename this to F8FMA?
Or use one of these:
#define HWCAP_F8MM8 (1UL << 35)
#define HWCAP_F8MM4 (1UL << 36)
There was a problem hiding this comment.
no for 2 reasons
- we are not planning to use fma and its okay for that feature to not be supported. (but it is supported)
- for consistency with BF16 which has a general type. But my intent is that if FP8 is supported but dot product and matrix multiply are not, then fallback on F8CVT
if you dont like FP8 as a type by itself (like BF16) then explicitely add F8CVT instead would be my preference. But that is less consistent with BF16 and FP16
There was a problem hiding this comment.
Sorry if I was not clear.
I mean that
CPUINFO_ARM_LINUX_FEATURE2_F8MM
Is currently set to
UINT64_C(0x0010000000000000)
Which it I understand correctly from my link is actually
HWCAP2_F8FMA
In the Linux header.
So it seems like the code as is is checking for fma
There was a problem hiding this comment.
Done. Switched to HWCAP_F8MM8 which is beyond the 32 bit int, but we can check directly:
if (hwcap_val & (1UL << 35)) { // HWCAP_F8MM8
fbarchard
force-pushed
the
fp8
branch
2 times, most recently
from
288e767 to
a716bc1
Compare
| *hwcap = (uint32_t)hwcap_val; | ||
| *hwcap2 = getauxval(AT_HWCAP2); | ||
| if ((uint64_t)hwcap_val & (UINT64_C(1) << 35)) { | ||
| *hwcap2 |= CPUINFO_ARM_LINUX_FEATURE2_F8MM; |
There was a problem hiding this comment.
We are checking hwcap_val, and then setting our local hwcap2 here.
So, we are basically stuffing something from feature into feature2 in cpuinfo, right?
What if that bit is already set for a different reason, eg
hwcap2_val & (UINT64_C(1) << 35) == true
already, but hwcap_val & (UINT64_C(1) << 35) == false?
We would report FP8MM support but it won't actually have it.
If the problem is the underlying linux feature bit fields are too wide, do we just need to widen the features/features2 fields in cpuinfo here?
There was a problem hiding this comment.
widening hwcap to uint64_t is likely the cleaner solution, but a bigger change.
fbarchard
changed the title
Expose ARM v8.7 / Armv9.6-A FP8 instructions across public API structs, Linux HWCAP2 / /proc/cpuinfo parsing, Apple sysctls, Windows processor feature checks, and isa_info: - Base FP8 (cpuinfo_has_arm_fp8 / FEAT_FP8 / HWCAP2_F8CVT / F8E4M3 / F8E5M2) - FP8 4-way Dot Product (cpuinfo_has_arm_f8dot / FEAT_FP8DOT4 / HWCAP2_F8DP4) - FP8 8-way Matrix Multiply-Accumulate into FP32 (cpuinfo_has_arm_f8mm / FEAT_F8F32MM / HWCAP_F8MM8) Fixes pytorch#412
2 participants
Add ARM v8.7 / Armv9.6-A FP8 instructions across public API structs, Linux HWCAP2 / /proc/cpuinfo parsing, Apple sysctls, Windows processor feature checks, and isa_info:
cpuinfo_has_arm_fp8/FEAT_FP8/HWCAP2_F8CVT)cpuinfo_has_arm_f8dot/FEAT_FP8DOT4/HWCAP2_F8DP4)cpuinfo_has_arm_f8mm/FEAT_F8F32MM/HWCAP_F8MM8/HWCAP2_F8FMA)Fixes #412