SME1 kernel support for AI workloads using ArmNN

[hpc-qti]

Hi @morgolock,

In continuation to the PR 1206
Need support for SME1 kernels in ComputeLibrary in arm_gemm

AI workloads using ArmNN library needs SME1 equivalent kernels in ComputeLibrary. This enables SME based acceleration for applications like Geekbench AI workloads - computer vision tests, NLP tests etc.

Below is the list of kernels identified that will be needed for the above use-case.

sme1_interleaved_nomerge_bf16fp32_mopa_1VLx4VL
sme1_interleaved_nomerge_bf16fp32_mopa_2VLx2VL
sme1_interleaved_nomerge_bf16fp32_mopa_4VLx1VL
sme1_interleaved_nomerge_fp16fp32_mopa_1VLx4VL
sme1_interleaved_nomerge_fp16fp32_mopa_2VLx2VL
sme1_interleaved_nomerge_fp16fp32_mopa_4VLx1VL
sme1_interleaved_nomerge_fp16fp32fp16_mopa_1VLx4VL
sme1_interleaved_nomerge_fp16fp32fp16_mopa_2VLx2VL
sme1_interleaved_nomerge_fp16fp32fp16_mopa_4VLx1VL
sme1_interleaved_nomerge_s8q_mopa_1VLx4VL
sme1_interleaved_nomerge_s8q_mopa_2VLx2VL
sme1_interleaved_nomerge_s8q_mopa_4VLx1VL
sme1_interleaved_nomerge_s8qfp32_mopa_1VLx4VL
sme1_interleaved_nomerge_s8qfp32_mopa_2VLx2VL
sme1_interleaved_nomerge_s8qfp32_mopa_4VLx1VL
sme1_interleaved_nomerge_s8s32_mopa_1VLx4VL
sme1_interleaved_nomerge_s8s32_mopa_2VLx2VL
sme1_interleaved_nomerge_s8s32_mopa_4VLx1VL
sme1_interleaved_nomerge_u8q_mopa_1VLx4VL
sme1_interleaved_nomerge_u8q_mopa_2VLx2VL
sme1_interleaved_nomerge_u8q_mopa_4VLx1VL

sme1_gemv_bf16fp32_dot_16VL
sme1_gemv_fp16_mla_16VL
sme1_gemv_fp16fp32fp16_dot_16VL
sme1_gemv_fp32_mla_16VL
sme1_gemv_fp32bf16fp32_dot_16VL
sme1_gemv_s8qa_dot_16VL
sme1_gemv_u8qa_dot_16VL

[morgolock]

Hi @hpc-qti

Thanks for raising this, I'll discuss it with the team.

[hpc-qti]

Hi @morgolock ,
Is there any update on this request?

[morgolock]

Hi @hpc-qti

When do you need these kernels?

I checked with the team, we're in the middle of a project reorganisation, so it’s hard to say when we’ll be able to deliver these kernels in ACL.

Hope this helps.

[hpc-qti]

Hi @morgolock ,

Thanks for the response. Since there is a tool to generate the kernels, can you help to provide the kernels by this month(Dec 2025)?

[morgolock]

HI @hpc-qti

Thanks, I'll discuss it with the team and see if we can land these kernels in December. As I explained previously, we are in the middle of project reorganization which means this could take longer than expected.

Thanks

[hpc-qti]

Hi @morgolock,

Is there any update on the kernels?

[morgolock]

Hi @hpc-qti

We are working on it, I'd expect this to be up for review in a couple of weeks max.

Sorry for the delay, between the Christmas holidays and the project reorg it was difficult to upload this in December.

[gunes-arm]

Update. The PR that will address this is up for review: #1248

[hpc-qti]

Thanks for the update. We will review the PR and get back if there are any issues.
The interleave function for the below kernels are with SME2 support.
Kernels:
sme_interleaved_nomerge_bf16fp32_mopa_1VLx4VL
sme_interleaved_nomerge_bf16fp32_mopa_2VLx2VL
sme_interleaved_nomerge_bf16fp32_mopa_4VLx1VL
Interleave functions:
sme2_interleave1VL_block2_fp32_bf16
sme2_interleave2VL_block2_fp32_bf16
sme2_interleave4VL_block2_fp32_bf16

Hence please help to add these functions which are required with SME variants for the usecase.

[hpc-qti]

Hi @morgolock

Do you have any comment on the above request?

[morgolock]

Hi @hpc-qti

I'll discuss this with the team and get back to you.

[hpc-qti]

Hi @morgolock ,

Please provide an update on above request

[gunes-arm]

Hi @hpc-qti , sorry for the radio silence on this. The requested kernels require much bigger refactrorings due to how templated implementations work. What's the level of importance of these implementations for your case?

[hpc-qti]

Hi @gunes-arm ,

Thanks for the response. The requested kernels are used by almost all the Geekbench AI FP32 workloads. Without these, SME kernels are not performant compared to NEON. Hence pls enable SME based bf16fp32 kernels for fp32 usecase.