How to implement Systolic Array in DSLX

[Mars-Cat2023]

Hi!

1. How do we write things about "vector<int>" in our XLS[cc]? I would like to read and write values from and to xls channels (i.e., __xls_channel). Do we support that? Or other alternatives?
   Or at least arrays like "int A[10];" "int A[10][10];". And what other array features do we support (How to access? etc.) What's the syntax I should use? It seems like we cannot do that with normal C++ code. Could you give me some small examples for this?

2. Could you tell me which standard C++ library do we support in XLS[cc]? (Maybe not the same library name, but XLS[cc] version) If so, how can I import that into my .cc code?

[allight]

XLS[cc] is a compiler that targets hardware. Specifically, it targets hardware in the sense that the final output is verilog that you can (eventually) send to a fab to get built into a physical chip. Though its syntax is c++ it actually only supports a small subset of what a full C++ compiler does.

To answer your questions:

1. XLS[cc] does not (and cannot) support std::vector (or any of the other resizable containers for that matter). Again this targets hardware. You only have whatever memory you decide to build. Anything that uses dynamically allocated memory (std::new, malloc, etc) cannot be used. You can use arrays (including multi-dimensioned arrays) like normal in C++ (eg: https://github.com/google/xls/blob/main/xls/contrib/xlscc/unit_tests/translator_pointer_test.cc#L38). You can also hook up external rams and use those as arrays (eg: https://github.com/google/xls/blob/main/xls/contrib/xlscc/examples/mem_iterated_read.cc).

2. As I said xls[cc] is not a c++ compiler. It does not support any c++ standard library in the way that clang does. The available standard library headers are here: https://github.com/google/xls/tree/main/xls/contrib/xlscc/synth_only if you want to take a look though.

[Mars-Cat2023]

It's very helpful. Thanks for your answer!
I have another question:

1. I just want to know that why it takes very much space when I build bazel?
   I know that it will compile 52,944 setup files and create a new /.cache workspace for all your future XLS programming.
   You need to do this bazel building for every time your cloning of this xls github repo (normally, you just need to do this clone once and run this command once), right?
   It seems like that there is no "uninstall" option available (if there is, please let me know the command for uninstalling).
   Let me ask this: I install with command "bazel build -c opt xls/...". However, if I want to manual uninstall the whole command, is it enough to recover all the space of memory just by deleting the xls github repo and /.cache folder (~20 GB)? Or there are other folders created by this build command?

[ericastor]

You say you "install" with the command bazel build -c opt xls/.... A few points:

1. That's a build command, not an installation; it's the equivalent of make, not of make install. All the outputs are restricted to the locations where Bazel puts its build outputs; https://bazel.build/remote/output-directories documents where that will be. (As you've noted, on Linux it ends up in ~/.cache/bazel by default.) If you clean that up, the outputs are gone.
2. In terms of cleaning up a workspace if you're done (which I assume is what you mean by an "uninstall" option), Bazel supports this using the bazel clean command. As the linked docs say, that should reclaim the space! However...
3. When you rerun bazel build, even after a fresh git pull - as long as you haven't run bazel clean in the interim, you'll find that it caches rather a lot of the work. (As such, you probably want to avoid deleting the cache if you can!) The only times that should be nearly as expensive as the first are those where we've updated the revision of LLVM involved in building XLS[cc], and even those should have a decent amount of caching in play.
4. Part of why it's taking up a lot of space & time is that building XLS[cc] involves compiling Clang from source. LLVM is a large & complex project; it's part of why we recommend that users who aren't using XLS[cc] avoid building it.

[Mars-Cat2023]

Thanks for sharing this.
My new project will use XLS[cc] needs to use the vector and matrix multiplication.

Two problems I met:
1. How to do regular Systolic Array Matrix Multiplication in XLS[cc] with:
Two Int8 matrices
Two fp8 matrices - E4M3, something XLS should support
Two bf16 matrices
2. The size of #pragma unroll:
According to the warning I received from your XLS, the max iteration for any loop is 100.
Are there any suggestions about how to implement matrix multiplication (using methods such as the basic for loops) to support the dimensionality size of ~300 (like matrix multiply between two 300*300 matrices)?

[ericastor]

I think this is a question for XLS[cc]'s maintainer at this point, rather than the rest of the contributors, since XLS[cc] is more of its own thing. @spurserh, want to weigh in?

[allight]

1. @spurserh would know for sure but I think that xls[cc] only implements normal ieee single and double precision floating point numbers, and even there I think only for comparisons and constexpr. As I said though @spurserh would be the one to know.

There is an arbitrary precision float implementation in dslx if you want to use that language instead. Take a look at our docs to learn more: https://google.github.io/xls/floating_point/. There isn't really any supported way to mix the xlscc with dslx stuff. With enough effort you could probably get something hacked together though (maybe inline apfloat compiled routines into asm blocks).

2. Again @spurserh would know for sure but I think that warning is just that, a warning, and it can generate unrolled loops of any size. It might take an inordinately long time to do so and generate a huge amount of code however.

[spurserh]

0. Arrays are supported, are you having some trouble with them? Indeed, std::vector cannot be supported in the usual way, since dynamic memory allocation doesn't have a meaning in hardware design. That isn't really an XLS[cc] constraint: a physical register array or SRAM on a chip just is the size it is. You could create a class that acts like std::vector while having a statically set size limit, ie storage size, though. For example we have classes that implement FIFOs in a natural C++ way, they just have a hard limit on the maximum number of elements in the queue.

1. Arbitrary precision integers and fixed point numbers are provided for by the XlsInt and XlsFixed classes in the synth_only directory. With regard to IEEE floating point, it's not supported at all currently, but it's on the TODO. It's not a lot of work, as the plan is to use the DSLX floating point library, which does provide arbitrary precision. I'm not sure about BF16, but you can always implement custom data types as classes in C++. I think this is more along the lines of what you want for the data types you describe.

2. The max is not 100 iterations, that's just a warning meant to be helpful to the user if something goes wrong with loop unrolling. You can set or disable those limits with the --max_unroll_iters and --warn_unroll_iters arguments to xlscc (see xlscc/main.cc).

[Mars-Cat2023]

Hello team,
I decided to use DSLX to implement systolic array in DSLX by myself.
The provided code https://github.com/google/xls/blob/main/xls/examples/matmul_4x4/matmul_4x4.x doesn't looks good. It seems like it is not completed yet and the logic doesn't seems totally correct (at least from my point of view).

But could you give me more hint about the status of your tools (especially syntax):
You have this warning "This feature is currently WIP and is not yet available."
Q1: I want to know, up to this moment, which "parametric" features do you supported? Really hope that you can give me some examples about this code. At least, telling me which line code in this picture is supported, which is not?
Q2:

• Q2.1: Can I use this to connect all ((COLS + 1)*ROWS) channels in one step, I mean, for example, east_outputs[0][0] connected with west_inputs[0][0], east_output[0][1] connected with west_inputs[0][1], etc.?
  let (east_outputs, west_inputs) = chan<F32>[COLS + u32:1][ROWS]("east_west");
  If not, what should I do?
• Q2. 2: Can I use this way to declare the channels (Also see the picture)?
  input_producers: chan<F32>[4][4] out;
• Q2. 3: Can I use this? I am very confused about your warning "This feature is not yet available." So "this" refers to what? For loop? For loop for spawning? or Channels arrays?
  Again, if I cannot use them, what should I write here?
  for (i, _) : (u32, ()) in range(u32:0, u32:4) { for (j, _) : (u32, ()) in range(u32:0, u32:4) { spawn Node(input_consumers[i][j], output_producers[i][j]); }(()); }(());

---

Q3: As is stated here, I also agree that the proc matmul is the most difficult part of this question.
Now, I have already implemented the node / block for each subunit (we have 16 subunits for two 4*4 matrices multiplication to store the 16 numbers of multiplication result). That part is correct, and I want to (and need to) combine these 16 subunits within one proc called matmul like the code here shown, but I need to know how to do the parametric? and to which level I can do the parametric? to simplify my code?
Here is the old incomplete version provided by your /examples/ folder as I cited before, I will change based on this:

I appreciate all your help. And this will be very valuable for my project and I will be willing to contribute to you if my systolic array does work. Thanks very much.

[proppy]

@Mars-Cat2023 I started to update the matmul example in b59c99a to properly use array of channels and lower to IR, hopefull that's useful.

In order to do repeative channel operation and spawn one need to use unroll_for! which also need to be documented as described in #1708.

[Mars-Cat2023]

@proppy
Thanks to your reply and your contributions on this direction!!
May I ask:

• (1) Have you finished your update of that matmul example?
• (2) Could you tell me to which extent you support the Advanced Features on https://google.github.io/xls/tutorials/how_to_use_procs/? It seems like that this documentation was written in 2020, and not updated later? (So, which features I can use? For example, Channel arrays and loop-based spawning? Parametric procs?) If you have more new references, especially about DSLX syntax (and semantics), please feel free to share with me. It would be VERY helpful.

[proppy]

(1) Have you finished your update of that matmul example?

Yes, I think https://github.com/google/xls/blob/main/xls/examples/matmul_4x4/matmul_4x4.x is in a good shape, there is still a few workaround for pending issues (like #1750) but it does demonstrate idiomatic usage of unroll_for w/ spawn and channel arrays.

(2) Could you tell me to which extent you support the Advanced Features.

I believe it does support them all, I did remove the WIP notice in ff61af4#diff-0cebac07846720b6a37bd631028935ef30da5f2e781a8d23688571d4e148f45c.

Hope that helps.

[Mars-Cat2023]

Hello @proppy,
First say thanks to your contributions.

I found a "Bug" (or maybe my understanding is not correct about the syntax of new DSLX).

• (1) May I ask when declare the channels, should I declare the second dimensions before first dimensions (it sounds weird to me, so please check your code):

Inconsistency:
The order of two dimensions Yello != Orange

[Mars-Cat2023]

(2) Correct me if I am wrong: It seems like that .../dev_tools/repl doesn't support any file that has import <...>;, so we can directly use the interpreter_main to interpret the DSLX (and if it failed to compile, interpreter would give some errors)?

I can successfully run the .../interpreter_main, but when I run .../dev_tools/repl, it seems like it cannot find the correct path built-in library (such as , when using import float32; statement, like this:

[Mars-Cat2023]

(4) Any explanation for this weird order of declaration? Why the inverse order is correct?

[Mars-Cat2023]

(3) May I ask how can we write an array like A[6] = [4,2,1,3,3,5]; in DSLX? Otherwise, how can we write test_proc with arbitrary inputs easily?

[ericastor]

Responding to your questions (1) and (4), RE: order of array indexing - this has been discussed before, e.g., #616 (comment)

Roughly, the issue here is a mismatch between Rust (which inspired the syntax of DSLX) and the syntax of other languages, including both SystemVerilog and most C-style languages. Currently, when you index into an array like F32[3][5], you're indexing from the outside in. So: an F32[3][5] array is a 5-element array of F32[3]'s, which are in turn 3-element arrays of F32s.

Your confusion is one more data point suggesting that we need to fix this mismatch in some way!

[Mars-Cat2023]

Thanks for the above answer.

May I ask a follow-up question (@proppy), how you compile that DXLS code into IR?

I looked at the reference https://google.github.io/xls/tools_quick_start/, and Issue #983. However, when I compile with the following command
./bazel-bin/xls/dslx/ir_convert/ir_converter_main --top=matmul_3x3 ./matrixmul.x > ./matrixmul.ir
I received the error like this:
Error: NOT_FOUND: Top-level type info not found for proc "matmul_3x3".

Actually, I have very similar struct as your sample code matmul_4x4.x provided in your example code folder. (But my version is 3x3).

Here is snippet of this top function:

proc matmul_3x3 {
    config(a_in: chan<FP>[3] in, z_out: chan<FP>[3] out, weightmatrix: FP[3][3]) {
        spawn matmul<u32:3, u32:3>(a_in, z_out, weightmatrix);
    }

    init { () }

    next(state: ()) { }
}

Could you share how you solve this? How to get the Verilog from this dslx? Thanks.

[allight]

That is a really unfortunate and confusing error message (@richmckeever PTAL)

The issue is that when IR is generated it requires that the entire 'config' be const evaluated except for boundary channels. That means the 'top' config may only have 'chan' arguments. So the actual error is that weightmatrix is not a constant value or a channel.

If you want to set weights dynamically you'd need to send them down through channels.

[allight]

I created #1850 to track improving that error message FYI.

[Mars-Cat2023]

Thanks for your answer.

After checking your long libraries and , I realize that we don't have "Division" in current apfloat.
I know that implementing division is very very challenging, so I should prioritize floating-point (FP) additions, subtractions or multiplications. And I know some tricks like using right shift to substitute ÷(2^k), where k is an integer.

But I want to know is that is there any suggestions about how to implement FP division manually? Or any bypass I can do for this kind of operation?

[proppy]

There is one here: https://github.com/google/xls/blob/main/third_party/xls_berkeley_softfloat/fpdiv_2x32.x

@richmckeever should we look at integrating this w/ apfloat proper?