02 / 06 · Count the traffic
Transfer Costs
Array dimensions determine the traffic. Bandwidth determines how quickly that traffic can move.
Transfer volume
Programming an array moves a whole matrix. Each MVM then moves only an input vector and an output vector. With 4-byte float32 values, the counts are:
| Transfer | Elements | Bytes |
|---|---|---|
| Program weights | mn | 4mn |
| Load input | n | 4n |
| Store output | m | 4m |
With m rows and n columns, the array consumes n input values and produces m output values per MVM. Adding columns therefore requires a larger input vector, while adding rows produces a larger output vector. Either change also increases the number of weights in the m × n matrix.
These are bytes, not cycles. Converting volume into time requires a bandwidth and a model of waiting.
What square-array scaling tells us
For square arrays, m = n = N. Doubling N produces four times the weight traffic but only twice the input and output traffic.
The graph converts this traffic into transfer time at bandwidth B, measured in bytes per cycle. The two lines show transfer time for the N × N weights and the N-element input vector. The N-element output vector has the same transfer time as the input.
Times are rounded up to whole cycles. These calculations cover transfer only, without array programming, analog execution, fixed overhead or contention. Increasing bandwidth lowers both curves; the axes stay fixed so the change remains visible.
256 × 256 array
Programming the array transfers 256 KiB of weights. Each MVM transfers 1 KiB into the input buffer and 1 KiB out of the output buffer.
2048 × 2048 array
Programming the array transfers 16 MiB of weights. Each MVM transfers 8 KiB into the input buffer and 8 KiB out of the output buffer.
One matrix transfer moves N/2 times as much data as one input–output pair. At 256 × 256, that is 128 pairs’ worth of traffic before the first result.
Parameters in the simulation
array_rows- Rows, m; output values per MVM.
array_cols- Columns, n; input values per MVM.