Two ways to connect the array

Separate input and output links

One shared link

05 / 06 · Balance bandwidth and computation

Required Bandwidth

Once programmed, how much bandwidth keeps one array computing instead of waiting?

Start with demand. An N × N float32 MVM consumes 4N input bytes and produces 4N output bytes. Divide by MVM time to budget continuous execution:

$B_{\text{separate}} \geq \frac{4N}{T_{\text{MVM}}}$ in each direction.

$B_{\text{shared}} \geq \frac{8N}{T_{\text{MVM}}}$ in total.

For 256 × 256 and 100 cycles, that is 10.24 B/cycle each way, or 20.48 B/cycle shared.

These graphs describe ideal steady-state operation: weights are already programmed, work is continuously available, and input transfer, execution and output transfer overlap perfectly. They exclude startup, request latency and scheduling delays.

1. Does the array stay busy?

Utilization is the share of time computing. A curve at 75% means 75 computing cycles per 100 elapsed cycles in the ideal model. The curve reaches 100% when bandwidth can supply inputs and remove outputs as fast as the array computes.

$U_{\text{ideal, separate}} = \min\left(1,\frac{BT_{\text{MVM}}}{4N}\right)$

$U_{\text{ideal, shared}} = \min\left(1,\frac{BT_{\text{MVM}}}{8N}\right)$

These are steady-state capacity bounds with continuous work and perfect overlap. Equal displayed B gives separate links twice the total capacity.

2. Does faster compute finish more work?

Throughput counts completed MVMs. Keep array size fixed, then compare execution times at the same bandwidth. High utilization alone is not the objective: a slower array can look busier and still complete less work.

The useful bandwidth is near each curve’s plateau. Beyond it, more link capacity adds little completed work.

Bandwidth for continuous computation

The maximum rate is $1/T_{\text{MVM}}$. The table gives the exact bandwidth where each ideal curve reaches that rate, rather than rounding up to a sampled setting.

Choose execution latency and delivery bandwidth together. A faster analog operation matters only if its inputs can arrive and its results can leave.

3. Is output limited by array size or link bandwidth?

Hold MVM time and bandwidth fixed. Moving right increases the array size, from 32 × 32 to 1024 × 1024; moving up delivers more output.

The rising region is array-size-bound: with MVM time fixed, a larger array produces more output per operation and therefore more output per cycle. The flat region is link-bandwidth-bound: the link cannot move data fast enough for a larger array to increase output throughput.

For 4-byte values, the balance point follows directly from MVM time and link bandwidth:

Independent links — $B$ bytes/cycle in each direction

$N_{\text{balance, independent}} = \dfrac{B\,T_{\text{MVM}}}{4}$

Shared link — $B$ bytes/cycle in total

$N_{\text{balance, shared}} = \dfrac{B\,T_{\text{MVM}}}{8}$

Here $T_{\text{MVM}}$ is the execution time in cycles, and $N$ is the number of rows and columns in the square array. Independent links each carry one $4N$-byte vector; a shared link carries both, totaling $8N$ bytes per MVM. These equations assume ideal overlap.

Below the balance point, increase array size to increase output. Above it, increase link bandwidth. This comparison holds MVM time fixed.

Utilization across array sizes and bandwidths

Separate links

The selected bandwidth is available in each direction: one link delivers inputs and the other removes outputs.

Shared link

The selected bandwidth is the total input/output budget. Both transfers share it. Equal B is not equal total hardware bandwidth.

Columns show the square-array dimension N; rows show bandwidth in bytes per cycle. Each cell gives the ideal percentage of time spent computing.

The maps have independent MVM-time controls and share the same 0–100% color scale. Match the MVM times to compare links. Select a cell to inspect that configuration in the utilization graph above.