AI Engine Tasks
On a Versal device, a TAPA task can run on the AI Engine array instead of the programmable logic (PL). The task keeps its C++ body; marking it moves it to an AI Engine tile, and every stream between it and a PL task becomes a PLIO connection. Software simulation is unchanged: an AI Engine task runs as an ordinary task.
The working example is tests/functional/aie-vadd/.
tests/functional/aie-roundtrip/ exercises two AIE stages separated by a PL
stage, covering PL → AIE → PL → AIE → PL data movement.
Marking a task
Mark a leaf task, or an upper task to put its whole subtree on the AI Engine:
using vec4f = tapa::vec_t<float, 4>;
[[tapa::target("aie")]]
void Add(tapa::istream<vec4f>& a, tapa::istream<vec4f>& b,
tapa::ostream<vec4f>& c, int n) {
[[tapa::pipeline]] for (int i = 0; i < n; ++i) c.write(a.read() + b.read());
}
The same choice can be made on the command line without editing the source, so one source builds both a PL-only and a PL+AI Engine design. The mapping wins over the attribute:
tapa analyze ... --target xilinx-vitis --target Add:aie
TASK:TARGET takes aie, hls or ignore; an unknown task name is an
error. Inside an AI Engine upper task, a descendant explicitly marked for
another target is an error. The top task cannot be an AI Engine task: the PL
kernel owns the control interface. A hybrid top must also keep at least one
scalar or memory (mmap) argument: a top with only stream ports packages as
ap_ctrl_none, a free-running kernel XRT can neither start nor wait on, so
tapa synth rejects it. (A PL-only design may still be a stream top; it just
cannot be run through XRT.)
What an AI Engine task may use
| Construct | On the AI Engine |
|---|---|
istream, ostream, istreams, ostreams | Yes. Elements must be a multiple of 32 bits. |
read(), write(), >>, <<, read(dst, n), write(src, n) | Yes, blocking. |
close, open, eot, try_eot, TAPA_WHILE_*EOT | No. AI Engine streams carry no end-of-transmission token: loop over an element count. |
try_read, try_write | Accepted, but they block on the AI Engine. Use them only where the schedule guarantees progress. |
peek, empty, full | No. AI Engine streams expose no occupancy. |
mmap, async_mmap, mmaps, hmap | No. Read memory in a PL task and stream the data in. |
| Scalars | Yes. |
| Local variables and arrays | Yes. They live on the kernel's stack, which the graph sizes as ADF's default 1 KB plus every local the body declares. The stack must fit in the tile's 32 KB data memory. |
[[tapa::pipeline]] | Lowered to chess_prepare_for_pipelining; every nested loop is flattened automatically, as in HLS nested-loop pipelining. |
[[tapa::unroll]] | Lowered to chess_flatten_loop when full and chess_unroll_loop(k) with a factor. |
tapa::wait(), tapa::reg() | Accepted. As in software simulation, wait does nothing and reg returns its argument: the core has no PL clock cycle to yield. |
HLS-only attributes such as partition, dependence, and tripcount | Ignored with a remark; they still affect the PL build. |
A PL task must not close() a stream that feeds an AI Engine task, because
the end-of-transmission token would be read as data.
An AI Engine task that a cycle of streams runs through, such as a ring of tasks or a round trip through a PL task, keeps every stream access behind its earlier writes, the order software simulation runs it in. The AI Engine compiler would otherwise issue a read ahead of an earlier write, which the cycle turns into a deadlock. The price is less overlap between the loop iterations of that task; tasks off any cycle keep it. HLS keeps no such order for a PL task: it may issue a read in the same cycle as an earlier write, or ahead of it.
tapa::fence() keeps every stream access after it behind every one before
it: on the AI Engine as a scheduler barrier, in HLS as a clock boundary, and
in software simulation, which runs in program order, as a no-op. A ring that
also builds for PL fences its writes from its reads and writes no more
between its reads than its streams hold. A source whose tasks
place one between each loop that writes a cycle and the loop that reads it
may define TAPA_AIE_PHASE_ORDERED_STREAMS before including tapa.h; those
tasks then skip the fence after every write, which is what lets a loop that
computes also write the cycle at full speed.
These habits decide AI Engine performance:
- Write the inner loop with
tapa::vec_t. On the AI Engine a nativevec_t<float, 4>is one vector register,a + bis one vector add, andacc += a * bis one multiply-accumulate. Other whole-wordvec_tshapes use scalar operations; scalar loops run far slower than native vector code. - Use a 32-bit loop counter and
[[tapa::pipeline]]on the hot loop. A 64-bit counter keeps the loop from being software-pipelined. Pipeline the loops that fill and drain local arrays too, so that a short loop nested in them, such as one over the parts of a vector, is flattened as well. - Accumulate into a local
vec_tvariable, not an array element. The AI Engine compiler keeps an array in memory, so each multiply-accumulate would wait on a load and a store. - Move whole vectors rather than single lanes. Reading or writing one lane of
a
vec_tcosts several instructions and a branch, even in an unrolled loop, and in the hot loop it costs more than the arithmetic it feeds. - A lane that only feeds a multiply is the exception:
acc += x[k] * bbroadcasts lanekof a non-const nativexinside the multiply-accumulate, with no extract. Copyxout of its array into a local first, so it is loaded once for all its lanes.tests/apps/cannonkeeps A, B and C as rows of vectors and multiplies each row of B by one lane of A this way. - Give each chain of multiply-accumulates its own accumulator. A float
multiply-accumulate into the result of the one before it issues every
second cycle; independent accumulators fill the cycles in between.
tests/apps/cannoncomputes four vectors of a row of C per iteration, one accumulator each. - Move a frame port's data whole, with
read(dst, n)andwrite(src, n). They test for a frame boundary once per run of a frame, whereread()andwrite()test twice per element, and the DMA moves the frames while the core computes; the core's own stream ports move only 32 bits per cycle.tests/apps/cannoncarries its ring in frames this way: each round but the last, a PE writes its blocks of A and B whole, multiplies, and then reads its neighbors' blocks whole. Its blocks from and to the PL travel as two halves on two frame ports, so over two PLIO channels at once.
Knobs
| Attribute | Where | Effect |
|---|---|---|
[[tapa::aie_frame(N)]] | istream/ostream parameter of an AI Engine leaf | The stream arrives as frames of N elements in tile memory, filled by DMA, instead of through the core's stream port. The body is unchanged. A final partial output frame is zero-padded; an input frame releases and discards unused padding when the task returns. |
[[tapa::aie_plio(W)]] | istream/ostream parameter of an AI Engine task | PLIO width W of 32, 64 or 128 bits for a boundary stream. |
[[tapa::aie_location(col, row)]] | AI Engine leaf task | Pins the task to a tile. Add a third argument to pick one instance of a task invoked several times. |
A task built for PL ignores these attributes, so a source annotated for the AI
Engine still builds for PL when --target leaves the task there.
Where the two targets want different code, guard the AI Engine side with
#if defined(TAPA_TARGET_AIE_). The AI Engine compile defines it, on x86sim
as on hardware; software simulation, tapacc and HLS compile the other branch.
tapacc therefore neither checks nor lowers the guarded code, so it should use
only ports and locals the other branch also uses, and loop attributes such as
[[tapa::pipeline]] take no effect there. tests/apps/cannon guards where its
PEs send their blocks: whole before multiplying on the AI Engine, a chunk at a
time while multiplying on PL.
Simulating the AI Engine tasks
Software simulation runs AI Engine tasks as ordinary tasks. To check the AI Engine build of those tasks, record the streams a software simulation moves and replay the ones that cross into the AI Engine:
mkdir logs
TAPA_STREAM_LOG_DIR=logs ./vadd-host 256
tapa --work-dir work.out analyze -f vadd.cpp -t VecAdd
tapa --work-dir work.out aiesim --stimulus logs --arg n=256 \
--part-num xcvc1902-vsva2197-2MP-e-S
tapa aiesim generates the ADF graph under work.out/aie/sim/, compiles it
for x86simulator, feeds each PL-to-AI Engine stream from its
<stream>.words log (zero-padded to a whole frame and PLIO beat), and compares
every AI Engine-to-PL stream with its log. Only zero padding is ignored;
surplus or nonzero output is an error.
A boundary stream is found by its variable name, so name the stream after its
variable: tapa::stream<vec4f> a_q("a_q"). Pass every top-task scalar the AI
Engine tasks read with --arg.
--cycle-accurate builds for hardware and runs aiesimulator instead, and
prints when the first and last beat of each output stream arrived and the mean
spacing between beats:
c_q: 1024 of 1024 words, beats at 230.4..1862.4 ns (6.40 ns apart)
AI Engine simulation matches software simulation
When the work directory holds the report.json / report.yaml a previous
tapa synth wrote, a run in which every output stream matches also folds
that timing into their aie block under aiesim, preserving the rest of
the report:
aie:
aiesim:
streams:
c_q:
beats: 256
first_beat_ns: 230.4
last_beat_ns: 1862.4
mean_beat_spacing_ns: 6.4
A stream with a single beat reports no mean_beat_spacing_ns, since it has
no interval to average. tapa aiesim also works on a work directory that
has only been analyzed, which has no report yet; then no report is created
and the timing stays on the console. Reports from before a later analyze
or a failed synth are stale and are not updated. A run with any output
mismatch leaves both files untouched.
On an AI Engine 1 device, a kernel reads about 32 bits from its input streams
per AI Engine cycle in total, however many stream ports it reads. That is one
128-bit beat every 3.2 ns at 1.25 GHz, so the two-input add above runs at
6.4 ns per output beat. Measure each kernel this way rather than planning on
the port count. A frame port ([[tapa::aie_frame(N)]]) is filled by DMA and
does not consume that budget.
Hybrid cosimulation
The PL half of the design can run as RTL while the AI Engine tasks run in
software, driven by the unchanged host program. Build the PL kernel without
the AI Engine graph and pass the .xo as the bitstream:
tapa --work-dir work.out compile -f vadd.cpp -t VecAdd \
--platform xilinx_vck190_base_202520_1 --skip-aie -o vadd.xo
./vadd-host --bitstream=vadd.xo 256
The kernel leaves one AXIS stream per AI Engine stream to the host. TAPA runs
the top task's body, binds each of those streams by name, and runs in
software the invocations on the AI Engine side of them. The PL invocations run
in the kernel, as in any .xo cosimulation. As with tapa aiesim, name each
boundary stream after its variable.
Building and linking
tapa compile (or analyze, synth, pack) builds both halves. The flow
must be xilinx-vitis, because v++ links the AI Engine graph with the PL
kernel:
tapa --work-dir work.out compile -f vadd.cpp -t VecAdd \
--platform xilinx_vck190_base_202520_1 -o vadd.xo
tapa link --package vadd.xo -o vadd.xclbin
-
tapa synthcompiles the AI Engine tasks intowork.out/aie/hw/libadf.awithaiecompiler --target=hw, which needs the AI Engine compiler licence, and synthesizes only the PL tasks. Each PL-to-AI Engine stream becomes an AXIS port of the PL kernel named after the stream, and its PLIO runs at the PL kernel clock. It warns about each such stream whose elements are narrower than 128 bits: the PL moves one element per cycle, and only a 128-bit PLIO keeps up with an AI Engine stream port.After a successful hardware AI Engine build,
report.jsonandreport.yamlhave a root-levelaieblock. It recordstiles_used, aplio_mapwith each logical PLIO's direction, interface column, width and consumed 64-bit channels, andper_kerneltile and program-memory usage. PL-only builds and--skip-aieomit this block. For the supported AIE1 flow, synth also logs each occupied interface column's usage against 8 PL-to-AI Engine and 6 AI Engine-to-PL 64-bit channels; a 128-bit PLIO consumes two channels. A mapping over either limit is an error. -
tapa packwrites the PL kernel asvadd.xoand, next to it, the graph asvadd.libadf.aand the PLIO connectivity asvadd.aie.cfg, onestream_connectper boundary stream between the kernel's compute unit (named after the top task) andai_engine_0. A VCK5000 build also writesvadd.aie_control.xo, a PL kernel that controls the graph through a dedicated PLIO; no APU control program is needed. -
tapa linkfinds the sidecars next to the.xo, links the graph, PL XOs and connectivity into an.xsa, and packages that into the.xclbin.
Iterating on the AI Engine only
After tapa analyze, rebuild just the hardware graph with the usual device and
clock arguments:
tapa --work-dir work.out synth --aie-only \
--platform xilinx_vck190_base_202520_1
--aie-only is the counterpart of --skip-aie, and the two flags cannot be
combined. It requires an AI Engine partition and does not stage or run HLS,
generate RTL, run post-synthesis utilization, or print a PL timing summary. A
successful build replaces work.out/aie/hw/libadf.a and makes a later pack
expect its sidecar; a failed build does not turn a full build into an
intentional PL-only package.
Existing PL results and floorplan state remain reusable only if the resolved
part number, platform, and requested clock period are identical to those of
the previous PL synthesis. Repeating AIE-only with those unchanged inputs
preserves flow.synthed=true and the floorplan. Changing any input clears both
the synthesized flag and floorplan, so run a full tapa synth before packing.
A work directory invalidated by tapa analyze is never promoted to synthesized
by AIE-only. In every case the typed PL metrics already in tapa.json are
preserved rather than replaced with zero or missing measurements.
If valid synthesized PL state and current report.json / report.yaml files
exist, AIE-only refreshes them from those preserved metrics and the new hardware
mapping. The rewrite removes aie.aiesim: run tapa aiesim --cycle-accurate
again to measure the new graph. Without valid synthesized PL state, AIE-only
does not invent a PL report or refresh stale reports; it also does not create
root reports when none exist. The compiler output and console remain available
until a later full PL synth writes the root report.
A stream into the AI Engine asserts TLAST on every beat. The AI Engine interface would otherwise hold a lone 32-bit word until a second one arrives.
Running the xclbin
tapa::invoke on the .xclbin runs both halves through XRT, with the
unchanged host program. Before it starts the PL kernel, the runtime writes
each scalar argument an AI Engine task reads into that task's runtime
parameter and runs the graph for one iteration. Once the PL kernel is done,
it waits for the graph to finish. A graph with a detached task runs until the
PL kernel is done and is then stopped; the next tapa::invoke resets it, so
its tasks start again from the top. The reported kernel time is still the PL
kernel's, from start to done.
VCK190 uses XRT's native graph API. On VCK5000, FRT instead sends register
writes to tapa_aie_pl_control in the PL: runtime parameters and the iteration
count before the PL kernel starts, then core-disable after it completes. No
APU control program or host xrtGraphOpen call is used. The control packets
broadcast to per-kernel packet controllers. FRT writes every distinct
iteration-register address, requires single-buffered RTPs, and rejects
colliding register addresses rather than controlling an ambiguous layout.