Fast Hardware Simulation

Purpose: Validate RTL correctness faster than Vitis cosimulation using TAPA's fast cosim.

When to use this: After tapa compile produces a .xo file, before the multi-hour v++ --link step. Fast cosim catches logic bugs in generated RTL in seconds rather than the ten-plus minutes Vitis cosimulation requires.

What you need

• A .xo kernel object from tapa compile (or a .zip for the xilinx-hls target)
• One of:
  • xsim: Requires a Vivado installation. Linux only. No version floor.
  • verilator: Open-source. Works on Linux and macOS. No Vivado required. Must be 5.044 or newer — see below.

%Error: rtl/Stream2Mmap_mmap_m_axi.v:2046:27: Expecting expression to be
        constant, but can't determine constant for FUNCREF 'log2'

Commands

Basic run

Pass the .xo file as the --bitstream argument:

./vadd --bitstream VecAdd.xo 1000

For the xilinx-hls target, a .zip file also works:

./vadd --bitstream VecAdd.zip 1000

Choosing a simulator backend

The default backend is xsim. To switch to Verilator:

./vadd --bitstream VecAdd.xo -cosim_simulator verilator 1000

Saving waveforms

Specify a persistent work directory and enable waveform saving:

./vadd --bitstream VecAdd.xo \
    -cosim_work_dir ./cosim_work \
    -xsim_save_waveform \
    1000

Setup-only and resume workflow

When you want to inspect the generated simulation environment before committing to a full run:

# Step 1: set up the simulation environment and stop before running
./vadd --bitstream VecAdd.xo \
    -cosim_work_dir ./cosim_work \
    -cosim_setup_only \
    1000

# Step 2: after inspecting, run post-simulation checks without re-simulating
./vadd --bitstream VecAdd.xo \
    -cosim_work_dir ./cosim_work \
    -cosim_resume_from_post_sim \
    1000

Parallel runs

When a host application calls tapa::invoke more than once — for example, a pipeline split into separate kernels each compiled to its own .xo file — TAPA launches all cosim instances concurrently. Each kernel is compiled independently and its .xo path is passed to its own tapa::invoke call via a separate bitstream flag:

// Host code: two separate kernels, each with its own bitstream flag
DEFINE_string(producer_bitstream, "", "XO for Producer kernel");
DEFINE_string(consumer_bitstream, "", "XO for Consumer kernel");

tapa::invoke(Producer, FLAGS_producer_bitstream, ...);
tapa::invoke(Consumer, FLAGS_consumer_bitstream, ...);

./app --producer_bitstream=producer.xo --consumer_bitstream=consumer.xo

If all instances share the same -cosim_work_dir, their simulation environments collide. Pass -cosim_work_dir_parallel to give each instance its own uniquely named subdirectory:

./app \
    --producer_bitstream=producer.xo \
    --consumer_bitstream=consumer.xo \
    -cosim_work_dir ./cosim_work \
    -cosim_work_dir_parallel

TAPA creates ./cosim_work/XXXXXX/ (a unique name per instance) so that the simulations run without interfering with each other's build artifacts.

Runtime flags reference

The following flags control fast cosim behavior when passed to the host executable. The canonical reference is Runtime Flags.

Expected output

Fast cosim completes in seconds for simple designs. A successful run prints the application's correctness result (e.g., PASS!) after the simulation finishes.

Debugging frozen simulations

Fast cosim has no timeout by default: a large design may legitimately need hours, so the runtime cannot choose a safe universal limit. For unattended runs, set an explicit wall-clock limit:

./vadd --bitstream VecAdd.xo -cosim_timeout_seconds 3600 1000

On expiry, TAPA terminates the simulator process group and reports which limit was reached. A timeout does not distinguish a deadlock from a workload that is merely too large; use a reduced input first, then inspect the saved work directory if the reduced case also stalls.

If the simulation becomes unresponsive:

1. Run with -cosim_work_dir to persist intermediate files.
2. Abort the simulation with Ctrl-C.
3. Open Vivado in GUI mode and source the run script the work directory kept:

   vivado -mode gui -source [work-dir]/run_cosim.tcl
   

This allows real-time observation and waveform analysis of the frozen state.

The work directory holds the rest of what a post-mortem needs alongside it: tb_<TOP>.sv is the generated testbench, vivado.log is the transcript of the run that hung, and run/ is the Vivado project itself (run/cosim.sim/sim_1/behav/xsim/ is where -xsim_save_waveform writes wave.wdb).

If something goes wrong

Next step: Vitis Cosimulation