Run Simulation

Use kub-simulate for end-user runs. It supports Slurm or local launcher execution and provides status inspection commands. This tutorial uses Apptainer runtime for both launchers.

--cemdb-root can be omitted in many cases:

If the current directory contains ./cemdb, kub-dataset and kub-simulate use the default cemdb/locations.
kub-dashboard auto-detects ./cemdb.

CEMDB_ROOT environment variable:

kub-dashboard uses CEMDB_ROOT as its default CEMDB path.
For kub-dataset and kub-simulate, use --cemdb-root when your data is not under ./cemdb/locations.

Default launcher mode is auto:

if Slurm tools are available, kub-simulate submits with Slurm
otherwise it falls back to local command-line execution

1. 0. Configure `kub-simulate` Profiles (Recommended)

Use kub-simulate config to store user defaults and avoid repeating long command lines.

Do not store dataset version in config. Keep --version explicit on each command because it belongs to the selected dataset you are running.

# Create .kub-simulate.toml in the current directory
kub-simulate config init

# Shared default for preprocess/run
kub-simulate config set cemdb_root cemdb/locations

# Apptainer Slurm profile
kub-simulate config set launcher slurm --profile-target apptainer-slurm
kub-simulate config set partition public --profile-target apptainer-slurm
kub-simulate config set runtime apptainer --profile-target apptainer-slurm
kub-simulate config set apptainer_exec /opt/apptainer/latest/bin/apptainer --profile-target apptainer-slurm
kub-simulate config set apptainer_image ./kub-master.sif --profile-target apptainer-slurm

# Apptainer local profile
kub-simulate config set launcher local --profile-target apptainer-local
kub-simulate config set runtime apptainer --profile-target apptainer-local
kub-simulate config set apptainer_exec /opt/apptainer/latest/bin/apptainer --profile-target apptainer-local
kub-simulate config set apptainer_image ./kub-master.sif --profile-target apptainer-local

# Review resulting file
kub-simulate config show

Profiles are applied to kub-simulate preprocess and kub-simulate run. Ensure ./kub-master.sif exists before submitting (see Apptainer Runtime Workflow).

2. Why Run `preprocess` First

kub-simulate preprocess prepares partitioned preprocessing assets (mesh/data splits) for specific MPI sizes before simulation submission.

In practice, it helps you:

generate the partition layouts your runs will use (for example 32, 64, 128)
avoid spending simulation job time on partition preparation
keep runs reproducible by reusing the same prepared partitions

By default, preprocessing is incremental: existing partitions are reused. Use --force only when you need a full rebuild of preprocessing assets.

3. 1. Preview Preprocessing Submission

kub-simulate preprocess arz \
  --version 0.1.0 \
  --profile apptainer-slurm \
  --partitions 32 64 128 \
  --dry-run

Dry Run Recommended First
Direct Submit

Use --dry-run to print the generated Slurm script and verify paths, partition, and arguments before submitting.

Drop --dry-run once the generated script looks correct for your queue and policy constraints.

4. 2. Submit Preprocessing Job

kub-simulate preprocess arz \
  --version 0.1.0 \
  --profile apptainer-slurm \
  --partitions 32 64 128

Optional full rebuild of preprocessing assets:

kub-simulate preprocess arz \
  --version 0.1.0 \
  --profile apptainer-slurm \
  --force

5. 3. Submit Simulation Job

kub-simulate run arz \
  --version 0.1.0 \
  --profile apptainer-slurm \
  --np 128

Optional runtime override example:

kub-simulate run arz \
  --version 0.1.0 \
  --profile apptainer-slurm \
  --np 128 \
  --time-stop 172800

Local launcher alternative:

kub-simulate run arz \
  --version 0.1.0 \
  --profile apptainer-local \
  --np 128

6. 4. Inspect Status

kub-simulate status arz \
  --version 0.1.0 \
  --cemdb-root cemdb/locations \
  --last 10

Inspect a single run in detail:

kub-simulate status arz \
  --version 0.1.0 \
  --cemdb-root cemdb/locations \
  --run-id run_YYYY-MM-DD_HH-MM-SS

7. 5. Confirm Run Artifacts Exist

find cemdb/locations/arz/v0.1.0/simulations/single -maxdepth 2 -name manifest.json | sort

8. Next Step

Proceed to Apptainer Runtime Workflow.