Quickstart

This page shows a minimal end-to-end run that produces residual plots and diagnostic tables. It assumes you already have a tempo2-compatible data set and a working tempo2 installation or container image.

Config Layout Quick Reference

Use this map when choosing where to put files:

  • configs/runs/: runnable mode profiles passed to --config.

  • configs/workflows/: multi-step orchestration files passed to workflow --file.

  • configs/workflows/steps/: reusable workflow step configs.

  • configs/catalogs/: reusable mapping/lookup data.

  • configs/rules/: declarative behavior policies (relabel/overlap/PQC profiles).

  • configs/state/lockfiles/: generated ingest lock snapshots/validation.

  • configs/settings/: legacy location; do not add new files.

For full layout guidance, see Configuration Layout.

Minimal configuration

Create a pipeline.toml file:

home_dir = "/data/epta"
singularity_image = "/images/tempo2.sif"
dataset_name = "EPTA"
branches = ["main", "EPTA"]
reference_branch = "main"
results_dir = "results"
jobs = 4

Run the pipeline

pleb --config pipeline.toml

CLI overrides

Any key from the TOML config can be supplied directly as a CLI flag and will override the file value. These are equivalent:

pleb --config pipeline.toml --jobs 8 --run_fix_dataset
pleb --config pipeline.toml --set jobs=8 --set run_fix_dataset=true

Boolean keys accept --key (true) and --no-key (false).

The pipeline prints a run tag pointing to the output directory. Inside that directory you will find:

  • Residual plots grouped by backend and pulsar.

  • Change reports comparing the configured branches.

  • Summary tables and QC artifacts (if enabled).

Optional QC stage

If you have pqc and libstempo installed, you can enable outlier and transient detection:

pleb --config pipeline.toml --qc

The QC stage is designed to highlight residual outliers, transient events, and structured behavior in residuals versus time, frequency, and orbital phase. [Coles2011] [Keith2013]

Declarative relabel rules

For post-QC backend splitting (for example, dividing one -sys into epoch sub-groups), point FixDataset at a TOML rules file:

pleb --config pipeline.toml \
  --set run_fix_dataset=true \
  --set fix_relabel_rules_path="configs/rules/relabel/relabel_rules.example.toml"

Declarative overlap rules

For duplicate-preference policies (for example, prefer dual-channel WSRT P2 rows over paired single-channel rows), use overlap rules:

pleb --config pipeline.toml \
  --set run_fix_dataset=true \
  --set fix_overlap_rules_path="configs/rules/overlap/overlap_rules.example.toml"

Per-backend PQC profiles

To override PQC thresholds for selected backends while keeping global defaults as fallback:

pleb --config pipeline.toml \
  --set run_pqc=true \
  --set pqc_backend_col="sys" \
  --set pqc_backend_profiles_path="configs/rules/pqc/backend_profiles.example.toml"

TOML-configurable outlier strategies

You can now choose the outlier decision strategy directly in TOML by listing which QC columns should be treated as outlier evidence.

pleb --config pipeline.toml \
  --set run_fix_dataset=true \
  --set fix_qc_remove_outliers=true \
  --set fix_qc_outlier_cols='["bad_point","robust_outlier","robust_global_outlier","bad_mad"]' \
  --set qc_report=true \
  --set qc_report_compact_pdf=true \
  --set qc_report_compact_outlier_cols='["bad_point","robust_outlier","robust_global_outlier","bad_mad"]'

Compact PDF report

To generate a compact final PDF summary of likely bad TOAs (with plots):

pleb --config pipeline.toml \
  --set qc_report=true \
  --set qc_report_compact_pdf=true \
  --set qc_report_compact_pdf_name="qc_compact_report.pdf"

The compact report also writes per-backend action CSVs under <qc_report_dir>/action_lists/ with exact flagged TOAs and reason columns.

Exact-overlap keep/drop catalog

The exact duplicate overlap map is loaded from configs/catalogs/system_tables/overlapped_timfiles.toml by default. To override:

pleb --config pipeline.toml \
  --set run_fix_dataset=true \
  --set fix_overlap_exact_catalog_path="configs/catalogs/system_tables/overlapped_timfiles.toml"

Next steps