Configuration Reference

This page explains how to use the TOML configuration effectively, especially for PQC detection strategy control.

If you only read one page before tuning detection behavior, read this one.

Configuration model and precedence

pleb uses a flat config model (PipelineConfig) with mode-specific subcommands for ingest, workflow, and QC report.

Precedence is:

  1. config file values (TOML/JSON),

  2. CLI --set key=value overrides,

  3. direct CLI flags (where available).

When both TOML and CLI are supplied, CLI wins.

Where to find the full key list

There are many handles, so use these discovery paths:

  • Human summary: this page + Quickstart

  • Full exhaustive key-by-key chapter (with examples): Full Settings Catalog

  • Full field list and defaults: API Reference (pleb.config.PipelineConfig)

  • Programmatic dump of all keys/defaults:

python - <<'PY'
from dataclasses import fields
from pleb.config import PipelineConfig
cfg = PipelineConfig(
    home_dir='.',
    singularity_image='tempo2.sif',
    dataset_name='dataset'
)
for f in fields(PipelineConfig):
    print(f"{f.name} = {getattr(cfg, f.name)!r}")
PY

Detection strategy: mental model

There are three independent strategy layers:

  1. PQC detection layer (pqc_*): what detectors run and with which thresholds.

  2. FixDataset action layer (fix_qc_*): which QC columns should trigger comment/delete actions in tim files.

  3. QC report compact layer (qc_report_compact_*): which columns define compact PDF triage decisions.

Most confusion comes from mixing these layers. Keep them explicit.

TOML strategy controls (most important keys)

PQC detector configuration

  • run_pqc: enable PQC stage.

  • pqc_backend_col: backend grouping key (sys, group, etc.).

  • pqc_backend_profiles_path: per-backend threshold overrides (TOML map).

  • pqc_fdr_q / pqc_tau_corr_minutes: bad-measurement detector behavior.

  • pqc_robust_enabled / pqc_robust_z_thresh / pqc_robust_scope: MAD-style robust outlier behavior.

  • pqc_step_* / pqc_dm_step_*: step and DM-step event sensitivity.

  • pqc_solar_* / pqc_orbital_phase_* / pqc_eclipse_*: event-domain detectors.

  • pqc_gaussian_bump_* / pqc_glitch_*: transient feature detectors.

FixDataset outlier action strategy

  • run_fix_dataset + fix_apply: enable/apply fix stage.

  • fix_qc_remove_outliers: turn on QC-driven action.

  • fix_qc_outlier_cols: explicit outlier columns to act on.

  • fix_qc_action: comment or delete (default is non-destructive comment).

  • fix_qc_remove_bad / fix_qc_remove_transients / fix_qc_remove_solar / fix_qc_remove_orbital_phase: selective action toggles by detector family.

  • fix_flag_sys_freq_rules_enabled: enable YAML-based system/group frequency overrides.

  • fix_flag_sys_freq_rules_path: path to flag_sys_freq_rules.yaml.

Compact report strategy

  • qc_report: generate report stage.

  • qc_report_compact_pdf: write compact PDF.

  • qc_report_compact_outlier_cols: explicit compact outlier strategy.

  • qc_cross_pulsar_enabled: optional post-QC coincidence scan across pulsars.

  • qc_cross_pulsar_window_days: MJD coincidence window (days).

  • qc_cross_pulsar_min_pulsars: minimum distinct pulsars required.

  • qc_cross_pulsar_include_outliers / qc_cross_pulsar_include_events: include outlier/event classes in coincidence clustering.

Per-backend PQC profile TOML

pqc_backend_profiles_path points to a TOML file with this shape:

[backend_profiles]
"LOFAR.*" = { robust_z_thresh = 6.0, fdr_q = 0.02 }
"WSRT.P2.334" = { delta_chi2_thresh = 18.0, robust_z_thresh = 5.5 }

Match priority is:

  1. exact backend match,

  2. glob match (fnmatch).

Practical strategy profiles

Conservative (few false positives)

run_pqc = true
pqc_fdr_q = 0.005
pqc_robust_z_thresh = 6.5
pqc_delta_chi2_thresh = 25.0
fix_qc_remove_outliers = true
fix_qc_outlier_cols = ["bad_point", "robust_outlier", "robust_global_outlier", "bad_mad"]
fix_qc_action = "comment"

Aggressive (maximum sensitivity)

run_pqc = true
pqc_fdr_q = 0.02
pqc_robust_z_thresh = 4.5
pqc_delta_chi2_thresh = 12.0
pqc_step_delta_chi2_thresh = 12.0
pqc_dm_step_delta_chi2_thresh = 12.0
fix_qc_remove_outliers = true
fix_qc_outlier_cols = ["bad_point", "robust_outlier", "robust_global_outlier", "bad_mad", "bad_hard"]
fix_qc_action = "comment"

Common pitfalls

  • Setting run_pqc=true alone does not modify tim files. You also need FixDataset action keys.

  • Using outlier_any for action policy can be too broad because it may include event-related flags depending on context. Prefer explicit columns.

  • Forgetting fix_qc_results_dir/fix_qc_branch can make FixDataset look like it ignored QC (it may be reading from the wrong run location).

  • Extremely low thresholds + aggressive action can over-comment structured groups. Tune with report-only first.

  • Cross-pulsar coincidence is reporting-only and never edits tim/par files.