Quickstart
==========

This page shows a minimal end-to-end run that produces residual plots and
diagnostic tables. It assumes you already have a tempo2-compatible data set
and a working tempo2 installation or container image.

Config Layout Quick Reference
-----------------------------

Use this map when choosing where to put files:

- ``configs/runs/``: runnable mode profiles passed to ``--config``.
- ``configs/workflows/``: multi-step orchestration files passed to ``workflow --file``.
- ``configs/workflows/steps/``: reusable workflow step configs.
- ``configs/catalogs/``: reusable mapping/lookup data.
- ``configs/rules/``: declarative behavior policies (relabel/overlap/PQC profiles).
- ``configs/state/lockfiles/``: generated ingest lock snapshots/validation.
- ``configs/settings/``: legacy location; do not add new files.

For full layout guidance, see :doc:`config_layout`.

Minimal configuration
---------------------

Create a ``pipeline.toml`` file:

.. code-block:: toml

   home_dir = "/data/epta"
   singularity_image = "/images/tempo2.sif"
   dataset_name = "EPTA"
   branches = ["main", "EPTA"]
   reference_branch = "main"
   results_dir = "results"
   jobs = 4

Run the pipeline
----------------

.. code-block:: bash

   pleb --config pipeline.toml

CLI overrides
-------------

Any key from the TOML config can be supplied directly as a CLI flag and will
override the file value. These are equivalent:

.. code-block:: bash

   pleb --config pipeline.toml --jobs 8 --run_fix_dataset
   pleb --config pipeline.toml --set jobs=8 --set run_fix_dataset=true

Boolean keys accept ``--key`` (true) and ``--no-key`` (false).

The pipeline prints a run tag pointing to the output directory. Inside that
directory you will find:

- Residual plots grouped by backend and pulsar.
- Change reports comparing the configured branches.
- Summary tables and QC artifacts (if enabled).

Optional QC stage
-----------------

If you have ``pqc`` and ``libstempo`` installed, you can enable outlier and
transient detection:

.. code-block:: bash

   pleb --config pipeline.toml --qc

The QC stage is designed to highlight residual outliers, transient events,
and structured behavior in residuals versus time, frequency, and orbital
phase. [Coles2011]_ [Keith2013]_

Declarative relabel rules
-------------------------

For post-QC backend splitting (for example, dividing one ``-sys`` into epoch
sub-groups), point FixDataset at a TOML rules file:

.. code-block:: bash

   pleb --config pipeline.toml \
     --set run_fix_dataset=true \
     --set fix_relabel_rules_path="configs/rules/relabel/relabel_rules.example.toml"

Declarative overlap rules
-------------------------

For duplicate-preference policies (for example, prefer dual-channel WSRT P2
rows over paired single-channel rows), use overlap rules:

.. code-block:: bash

   pleb --config pipeline.toml \
     --set run_fix_dataset=true \
     --set fix_overlap_rules_path="configs/rules/overlap/overlap_rules.example.toml"

Per-backend PQC profiles
------------------------

To override PQC thresholds for selected backends while keeping global defaults
as fallback:

.. code-block:: bash

   pleb --config pipeline.toml \
     --set run_pqc=true \
     --set pqc_backend_col="sys" \
     --set pqc_backend_profiles_path="configs/rules/pqc/backend_profiles.example.toml"

TOML-configurable outlier strategies
------------------------------------

You can now choose the outlier decision strategy directly in TOML by listing
which QC columns should be treated as outlier evidence.

.. code-block:: bash

   pleb --config pipeline.toml \
     --set run_fix_dataset=true \
     --set fix_qc_remove_outliers=true \
     --set fix_qc_outlier_cols='["bad_point","robust_outlier","robust_global_outlier","bad_mad"]' \
     --set qc_report=true \
     --set qc_report_compact_pdf=true \
     --set qc_report_compact_outlier_cols='["bad_point","robust_outlier","robust_global_outlier","bad_mad"]'

Compact PDF report
------------------

To generate a compact final PDF summary of likely bad TOAs (with plots):

.. code-block:: bash

   pleb --config pipeline.toml \
     --set qc_report=true \
     --set qc_report_compact_pdf=true \
     --set qc_report_compact_pdf_name="qc_compact_report.pdf"

The compact report also writes per-backend action CSVs under
``<qc_report_dir>/action_lists/`` with exact flagged TOAs and reason columns.

Exact-overlap keep/drop catalog
-------------------------------

The exact duplicate overlap map is loaded from
``configs/catalogs/system_tables/overlapped_timfiles.toml`` by default. To override:

.. code-block:: bash

   pleb --config pipeline.toml \
     --set run_fix_dataset=true \
     --set fix_overlap_exact_catalog_path="configs/catalogs/system_tables/overlapped_timfiles.toml"

Next steps
----------

- For details on the CLI, see :doc:`cli`.
- For practical tuning of TOML settings and detection strategies, see
  :doc:`configuration_reference`.
- For statistical context (chi-square, red noise, DM variations), see
  :doc:`concepts`.