process_holography

A prefect2 based pipeline to process holography data taken by ASKAP. It will use components of the aces module, which will be parallelised in a beam-wise manner where possible. The output data products are FITS files that are compatible with the linmos package of yandasoft.

Module Contents

Functions

get_model_resource(→ str)	Look up and return a path to a model file that corresponds to a known bright source that
get_holography_src(→ str)	Identifies the source that holography was carried out against by examining a measurement set. This intended to
task_beamset_from_ms(→ None)	A wrapper around the beamset_from_ms main entrypoint that converts MSs to a
task_divmodel(→ None)	A wrapper around the divmodel main entry point that corrects holography
task_dump_point(→ None)	A wrapper around the dump_point main entry point that extracts pointing information
task_grid_holography(→ None)	A wrapper around the grid_holography main entrypoint, which will grid and interpolate holography data.
task_merge_holography(→ None)	A wrapper around the merge_holography main entry point that collates holgraphy results together.
task_clean_holography(→ None)	A wrapper around the cleaan_holography main entry point that corrects holography
task_plot_summary(→ None)	A wrapper around the plot_summary main entry point to create visualisations of the holograhy data.
get_cal_holo_mslist(→ list[str])	Get list of calibrated MSs from holography dir
task_copy_final_beams(→ None)	Copy the constructed beams from the working directory to a final location
flow_holography(→ None)	The prefect workflow to execute the holography processing and analysis pipeline.
main(sbid[, workdir, grid_ant, cluster, beam_path, ...])	The main driver of the holography pipeline procedure. This entrypoint is responsible for
cli()

Attributes

KNOWN_HOLO_MODELS
logger

process_holography.KNOWN_HOLO_MODELS: tuple[str] = ('virgo',)

process_holography.logger

process_holography.get_model_resource(holo_src: str) → str

Look up and return a path to a model file that corresponds to a known bright source that holography was performed against.

Args:

holo_src (str): Name of the target source, as retreived from the field table of a measurement set

Raises:

ValueError: Raised when a model file is expected but not found through package resources.

Returns:

str: Path to a model file to use for a known source.

process_holography.get_holography_src(ms: str | pathlib.Path) → str

Identifies the source that holography was carried out against by examining a measurement set. This intended to be used fo any additional processing/corrections (such as the case of virgo)

Args:

ms (Union[str,Path]): A measurement set to inspect to identify the target source holography was carried out against

Returns:

str: Name of the target source

process_holography.task_beamset_from_ms(*args, **kwargs) → None

A wrapper around the beamset_from_ms main entrypoint that converts MSs to a beamset. All args and kwargs are passed into aces.holography.beamset_from_ms.main.

process_holography.task_divmodel(*args, **kwargs) → None

A wrapper around the divmodel main entry point that corrects holography data. All args and kwargs are passed into aces.holography.divmodel.main.

process_holography.task_dump_point(*args, **kwargs) → None

A wrapper around the dump_point main entry point that extracts pointing information from an MS. All args and kwargs are passed into aces.holography.dump_point.main.

process_holography.task_grid_holography(beam: int, sbid: int, holo_dir: str, ant: int) → None

A wrapper around the grid_holography main entrypoint, which will grid and interpolate holography data.

Args:

beam (int): The beam number that will be processed from the beamset sbid (int): The SBID number of the holography data to process holo_dir (str): The path of the data that will be processed ant (int): Reference antenna used throughout the processing

process_holography.task_merge_holography(*args, **kwargs) → None

A wrapper around the merge_holography main entry point that collates holgraphy results together. All args and kwargs are passed into aces.holography.merge_holography.main.

process_holography.task_clean_holography(*args, **kwargs) → None

A wrapper around the cleaan_holography main entry point that corrects holography data from RFI. All args and kwargs are passed into aces.holography.clean_holography.main.

process_holography.task_plot_summary(*args, **kwargs) → None

A wrapper around the plot_summary main entry point to create visualisations of the holograhy data. data. All args and kwargs are passed into aces.holography.plot_summmary.main.

process_holography.get_cal_holo_mslist(sbid: int, workdir: str) → list[str]

Get list of calibrated MSs from holography dir

Args:

sbid (int): Holography SBID workdir (str): Working directory

Returns:

list[str]: Calibrated holography MSs

process_holography.task_copy_final_beams(final_path: pathlib.Path, sbid: int, dry_run: bool = False) → None

Copy the constructed beams from the working directory to a final location

Args:

final_path (Path): Path to install the constructed beams into. This must exists and be a directory. sbid (int): The SBID of the holography observation, which is used when constructing the files to copy. dry_run (bool, optional): If True, the copying process is only reported, and no actual copying is performed. Defaults to False.

process_holography.flow_holography(workdir: str, sbid: int, grid_ant: int | None = None, beam_path: pathlib.Path = None) → None

The prefect workflow to execute the holography processing and analysis pipeline.

This flow brings together the main stages that are implemented in other scripts, and wraps them up with the prefect logic to allow them to be called.

The workflow expects that in the current working directory there to be a folder whose name is the sbid to be processed, and the measurement sets to be processed have been copied there already

Args:

workdir (str): Where the data processing will be carried out, including creation of products sbid (int): SBID number of the target holography observation to be procesed grid_ant (Optional[int], optional): Reference antenna used in observation. If None, attempts are made to dderive it from the data. Defaults to None. beam_path (Path, optional): The final location to copy the output beams to. If None, no copying is performed. Defaults to None.

process_holography.main(sbid: int, workdir: pathlib.Path | None = None, grid_ant: int | None = None, cluster: str = 'galaxy_small', beam_path: pathlib.Path = None, schedule_block_path: pathlib.Path = None, emit_clink: bool = False)

The main driver of the holography pipeline procedure. This entrypoint is responsible for

• creating a dask task runner for use between subflows

• run the sub-flow to find and copy measurement sets, if requested

• run the sub-flow to perform the actual holography processing

• emit a clink even if requested on completion of the two sub-flows

Args:

sbid (int): SBID of the observation used to collect the holography data workdir (Optional[Path], optional): The directory that the holography data will be processed in. There is an attempt to change to this directory. Defaults to None. grid_ant (Optional[int], optional): Antenna number that was the ‘reference’ antenna during the holography observation. Defaults to None. cluster (str, optional): The name of the cluster configuration file to load. Defaults to “galaxy_small”. beam_path (Path, optional): Path to install the constructed holography beams into. If None, the beams are not copied. Defaults to None. schedule_block_path (Path, optional): Path to the space on disk where the Measurement sets are stored (inside a SBID folder). If not None and the SBID folder exists with the 36 measurementsets, these are copied over and delete the measurments in the workdir, if they exists. Defaults to None. emit_clink (bool, optional): Whether a clink event message will be raised to process manager

process_holography.cli()