:py:mod:`survey_status`
=======================
.. py:module:: survey_status
.. autoapi-nested-parse::
#!/usr/bin/env python
Module Contents
---------------
Functions
~~~~~~~~~
.. autoapisummary::
survey_status.vprint
survey_status.arg_init
survey_status.dec2deg
survey_status.from_ra_dec_deg
survey_status.get_scan_info_from_ms
survey_status.parset_to_field_table
survey_status.obs_to_field_table
survey_status.save_db
survey_status.get_desc_for_epoch
survey_status.main
Attributes
~~~~~~~~~~
.. autoapisummary::
survey_status.explanation
survey_status.survey_colnames
survey_status.survey_coltypes
survey_status.desc_colnames
survey_status.desc_coltypes
survey_status.states
survey_status.not_observed
survey_status.observed
survey_status.imaged
survey_status.global_verbose
.. py:data:: explanation
:value: Multiline-String
.. raw:: html
Show Value
.. code-block:: python
"""
survey_status.py is the script to keep track of the bookkeeping for ASKAP surveys.
It is a prototype, derived from the equivalent software written for RACS.
----------------------------------------------------------------------
This script initiates the database for an epoch of observations for an ASKAP survey.
It also allows updates to be made with each new observing session.
The life cycle of this database is as follows.
1. Creation from observing parset. This can be done before the observation. The given parset must hold the
full specification of this epoch survey; that is all fields are defined even if only a subset are observed in any
given scheduling block.
2. Add observing information; from the field to SBID mapping provided, the MeasurementSets are read for information
that is added to the database.
3. Add image information.
The database consists of a description table and a set of directories containing other tables. For each survey epoch
there is a row in the description file, and a directory of tables.
In each epoch directory are tables, including:
- field_table : with a row for each field observation (within a single epoch a field observation maybe repeated).
- beam_table : one table for each row in the field_table, each with a row for each beam.
- other tables as needed.
(1) To create a database and initialise the field_table
survey_status.py -s -e -p
(2) To update the database with observation information
survey_status.py -s -e -s
Steps 1 and 2 can be performed with a single execution as
survey_status.py -s -e -p -s
(3) To update the database with image-specific information, selected according to the image_info argument:
M - mosaic image
B - beam images
S - image statistics
A - astrometry (beam-beam)
C - catalogue compare
P - polarization leakage
first csv file to keep track of the observations. This
is run by calling:
survey_status.py -s -e -p -s
- .parset is the parset used to create the observations.
- .csv is the observing record (original from Aidan). However,
easy to re-create. The file needs to be in CSV format and contain the columns:
id,field,selected,scheduled,sbid,observed,processed,quality,casda
For example:
10,RACS_test4_1.05_0805-71A,,,,,,,
100,RACS_test4_1.05_2131-56A,1,1,8676,1,,,
The files created are:
descriptor - created if necessary, appended otherwise; has columns:
EPOCH, OBS_FREQ, FOOTPRINT, PITCH, ROTATION, DURATION, POL_MODE, DB_FILE
data_base directory that contains:
field_data - the master csv file for the bookkeeping with columns:
INDEX, SRC, FIELD_NAME, SBID, SCAN, CAL_SBID, STATE, RA_HMS, DEC_DMS, RA_DEG, DEC_DEG,
GAL_LONG, GAL_LAT, POL_AXIS, SCAN_START, SCAN_LEN, MED_RMS_uJy,
MODE_RMS_uJy, STD_RMS_uJy, MIN_RMS_uJy, MAX_RMS_uJy,
PSF_MAJOR, PSF_MINOR, PSF_ANGLE
beam_data - for each field observation (row in field_data):
BEAM_NUM, DATE, RA_HMS, DEC_DMS, RA_DEG, DEC_DEG, GAL_LONG, GAL_LAT,
PSF_MAJOR, PSF_MINOR, PSF_ANGLE
The last 8 columns will be empty as these fill up after calibration and imaging.
"""
.. raw:: html
Other columns of data that would be useful:
1. Information about data products:
- date produced
- software version
- how many beam (valid) images for each Stokes parameter (MFS)
- how many (valid) mosaic images for each Stokes parameter (MFS)
- how many cubes for each Stokes parameter, and whether reduced to cubelets.
survey_status.py -p survey_epoch_<#>.parset -s survey_record_epoch_<#>.csv -g .csv -u
- survey_epoch_<#>.parset is the parset used to create the observations.
- survey_record_epoch_<#>.csv is the observing record (original from Aidan). However,
easy to re-create. The file needs to be in CSV format and contain the columns:
id,field,selected,scheduled,sbid,observed,processed,quality,casda
For example:
10,RACS_test4_1.05_0805-71A,,,,,,,
100,RACS_test4_1.05_2131-56A,1,1,8676,1,,,
- .csv is the updated bookkeeping csv file. Originally this file would be
downloaded from google sheets as this is where the modifying is done as the processing
of the fields commences. This will have the columns:
SRC FIELD_NAME, SBID, STATE, RA_HMS, DEC_DMS, RA_DEG, DEC_DEG, GAL_LONG, GAL_LAT,
OBS_FREQ, FOOTPRINT, PITCH, ROTATION, DURATION, POL_MODE, POL_AXIS, SB_TIME,
SB_START_DATE, SB_START_TIME, SB_END_DATE, SB_END_TIME, CAL_SBID, MED_RMS_uJy,
MODE_RMS_uJy, STD_RMS_uJy, MIN_RMS_uJy, MAX_RMS_uJy, PSF_MAJOR, PSF_MINOR, PSF_ANGLE
Using the -u (--update) lets you update the csv file with the new fields that have been observed.
Will created a new csv file with the same information as before, just with new SBIDs if required.
The required input are:
- survey_epoch_<#>.parset is the parset used to create the observations.
- survey_record_epoch_<#>.csv is the observing record (original from Aidan). However,
easy to re-create. The file needs to be in CSV format and contain the columns:
id,field,selected,scheduled,sbid,observed,processed,quality,casda
For example:
10,RACS_test4_1.05_0805-71A,,,,,,,
100,RACS_test4_1.05_2131-56A,1,1,8676,1,,,
- .csv is the updated bookkeeping csv file. Originally this file would be
downloaded from google sheets as this is where the modifying is done as the processing
of the fields commences. This will have the columns:
SRC FIELD_NAME, SBID, STATE, RA_HMS, DEC_DMS, RA_DEG, DEC_DEG, GAL_LONG, GAL_LAT,
OBS_FREQ, FOOTPRINT, PITCH, ROTATION, DURATION, POL_MODE, POL_AXIS, SB_TIME,
SB_START_DATE, SB_START_TIME, SB_END_DATE, SB_END_TIME, CAL_SBID, MED_RMS_uJy,
MODE_RMS_uJy, STD_RMS_uJy, MIN_RMS_uJy, MAX_RMS_uJy, PSF_MAJOR, PSF_MINOR, PSF_ANGLE
Using the -u (--update) lets you update the csv file with the new fields that have been observed.
Using the -a (--analysis) lets you update the csv file with the new fields that have been observed.
- survey_stats_.csv is the output file from survey_analysis.py which returns the image
statistics. The file has columns:
FLD_NAME,MED_RMS_uJy,MODE_RMS_uJy,STD_RMS_uJy,MIN_RMS_uJy,MAX_RMS_uJy,PSF_MAJ,PSF_MIN,PSF_PA
.. py:data:: survey_colnames
:value: ['SURVEY_NAME']
.. py:data:: survey_coltypes
:value: ['S12']
.. py:data:: desc_colnames
:value: ['EPOCH', 'OBS_FREQ', 'FOOTPRINT', 'PITCH', 'ROTATION', 'DURATION', 'POL_MODE', 'DATABASE',...
.. py:data:: desc_coltypes
:value: ['i4', 'f4', 'S20', 'f4', 'f4', 'f4', 'S12', 'S80', 'S80']
.. py:data:: states
.. py:data:: not_observed
.. py:data:: observed
.. py:data:: imaged
.. py:data:: global_verbose
:value: False
.. py:function:: vprint(*args)
.. py:function:: arg_init()
.. py:function:: dec2deg(ra_hms, dec_dms)
DEC2DEG - convert coordinates to decimal degrees and galactic coordinates
:param ra_hms: RA in hh:mm:ss.sssss
:param dec_dms: DEC in dd:mm.ss.ssss
:return: [RA in degrees, DEC in degrees, Long in degrees, Lat in degrees]
.. py:function:: from_ra_dec_deg(ra, dec)
.. py:function:: get_scan_info_from_ms(sb)
.. py:function:: parset_to_field_table(parset_file)
.. py:function:: obs_to_field_table(field_table, mapping_file)
.. py:function:: save_db(survey_tab, survey_file, desc_tab, desc_file, field_tab, field_file)
.. py:function:: get_desc_for_epoch(table, epoch)
.. py:function:: main()