Stage 2

Created on Thurs Jul 21 17:33 2022

@author: MCR

Custom JWST DMS pipeline steps for Stage 2 (Spectroscopic processing).

class exotedrf.stage2.AssignWCSStep(datafiles, output_dir='./')

Wrapper around default calwebb_spec2 Assign WCS step.

run(save_results=True, force_redo=False, **kwargs): Method to run the step.

class exotedrf.stage2.BackgroundStep(input_data, baseline_ints, background_model, output_dir='./')

Wrapper around custom Background Subtraction step.

run(save_results=True, force_redo=False, do_plot=False, show_plot=False, **kwargs): Method to run the step.

class exotedrf.stage2.BadPixStep(input_data, baseline_ints, output_dir='./')

Wrapper around custom Bad Pixel Correction Step.

run(space_thresh=15, time_thresh=10, box_size=5, save_results=True, force_redo=False, do_plot=False, show_plot=False): Method to run the step.

class exotedrf.stage2.FlatFieldStep(datafiles, output_dir='./')

Wrapper around default calwebb_spec2 Flat Field Correction step.

run(save_results=True, force_redo=False, **kwargs): Method to run the step.

class exotedrf.stage2.SourceTypeStep(datafiles, output_dir='./')

Wrapper around default calwebb_spec2 Source Type Determination step.

run(save_results=True, force_redo=False, **kwargs): Method to run the step.

class exotedrf.stage2.TracingStep(input_data, deepframe, output_dir='./')

Wrapper around custom Tracing Step.

run(pixel_flags=None, generate_order0_mask=True, f277w=None, calculate_stability=True, pca_components=10, save_results=True, force_redo=False, generate_lc=True, baseline_ints=None, smoothing_scale=None, do_plot=False, show_plot=False): Method to run the step.

exotedrf.stage2.backgroundstep(datafiles, background_model, baseline_ints, output_dir='./', save_results=True, fileroots=None, fileroot_noseg='', scale1=None, background_coords1=None, scale2=None, background_coords2=None, differential=False)

Background subtraction must be carefully treated with SOSS observations. Due to the extent of the PSF wings, there are very few, if any, non-illuminated pixels to serve as a sky region. Furthermore, the zodi background has a unique stepped shape, which would render a constant background subtraction ill-advised. Therefore, a background subtracton is performed by scaling a model background to the counts level of a median stack of the exposure. This scaled model background is then subtracted from each integration.

Parameters:

datafiles (array-like[str], array-like[CubeModel]) – Paths to data segments for a SOSS exposure, or the datamodels themselves.
background_model (array-like[float]) – Background model. Should be 2D (dimy, dimx)
baseline_ints (array-like[int]) – Integrations of ingress and egress.
output_dir (str) – Directory to which to save outputs.
save_results (bool) – If True, save outputs to file.
fileroots (array-like[str]) – Root names for output files.
fileroot_noseg (str) – Root name with no segment information.
scale1 (float, array-like[float], None) – Scaling value to apply to background model to match data. Will take precedence over calculated scaling value. If applied at group level, length of scaling array must equal ngroup.
background_coords1 (array-like[int], None) – Region of frame to use to estimate the background. Must be 1D: [x_low, x_up, y_low, y_up].
scale2 (float, array-like[float], None) – Scaling value to apply to background model to match post-step data. Will take precedence over calculated scaling value. If applied at group level, length of scaling array must equal ngroup.
background_coords2 (array-like[int], None) – Region of frame to use to estimate the post-step background. Must be 1D: [x_low, x_up, y_low, y_up].
differential (bool) – if True, calculate the background scaling seperately for the pre- and post-step frame.

Returns:

results (array-like[CubeModel]) – Input data segments, corrected for the background.
model_scaled (array-like[float]) – Background model, scaled to the flux level of each group median.

exotedrf.stage2.badpixstep(datafiles, baseline_ints, space_thresh=15, time_thresh=10, box_size=5, output_dir='./', save_results=True, fileroots=None, fileroot_noseg='', do_plot=False, show_plot=False)

Identify and correct outlier pixels remaining in the dataset, using both a spatial and temporal approach. First, find spatial outlier pixels in the median stack and correct them in each integration via the median of a box of surrounding pixels. Then flag outlier pixels in the temporal direction and again replace with the surrounding median in time.

Parameters:

datafiles (array-like[str], array-like[RampModel]) – List of paths to datafiles for each segment, or the datamodels themselves.
baseline_ints (array-like[int]) – Integrations of ingress and egress.
space_thresh (int) – Sigma threshold for a deviant pixel to be flagged spatially.
time_thresh (int) – Sigma threshold for a deviant pixel to be flagged temporally.
box_size (int) – Size of box around each pixel to test for deviations.
output_dir (str) – Directory to which to output results.
save_results (bool) – If True, save results to file.
fileroots (array-like[str], None) – Root names for output files.
fileroot_noseg (str) – Root file name with no segment information.
do_plot (bool) – If True, do the step diagnostic plot.
show_plot (bool) – If True, show the step diagnostic plot instead of/in addition to saving it to file.

Returns:

data (list[CubeModel]) – Input datamodels for each segment, corrected for outlier pixels.
deepframe (array-like[float]) – Final median stack of all outlier corrected integrations.

exotedrf.stage2.make_order0_mask_from_f277w(f277w, thresh_std=1, thresh_size=10)

Locate order 0 contaminants from background stars using an F277W filter: exposure data frame.

Parameters:

f277w (array-like[float]) – An F277W filter exposure, superbias and background subtracted.
thresh_std (int) – Threshold above which a group of pixels will be flagged.
thresh_size (int) – Size of pixel group to be considered an order 0.

Returns:

mask – Frame with locations of order 0 contaminants.

Return type:

array-like[int]

exotedrf.stage2.run_stage2(results, background_model, baseline_ints, save_results=True, force_redo=False, space_thresh=15, time_thresh=15, calculate_stability=True, pca_components=10, timeseries=None, timeseries_o2=None, oof_method='scale-achromatic', root_dir='./', output_tag='', smoothing_scale=None, skip_steps=None, generate_lc=True, inner_mask_width=40, outer_mask_width=70, pixel_masks=None, generate_order0_mask=True, f277w=None, do_plot=False, show_plot=False, centroids=None, **kwargs)

Run the exoTEDRF Stage 2 pipeline: spectroscopic processing, using a combination of official STScI DMS and custom steps. Documentation for the official DMS steps can be found here: https://jwst-pipeline.readthedocs.io/en/latest/jwst/pipeline/calwebb_spec2.html

Parameters:

results (array-like[str], array-like[CubeModel]) – exoTEDRF Stage 1 output files.
background_model (array-like[float]) – SOSS background model.
baseline_ints (array-like[int]) – Integrations of ingress and egress.
save_results (bool) – If True, save results of each step to file.
force_redo (bool) – If True, redo steps even if outputs files are already present.
space_thresh (int) – Sigma threshold for pixel to be flagged as an outlier spatially.
time_thresh (int) – Sigma threshold for pixel to be flagged as an outlier temporally.
calculate_stability (bool) – If True, calculate the stability of the SOSS trace over the course of the TSO using a PCA method.
pca_components (int) – Number of PCA components to calculate.
timeseries (array-like[float], None) – Normalized 1D or 2D light curve(s) for order 1.
timeseries_o2 (array-like[float], None) –

Normalized 2D light curves for order 2. Only necessary if oof_method
is “scale-chromatic”.
oof_method (str) – 1/f correction method. Options are “scale-chromatic”, “scale-achromatic”, “scale-achromatic-window”, or “solve”.
root_dir (str) – Directory from which all relative paths are defined.
output_tag (str) – Name tag to append to pipeline outputs directory.
smoothing_scale (int, None) – Timescale on which to smooth the lightcurve.
skip_steps (array-like[str], None) – Step names to skip (if any).
generate_lc (bool) – If True, produce a smoothed order 1 white light curve.
inner_mask_width (int) – Inner mask width, in pixels, around the trace centroids.
outer_mask_width (int) – Outer mask width, in pixels, around the trace centroids.
pixel_masks (None, str, array-like[str]) – Paths to files containing existing pixel flags to which the trace mask should be added. Only necesssary if generate_tracemask is True.
generate_order0_mask (bool) – If True, generate a mask of order 0 cotaminants using an F277W filter exposure.
f277w (None, str, array-like[float]) – F277W filter exposure which has been superbias and background corrected. Only necessary if generate_order0_mask is True.
do_plot (bool) – If True, make step diagnostic plots.
show_plot (bool) – Only necessary if do_plot is True. Show the diagnostic plots in addition to/instead of saving to file.
centroids (str, None) – Path to file containing trace positions for all orders.

Returns:

results – Datafiles for each segment processed through Stage 2.

Return type:

array-like[CubeModel]

exotedrf.stage2.soss_stability_pca(cube, n_components=10, outfile=None, do_plot=False, show_plot=False)

Calculate the stability of the SOSS trace over the course of a TSO using a PCA method.

Parameters:

cube (array-like[float]) – Cube of TSO data.
n_components (int) – Maximum number of principle components to calcaulte.
outfile (None, str) – File to which to save plot.
do_plot (bool) – If True, do the step diagnostic plot.
show_plot (bool) – If True, show the step diagnostic plot instead of/in addition to saving it to file.

Returns:

pcs (array-like[float]) – Extracted principle components.
var (array-like[float]) – Explained variance of each principle component.

exotedrf.stage2.tracingstep(datafiles, deepframe=None, calculate_stability=True, pca_components=10, pixel_flags=None, generate_order0_mask=False, f277w=None, generate_lc=True, baseline_ints=None, smoothing_scale=None, output_dir='./', save_results=True, fileroot_noseg='', do_plot=False, show_plot=False)

A multipurpose step to perform some initial analysis of the 2D dataframes and produce products which can be useful in further reduction iterations. The four functionalities are detailed below: 1. Locate the centroids of all three SOSS orders via the edgetrigger algorithm. 2. (optional) Generate a mask of order 0 contaminants from background stars. 3. (optional) Calculate the stability of the SOSS traces over the course of the TSO. 4. (optional) Create a smoothed estimate of the order 1 white light curve.

Parameters:

datafiles (array-like[str], array-like[RampModel]) – List of paths to datafiles for each segment, or the datamodels themselves.
deepframe (str, array-like[float], None) – Path to median stack file, or the median stack itself. Should be 2D (dimy, dimx). If None is passed, one will be generated.
calculate_stability (bool) – If True, calculate the stabilty of the SOSS trace over the TSO using a PCA method.
pca_components (int) – Number of PCA stability components to calcaulte.
pixel_flags (None, str, array-like[str]) – Paths to files containing existing pixel flags to which the trace mask should be added. Only necesssary if generate_tracemask is True.
generate_order0_mask (bool) – If True, generate a mask of order 0 cotaminants using an F277W filter exposure.
f277w (None, str, array-like[float]) – F277W filter exposure which has been superbias and background corrected. Only necessary if generate_order0_mask is True.
generate_lc (bool) – If True, also produce a smoothed order 1 white light curve.
baseline_ints (array-like[int]) – Integrations of ingress and egress. Only necessary if generate_lc=True.
smoothing_scale (int, None) – Timescale on which to smooth the lightcurve. Only necessary if generate_lc=True.
output_dir (str) – Directory to which to save outputs.
save_results (bool) – If Tre, save results to file.
fileroot_noseg (str) – Root file name with no segment information.
do_plot (bool) – If True, do the step diagnostic plot.
show_plot (bool) – If True, show the step diagnostic plot instead of/in addition to saving it to file.

Returns:

centroids (array-like[float]) – Trace centroids for all three orders.
order0mask (array-like[bool], None) – If requested, the order 0 mask.
smoothed_lc (array-like[float], None) – If requested, the smoothed order 1 white light curve.