Utils

Created on Wed Jul 20 08:51 2022

@author: MCR

Miscellaneous pipeline tools.

exotedrf.utils.collapse_f277w_exposure(data, outdir='./')

Do rough 1/f noise removal and collapse an F277W exposure cube to make the final calibrated data product.

Parameters:

  • data (jwst.datamodel) – F277W exposure datamodel after ramp fitting.

  • outdir (str) – Directory to which to save outputs.

exotedrf.utils.convert_flux_units(wave, flux)

Convert extracted flux from units of MJy (output by the pipeline by default) to erg/s/cm^2/µm. Flux should still be scaled by (Rstar/Dist)**2 to correct for flux at Earth effect.

Parameters:

  • wave (array-like(float)) – Wavelength array in µm

  • flux (array-like(float)) – Flux array in MJy. Should be the same size as wave.

Returns:

flux_converted – Flux array converted to erg/s/cm^2/µm.

Return type:

ndarray(float)

exotedrf.utils.do_replacement(frame, badpix_map, dq=None, xbox_size=5, ybox_size=0)

Replace flagged pixels with the median of a surrounding box.

Parameters:

  • frame (array-like[float]) – Data frame.

  • badpix_map (array-like[bool]) – Map of pixels to be replaced.

  • dq (array-like[int]) – Data quality flags.

  • xbox_size (int) – Size of box to consider – x-direction.

  • ybox_size (int) – Size of box to consider – y-direction.

Returns:

  • frame_out (array-like[float]) – Input frame wth pixels interpolated.

  • dq_out (array-like[int]) – Input dq map with interpolated pixels set to zero.

exotedrf.utils.download_stellar_spectra(st_teff, st_logg, st_met, outdir, silent=False)

Download a grid of PHOENIX model stellar spectra.

Parameters:

  • st_teff (float) – Stellar effective temperature.

  • st_logg (float) – Stellar log surface gravity.

  • st_met (float) – Stellar metallicity as [Fe/H].

  • outdir (str) – Output directory.

  • silent (bool) – If True, do not show any prints.

Returns:

  • wfile (str) – Path to wavelength file.

  • ffiles (list[str]) – Path to model stellar spectrum files.

exotedrf.utils.fancyprint(message, msg_type='INFO')

Fancy printing statement mimicking logging. Basically a hack to get around complications with the STScI pipeline logging.

Parameters:

  • message (str) – Message to print.

  • msg_type (str) – Type of message. Mirrors the jwst pipeline logging.

exotedrf.utils.format_out_frames(out_frames)

Create a mask of baseline flux frames for lightcurve normalization.

Parameters:

out_frames (array-like[int], int) – Integration numbers of ingress and/or egress.

Returns:

baseline_ints – Array of baseline frames.

Return type:

array-like[int]

exotedrf.utils.format_out_frames_2(out_frames, max_nint)

Format the indices of baseline frames.

Parameters:

  • out_frames (array-like[int], int) – Integration numbers of ingress and/or egress.

  • max_nint (int) – Number of integrations in the exposure.

Returns:

baseline_ints – Indicaes of baseline frames.

Return type:

array-like[int]

exotedrf.utils.get_centroids_miri(deepframe, ystart=0, yend=None, save_results=True, save_filename='', allow_slope=False)

Get the MIRI trace centroids via the edgetrigger method.

Parameters:

  • deepframe (array-like[float]) – Median stack.

  • ystart (int) – Starting y-pixel position on the frame.

  • yend (int, None) – Ending y-pixel position on the frame.

  • save_results (bool) – If True, save results to file.

  • save_filename (str) – Filename of save file.

  • allow_slope (bool) – If True, fit a first order polynomial to trace positions, allowing for the trace to be tilted relative to the vertical.

Returns:

cens – X and Y centroids.

Return type:

array-like[float]

exotedrf.utils.get_centroids_nirspec(deepframe, xstart=0, xend=None, save_results=True, save_filename='')

Get the NIRSpec trace centroids via the edgetrigger method.

Parameters:

  • deepframe (array-like[float]) – Median stack.

  • xstart (int) – Starting x-pixel position on the frame.

  • xend (int, None) – Ending x-pixel position on the frame.

  • save_results (bool) – If True, save results to file.

  • save_filename (str) – Filename of save file.

Returns:

cens – X and Y centroids.

Return type:

array-like[float]

exotedrf.utils.get_centroids_soss(deepframe, tracetable, subarray, save_results=True, save_filename='')

Get the SOSS trace centroids for all three orders via the edgetrigger method.

Parameters:

  • deepframe (array-like[float]) – Median stack.

  • tracetable (str) – Path to SpecTrace reference file.

  • subarray (str) – Subarray identifier.

  • save_results (bool) – If True, save results to file.

  • save_filename (str) – Filename of save file.

Returns:

  • cen_o1 (array-like[float]) – Order 1 X and Y centroids.

  • cen_o2 (array-like[float]) – Order 2 X and Y centroids.

  • cen_o3 (array-like[float]) – Order 3 X and Y centroids.

exotedrf.utils.get_default_header()

Format the default header for the lightcurve file.

Returns:

  • header_dict (dict) – Header keyword dictionary.

  • header_commets (dict) – Header comment dictionary.

exotedrf.utils.get_dq_flag_metrics(dq_map, flags)

Take a data quality map and extract a map of pixels which are flagged for a specific reason. A list of data quality flags can be found here: https://jwst-reffiles.stsci.edu/source/data_quality.html.

Parameters:

  • dq_map (array-like(float)) – Map of data quality flags.

  • flags (list[str], str) – Flag types to find.

Returns:

flagged – Boolean map where True values have the applicable flag.

Return type:

np.array(bool)

exotedrf.utils.get_exouprf_built_in_models(model)

Print names of exoUPRF bullt in models.

exotedrf.utils.get_filename_root(datafiles)

Get the file name roots for each segment. Assumes that file names follow the default jwst pipeline structure and are in correct segment order.

Parameters:

datafiles (array-like[str], array-like[jwst.datamodel]) – Datamodels, or paths to datamodels for each segment.

Returns:

fileroots – List of file name roots.

Return type:

list[str]

exotedrf.utils.get_filename_root_noseg(fileroots)

Get the file name root for a SOSS TSO with no segment information.

Parameters:

fileroots (array-like[str]) – File root names for each segment.

Returns:

fileroot_noseg – File name root with no segment information.

Return type:

str

exotedrf.utils.get_instrument_name(datafile)

Get name of instrument.

Parameters:

datafile (str, datamodel) – Path to datafile or datafile itself.

Returns:

instrument – Name of instrument.

Return type:

str

exotedrf.utils.get_interp_box(data, xbox_size, ybox_size, i, j)

Get median and standard deviation of a box centered on a specified pixel.

Parameters:

  • data (array-like[float]) – Data frame.

  • xbox_size (int) – Size of box to consider – x-direction.

  • ybox_size (int) – Size of box to consider – y-direction.

  • i (int) – X pixel.

  • j (int) – Y pixel.

Returns:

box_properties – Median and standard deviation of pixels in the box.

Return type:

array-like

exotedrf.utils.get_miri_dropped_frames(dq_map)
Parameters:

dq_map (array-like(int))

Returns:

dropped_groups – List of dropped MIRI groups.

Return type:

ndarray(int)

exotedrf.utils.get_nrs_detector_name(datafile)

Get name of detector.

Parameters:

datafile (str, datamodel) – Path to datafile or datafile itself.

Returns:

detector – Name of detector.

Return type:

str

exotedrf.utils.get_nrs_grating(datafile)

Get name of grating.

Parameters:

datafile (str, datamodel) – Path to datafile or datafile itself.

Returns:

grating – Name of grating.

Return type:

str

exotedrf.utils.get_nrs_trace_start(detector, subarray, grating)

Determine the x pixel position of the trace starting point for a given NIRSpec observation.

Parameters:

  • detector (str) – Detector identifier.

  • subarray (str) – Subarray identifier.

  • grating (str) – Grating identifier.

Returns:

xstart – Starting x-pixel position of the trace.

Return type:

int

exotedrf.utils.get_soss_subarray(datafile)

Get name of subarray.

Parameters:

datafile (str, datamodel) – Path to datafile or datafile itself.

Returns:

subarray – Name of subarray.

Return type:

str

exotedrf.utils.get_stellar_param_grid(st_teff, st_logg, st_met)

Given a set of stellar parameters, determine the neighbouring grid points based on the PHOENIX grid steps.

Parameters:

  • st_teff (float) – Stellar effective temperature.

  • st_logg (float) – Stellar log surface gravity.

  • st_met (float) – Stellar metallicity as [Fe/H].

Returns:

  • teffs (list[float]) – Effective temperature grid bounds.

  • loggs (list[float]) – Surface gravity grid bounds.

  • mets (list[float]) – Metallicity grid bounds.

exotedrf.utils.get_wavebin_limits(wave)

Determine the upper and lower limits of wavelength bins centered on a given wavelength axis.

Parameters:

wave (array-like[float]) – Wavelengh array.

Returns:

  • bin_low (array-like[float]) – Lower edge of wavelength bin.

  • bin_up (array-like[float]) – Upper edge of wavelength bin.

exotedrf.utils.interpolate_stellar_model_grid(model_files, st_teff, st_logg, st_met)

Given a grid of stellar spectrum files, interpolate the model spectra to a set of stellar parameters.

Parameters:

  • model_files (list[str]) – List of paths to stellar spectra at grid points.

  • st_teff (float) – Stellar effective temperature.

  • st_logg (float) – Stellar log surface gravity.

  • st_met (float) – Stellar metallicity as [Fe/H].

Returns:

model_interp – Model stellar spectrum interpolated to the input paramaters.

Return type:

array-like(float)

exotedrf.utils.line_mle(x, y, e)

Analytical solution for Chi^2 of fitting a straight line to data. All inputs are assumed to be 2D (dimy, dimx).

Parameters:

  • x (array-like[float]) – X-data. Median stack for 1/f correction.

  • y (array-like[float]) – Y-data. Data frames for 1/f correction.

  • e (array-like[float]) – Errors.

Returns:

  • m_e (np.array(float)) – “Slope” values for even numbered columns.

  • b_e (np.array(float)) – “Intercept” values for even numbered columns.

  • m_o (np.array(float)) – “Slope” values for odd numbered columns.

  • b_o (np.array(float)) – “Intercept” values for odd numbered columns.

exotedrf.utils.make_baseline_stack_dm(datafiles, baseline_ints)

For a given set of input files, make a deep stack of only the integrations part of the timeseries baseline – for datamodel inputs.

Parameters:

  • datafiles (array-like(str), array-like(CubeModel), array-like(RampModel)) – Input datafiles.

  • baseline_ints (array-like(int)) – Integration numbers of the baseline.

Returns:

stack – Deep stack of the baseline integrations.

Return type:

np.ndarray(float)

exotedrf.utils.make_baseline_stack_fits(datafiles, baseline_ints)

For a given set of input files, make a deep stack of only the integrations part of the timeseries baseline – for fits file inputs.

Parameters:

  • datafiles (array-like(str)) – Input datafiles.

  • baseline_ints (array-like(int)) – Integration numbers of the baseline.

Returns:

stack – Deep stack of the baseline integrations.

Return type:

np.ndarray(float)

exotedrf.utils.make_baseline_stack_general(datafiles, baseline_ints)

Wrapper around the above two functions to take either fits or datamodel inputs and create a baseline stack. .

Parameters:

  • datafiles (array-like(str), jwst.datamodel) – Input datafiles.

  • baseline_ints (array-like(int)) – Integration numbers of the baseline.

Returns:

stack – Deep stack of the baseline integrations.

Return type:

np.ndarray(float)

exotedrf.utils.make_custom_superbias(datafiles)

Cunstruct a custom superbias frame by stacking all of the 0th group frames in a TSO.

Parameters:

datafiles (array-like(str)) – List of paths to datafiles.

Returns:

superbias – Custom superbias reference frame.

Return type:

ndarray(float)

exotedrf.utils.make_deepstack(cube)

Make deep stack of a TSO.

Parameters:

cube (array-like[float]) – Stack of all integrations in a TSO

Returns:

deepstack – Median of the input cube along the integration axis.

Return type:

array-like[float]

exotedrf.utils.make_soss_tracemask(xpix, ypix, mask_width, dimy, dimx, invert=False)

Construct a mask of a SOSS trace where 1-valued pixels denote the trace and 0-valued pixels not the trace.

Parameters:

  • xpix (array-like(float)) – X-positions of the trace.

  • ypix (array-like(float)) – Y-position of trace.

  • mask_width (int) – Full width of the trace mask.

  • dimy (int) – Y-dimension of the mask.

  • dimx (int) – X-dimension of the mask

  • invert (bool) – If True, make 0-valued pixels the trace.

Returns:

mask – SOSS trace mask.

Return type:

array-like(int)

exotedrf.utils.mask_reset_artifact(datafile)

Routine to mask the detector reset artifact. Observations (with NIRISS and NIRSpec at least) have a line of bright pixels that move down the detector one row at a time each integration. IDK why exactly, its a “detector reset artifact” according to Loïc. It needs to be masked.

Parameters:

datafile (str, jwst.datamodel) – Path to datafile, or opened jwst.datamodel.

Returns:

artifact – Integer array with 1s denoting locations of the reset artifact.

Return type:

np.ndarray

exotedrf.utils.open_filetype(datafile)

Open a datamodel whether it is a path, or the datamodel itself.

Parameters:

datafile (str, jwst.datamodel) – Datamodel or path to datamodel.

Returns:

data – Opened datamodel.

Return type:

jwst.datamodel

Raises:

ValueError – If the filetype passed is not str or jwst.datamodel.

exotedrf.utils.outlier_resistant_variance(data)

Calculate the varaince of some data along the 0th axis in an outlier resistant manner.

exotedrf.utils.parse_config(config_file)

Parse a yaml config file.

Parameters:

config_file (str) – Path to config file.

Returns:

config – Dictionary of config parameters.

Return type:

dict

exotedrf.utils.robust_polyfit(x, y, order, maxiter=5, nstd=3.0)

Perform a robust polynomial fit.

Paremeters

xarray, list

Independent fitting variable.

yarray, list

Dependent fitting variable.

orderint

Polynomial order to fit.

maxiterint

Number of iterations for outlier rejection.

nstdfloat

Number of standard deviations to consider a value an outlier.

returns:

res – The best fitting polynomial parameters.

rtype:

array[float]

exotedrf.utils.save_extracted_spectra(filename, data, names, units, header_dict=None, header_comments=None, save_results=True)

Pack stellar spectra into a fits file.

Parameters:

  • filename (str) – File to which to save results.

  • data (array-like[float]) – Data to save.

  • names (array-like[str]) – Names of data.

  • units (array-like[str]) – Units of dat.

  • header_dict (dict) – Header keywords and values.

  • header_comments (dict) – Header comments.

  • save_results (bool) – If True, save results to file.

Returns:

param_dict – Lightcurve parameters packed into a dictionary.

Return type:

dict

exotedrf.utils.save_ld_priors(wave, ld, order, target, m_h, teff, logg, outdir, ld_model_type, observing_mode)

Write model limb darkening parameters to a file to be used as priors for light curve fitting.

Parameters:

  • wave (array-like[float]) – Wavelength axis.

  • ld (list[float]) – Model limb darkening values.

  • order (int) – SOSS order.

  • target (str) – Name of the target.

  • m_h (float) – Host star metallicity.

  • teff (float) – Host star effective temperature.

  • logg (float) – Host star gravity.

  • outdir (str) – Directory to which to save file.

  • ld_model_type (str) – Limb darkening model identifier.

  • observing_mode (str) – Observing mode identifier.

exotedrf.utils.scatter_normalize_cube(cube, window=5)

Determine at what level each pixel in a datacube deviates from the surrounding temporal median.

Parameters:

  • cube (np.ndarray(float)) – 3D datacube.

  • window (int) – Window size to median filter, must be odd.

Returns:

  • scale (np.ndarray(float)) – Cube highlighting the extent to which each piexl deviates from the surrounding median.

  • cube_filt (np.ndarray(float)) – median filtered input datacube.

exotedrf.utils.sigma_clip_lightcurves(flux, thresh=5, window=5)

Sigma clip outliers in time from final lightcurves.

Parameters:

  • flux (array-like[float]) – Flux array.

  • thresh (int) – Sigma level to be clipped.

  • window (int) – Window function to calculate median. Must be odd.

Returns:

flux_clipped – Flux array with outliers

Return type:

array-like[float]

exotedrf.utils.sort_datamodels(datafiles)

Sort a list of jwst datamodels or filenames in chronological order by segment.

Parameters:

datafiles (array-like(str), array-like(datamodel)) – List of jwst datamodels or filenames.

Returns:

files_sorted – Inputs sorted in chronological order.

Return type:

np.array

exotedrf.utils.unpack_atoca_spectra(datafile, quantities=('WAVELENGTH', 'FLUX', 'FLUX_ERROR'))

Unpack useful quantities from extract1d outputs.

Parameters:

  • datafile (str, MultiSpecModel) – Extract1d output, or path to the file.

  • quantities (tuple(str)) – Quantities to unpack.

Returns:

all_spec – Dictionary containing unpacked quantities for each order.

Return type:

dict

exotedrf.utils.unpack_input_dir(indir, mode, filter_detector, filetag='')

Get all segment files of a specified exposure type from an input data directory.

Parameters:

  • indir (str) – Path to input directory.

  • mode (str) – Instrument mode. Currently tested are “NIRISS/SOSS” and “NIRSpec/G395H”. Though other NIRSpec gratings are also supported.

  • filter_detector (str) – Filter or detector used. For SOSS, either “CLEAR” or “F277W”. For NIRSpec, either “NRS1” or “NRS2”.

  • filetag (str) – File name extension of files to unpack.

Returns:

segments – File names of the requested exposure and file tag in chronological order.

Return type:

ndarray[str]

exotedrf.utils.verify_path(path)

Verify that a given directory exists. If not, create it.

Parameters:

path (str) – Path to directory.