Bright Structure Identification#

Bright structure identification is a necessary step for clearing the data of erroneous signal from bright structures visible in the instrument field of view, such as terrestrial aurora.

Concept#

Given a time sequence of images, this module identifies “spikes” that exceed a threshold in a single frame. The details of this process are included in the documented function below.

Applying correction#

Bright structure identification and marking is carried out in the punchbowl.level2.bright_structure.run_zspike function:

punchbowl.level2.bright_structure.run_zspike(data: ndarray, uncertainty: ndarray, threshold: float = 4, required_yes: int = 6, veto_limit: int = 2, diff_method: str = 'sigma', dilation: int = 0, blur_scale: float = 1.0, blur_threshold: float | None = 0.15, index_of_interest: int | None = None) → ndarray

Identify bright structures in temporal series of images.

Given a time sequence of images, identify “spikes” that exceed a threshold in a single frame.

Diffuse bright structures over the background F-corona are identified and marked in the data using in-band “bad value” marking (which is supported by the FITS standard). Data marking follows the existing ZSPIKE temporal despiking algorithm to identify auroral transients.

This is a voting algorithm based on ZSPIKE.PRO from the 1990s, which is available in the solarsoft distribution from Lockheed Martin.

ZSPIKE was originally used to identify cosmic rays, and was adopted on STEREO for on-board despiking during exposure accumulation. For this application ZSPIKE is ideal because it does not rely on the spatial structure of spikes, only their temporal structure. Both cosmic rays and, if present, high-altitude aurora are transient and easily detected with ZSPIKE.

The algorithm assembles “votes” from the images surrounding each one in a stream, to determine whether a particular pixel is a good candidate for a temporal spike. If the pixel is sufficiently bright compared to its neighbors in time, it is marked bad. “Bad values” are stored in the DRP for file quality marking outlined in Module Quality Marking.

There are two methods of identifying if a pixel is above a given threshold, and therefore considered a spike. diff_method=’abs’ or ‘sigma’.

diff_method ‘abs’ represents the absolute difference, and is the default. If set, this is an absolute difference, in DN, required for a pixel to ‘vote’ its central value. If the central value is this much higher than a given voting value, then the central value is voted to be a spike. If it’s this much lower, the veto count is incremented.

If diff_method ‘sigma’ is set if, then each pixel is treated as a time series and the calculated sigma (RMS variation from the mean) of the timeseries is used to calculate a difference threshold at each location.

The threshold is the value over which a pixel is voted as a spike. The threshold should be different depending diff_method.

Parameters:

  • data (np.ndarray) – data to operate on - this is expected to have an odd number of frames, or a frame_of_interest should be inserted.

  • uncertainty (np.ndarray) – data to operate on - this is expected to have an odd number of frames, or a frame_of_interest should be inserted.

  • threshold (float) – This is the threshold over which a pixel is voted as a spike.

  • required_yes (int) – (default is 4) - number of ‘voting’ frames that must vote the value is a spike, for it to be marked as such.

  • veto_limit (int) – (default is 2) - number of ‘voting’ frames that must vote NO to veto marking the central value as a spike.

  • diff_method (str) – This is the method by which the threshold is set. ‘abs’ treats each pixel independently and finds the absolute difference, and ‘sigma’ treats each corresponding pixel as a time series and the calculated RMS variation from the mean of the timeseries is used to calculate a difference at each location.

  • dilation (int) – If nonzero, this is the number of times to morphologically dilate pixels marked as bright structures.

  • blur_scale (float) – The scale of the gaussian kernel used for convolving with the mask for blurring.

  • blur_threshold (float) – The threshold value used to cast a blurred mask array back to a boolean data type.

  • index_of_interest (int) – if you have an even number of frames, or you don’t want to find spikes in the center frame, the frame_of_interest will be used. index_of_interest starts from 0, therefore the center frame from 21 frames will be 10.

Returns:

a frame matching the dimensions of the frame of interest with True flagging bad pixels, and False flagging good data points

Return type:

np.ndarray

If you wish to incorporate this as a Prefect task in a custom pipeline, using something like the punchbowl.level2.bright_structure.identify_bright_structures_task is recommended.