punchbowl.util#
Attributes#
Classes#
Interface for passing callable objects instead of file paths to be loaded. |
Functions#
|
Check that the input array is square. |
|
Prefect task to write an image to disk. |
|
Prefect task to load data for processing. |
|
Compute average datetime from a list of datetimes. |
|
Calculate the nan percentile of a 3D cube. Isn't as fast as possible on a single core, but parallelizes very well. |
|
Sorts a 3D cube along the first axis. |
|
Interpolates between two data objects. |
|
Load the specified spacecraft mask. |
|
Find the first cube that's not None in a list of NDCubes. |
Module Contents#
- punchbowl.util.validate_image_is_square(image: numpy.ndarray) None[source]#
Check that the input array is square.
- punchbowl.util.output_image_task(data: ndcube.NDCube, output_filename: str) None[source]#
Prefect task to write an image to disk.
- Parameters:
data (NDCube) – data that is to be written
output_filename (str) – where to write the file out
- Return type:
None
- punchbowl.util.load_image_task(input_filename: str, include_provenance: bool = True, include_uncertainty: bool = True) ndcube.NDCube[source]#
Prefect task to load data for processing.
- Parameters:
input_filename (str) – path to file to load
include_provenance (bool) – whether to load the provenance layer
include_uncertainty (bool) – whether to load the uncertainty layer
- Returns:
loaded version of the image
- Return type:
NDCube
- punchbowl.util.average_datetime(datetimes: list[datetime.datetime]) datetime.datetime[source]#
Compute average datetime from a list of datetimes.
- punchbowl.util.nan_percentile(array: numpy.ndarray, percentile: float | list[float]) float | numpy.ndarray[source]#
Calculate the nan percentile of a 3D cube. Isn’t as fast as possible on a single core, but parallelizes very well.
It’s documented that numba’s sort is slower than numpy’s, and this runs single-threaded ~half as fast as the old implementation using numpy. But this parallelizes extremely well, even up to 128 cores for a 1kx2kx2k cube! Thread count can be configured by setting numba.config.NUMBA_NUM_THREADS
The .copy() for each sequence means that, even though percentiling along the zeroth dimension seems wrong from a CPU cache standpoint, transposing the input cube makes very little difference (much less than the time cost of copying the cube into a transposed orientation!). Disabling the copy for a well-dimensioned array doesn’t make a clear difference to execution time.
The nan handling appears to add only negligible computation time
- punchbowl.util.parallel_sort_first_axis(array: numpy.ndarray, handle_nans: bool = False, inplace: bool = False) numpy.ndarray[source]#
Sorts a 3D cube along the first axis.
Parallelizes very well on punch190 and phoenix.
It’s documented that numba’s sort is slower than numpy’s, but this parallelizes extremely well, even up to 64 cores for a 1kx2kx2k cube! Thread count can be configured by setting numba.config.NUMBA_NUM_THREADS
The .copy() for each sequence means that, even though sorting along the zeroth dimension seems wrong from a CPU cache standpoint, transposing the input cube makes very little difference (much less than the time cost of copying the cube into a transposed orientation!).
If handle_nans is True, NaNs are explicitly sorted to the high end of the array. Numba’s sort appears to do this anyway and still sorts the rest of the array correctly, but the flag ensures this behavior with a speed penalty.
Sorting in-place offers a ~50% speed boost in a 1kx2kx2k test case.
- punchbowl.util.interpolate_data(data_before: ndcube.NDCube, data_after: ndcube.NDCube, reference_time: datetime.datetime, time_key: str = 'DATE-OBS', allow_extrapolation: bool = False) numpy.ndarray[source]#
Interpolates between two data objects.
- punchbowl.util.load_spacecraft_mask(path_mask: str) numpy.ndarray[source]#
Load the specified spacecraft mask.
- punchbowl.util.find_first_existing_file(inputs: list[ndcube.NDCube]) ndcube.NDCube | None[source]#
Find the first cube that’s not None in a list of NDCubes.
- punchbowl.util.T#