punchbowl.data.punch_io#

Attributes#

`_ROOT`
`CALIBRATION_ANNOTATION`

Classes#

DefaultFormatter

A formatter that doesn't fail if a keyword is missing. Used for quicklook.

Functions#

`write_file_hash`(→ None)	Create a SHA-256 hash for a file.
`get_base_file_name`(→ str)	Determine the base file name without file type extension.
`_header_to_xml`(→ lxml.etree.Element)	Convert image header metadata into an XML Tree that can be inserted into a JPEG2000 file header.
`_generate_jp2_xmlbox`(→ glymur.jp2box.XMLBox)	Generate the JPEG2000 XML box to be inserted into the JPEG2000 file.
`write_ndcube_to_quicklook`(, write_hash)	Write an NDCube to a Quicklook format as a jpeg.
`write_quicklook_to_mp4`(→ None)	Write a list of input quicklook jpeg2000 files to an output mp4 animation.
`write_ndcube_to_fits`(→ None)	Write an NDCube as a FITS file.
`_make_provenance_hdu`(→ astropy.io.fits.BinTableHDU)
`_pack_uncertainty`(→ numpy.ndarray)	Compress the uncertainty for writing to file.
`_unpack_uncertainty`(→ numpy.ndarray)	Uncompress the uncertainty when reading from a file.
`_update_statistics`(...)	Update image statistics in metadata before writing to file.
`load_ndcube_from_fits`(→ ndcube.NDCube)	Load an NDCube from a FITS file.
`_load_many_cubes_caller`(→ ndcube.NDCube \| str)
`load_many_cubes_iterable`(→ list[ndcube.NDCube \| str])	Load many NDCubes in parallel.
`load_many_cubes`(→ list[ndcube.NDCube \| str])	Load many NDCubes in parallel.
`check_outlier`(→ bool)	Check the input data cube for outlier flagging.
`encode_outliers`(→ int)	Encode the input data cube to return the outlier status for spacecraft 4321.
`decode_outliers`(→ dict)	Decode the input data cube to return the outlier status for spacecraft 4321.
`remix_collection`(, indices, Ellipsis] =, angles, ...)	Create an NDCollection of image cubes primarily used for solpolpy.

Module Contents#

punchbowl.data.punch_io._ROOT = b'.'#

punchbowl.data.punch_io.CALIBRATION_ANNOTATION = '{OBSRVTRY} - {TYPECODE}{OBSCODE} - {DATE-OBS} - exptime: {EXPTIME} s - polarizer: {POLAR} deg'#

punchbowl.data.punch_io.write_file_hash(path: str) → None[source]#: Create a SHA-256 hash for a file.

punchbowl.data.punch_io.get_base_file_name(cube: ndcube.NDCube) → str[source]#: Determine the base file name without file type extension.

class punchbowl.data.punch_io.DefaultFormatter[source]#

Bases: string.Formatter

A formatter that doesn’t fail if a keyword is missing. Used for quicklook.

get_field(field_name: str, args: Any, kwargs: Any) → str[source]#: Provide a special getter that returns the name if it fails.

punchbowl.data.punch_io._header_to_xml(header: astropy.io.fits.Header) → lxml.etree.Element[source]#

Convert image header metadata into an XML Tree that can be inserted into a JPEG2000 file header.

(Helper function adapted from SunPy)

punchbowl.data.punch_io._generate_jp2_xmlbox(header: astropy.io.fits.Header) → glymur.jp2box.XMLBox[source]#

Generate the JPEG2000 XML box to be inserted into the JPEG2000 file.

(Helper function adapted from SunPy)

punchbowl.data.punch_io.write_ndcube_to_quicklook(cube: ndcube.NDCube, filename: str, layer: int | str | None = 'tb', vmin: float = 1e-14, vmax: float = 1e-12, include_meta: bool = True, annotation: str | None = None, color: bool = False, gamma: float = 1 / 2.2, trim_edge: float | tuple[float, float] | list[float, float] | None = (0.081, 0.705), write_hash: bool = False) → None[source]#

Write an NDCube to a Quicklook format as a jpeg.

Parameters:

cube (NDCube) – data cube to visualize
filename (str) – path to save output, must end in .jp2, .j2k, .jpeg, .jpg
layer (int | str | None) – if the cube is 3D and an integer is provided, selects cube.data[layer] for visualization if the cube is 3D and the string ‘tB’ is provided, visualizes the computed total brightness
vmin (float) – the lower limit value to visualize
vmax (float) – the upper limit value to visualize
include_meta (bool) – whether to include metadata in the JPEG2000 file
annotation (str | None) – a formatted string to add to the bottom of the image as a label can access metadata by key, e.g. “typecode={TYPECODE}” would write the data’s typecode into the image
color (bool) – flag to generate RGB quicklook files, grayscale by default
gamma (float) – power law exponent used for color normalization
trim_edge (float, tuple[float, float], list[float, float], None) – Option to trim the edges of quicklook products to the specified fractional radial distance. One input value trims the outer boundary only, while two trim both the inner and outer boundaries. A reasonable set of values are (0.081, 0.705) for the inner and outer boundaries.
write_hash (bool) – writes a .sha hash for each file, this is intended for QuickPUNCH products where the SHA is used

Return type:

None

punchbowl.data.punch_io.write_quicklook_to_mp4(files: list[str], filename: str, framerate: int = 5, resolution: int = 1024, codec: str = 'libx264', ffmpeg_cmd: str = 'ffmpeg') → None[source]#

Write a list of input quicklook jpeg2000 files to an output mp4 animation.

Parameters:

files (list[str]) – List of input files to animate
filename (str) – Output filename
framerate (int, optional) – Frame rate (default 5)
resolution (int, optional) – Output resolution (default 1024)
codec (str, optional) – Codec to use for encoding. For GPU acceleration. “h264_videotoolbox” can be used on ARM Macs, “h264_nvenc” can be used on Intel machines.
ffmpeg_cmd (str) – path to the ffmpeg executable

punchbowl.data.punch_io.write_ndcube_to_fits(cube: ndcube.NDCube, filename: str, overwrite: bool = False, write_hash: bool = True, skip_stats: bool = False, skip_wcs_conversion: bool = False, uncertainty_quantize_level: float = 16) → None[source]#: Write an NDCube as a FITS file.

punchbowl.data.punch_io._make_provenance_hdu(filenames: list[str]) → astropy.io.fits.BinTableHDU[source]#

punchbowl.data.punch_io._pack_uncertainty(cube: ndcube.NDCube) → numpy.ndarray[source]#: Compress the uncertainty for writing to file.

punchbowl.data.punch_io._unpack_uncertainty(uncertainty_array: numpy.ndarray, data_array: numpy.ndarray) → numpy.ndarray[source]#: Uncompress the uncertainty when reading from a file.

punchbowl.data.punch_io._update_statistics(cube: ndcube.NDCube, modify_inplace: bool = False) → punchbowl.data.meta.NormalizedMetadata[source]#: Update image statistics in metadata before writing to file.

punchbowl.data.punch_io.load_ndcube_from_fits(path: str | pathlib.Path, key: str = ' ', include_provenance: bool = True, include_uncertainty: bool = True, dtype: type = float) → ndcube.NDCube[source]#: Load an NDCube from a FITS file.

punchbowl.data.punch_io._load_many_cubes_caller(path: str | pathlib.Path, kwargs: dict, allow_errors: bool) → ndcube.NDCube | str[source]#

punchbowl.data.punch_io.load_many_cubes_iterable(paths: list[str | pathlib.Path], n_workers: int | None = None, allow_errors: bool = False, **kwargs: dict) → list[ndcube.NDCube | str][source]#

Load many NDCubes in parallel.

Does not fork so as to be Prefect-compatible.

When used as an iterator, cubes are yielded as they are loaded, allowing e.g. progress messages to be printed

Parameters:

paths – File paths to load.
n_workers – Number of parallel workers to use. A large number may overwhelm the main thread (which has to receive each loaded cube), limiting the speed benefits of using many workers.
allow_errors – If True, if an exception is raised when loading a cube, the traceback is yielded rather than an NDCube. If False, exceptions are raised in the normal way.
kwargs – Extra args are passed to load_NDCube_from_fits

punchbowl.data.punch_io.load_many_cubes(paths: list[str | pathlib.Path], n_workers: int | None = None, allow_errors: bool = False, progress_bar: bool = False, **kwargs: dict) → list[ndcube.NDCube | str][source]#

Load many NDCubes in parallel.

Does not fork so as to be Prefect-compatible.

Parameters:

paths – File paths to load.
n_workers – Number of parallel workers to use. A large number may overwhelm the main thread (which has to receive each loaded cube), limiting the speed benefits of using many workers.
allow_errors – If True, if an exception is raised when loading a cube, the traceback is yielded rather than an NDCube. If False, exceptions are raised in the normal way.
progress_bar – If True, show a progress bar
kwargs – Extra args are passed to load_NDCube_from_fits

Return type:

A list of NDCubes (or traceback strings)

punchbowl.data.punch_io.check_outlier(cube: ndcube.NDCube) → bool[source]#: Check the input data cube for outlier flagging.

punchbowl.data.punch_io.encode_outliers(cubes: list[ndcube.NDCube]) → int[source]#: Encode the input data cube to return the outlier status for spacecraft 4321.

punchbowl.data.punch_io.decode_outliers(cube: ndcube.NDCube) → dict[source]#: Decode the input data cube to return the outlier status for spacecraft 4321.

punchbowl.data.punch_io.remix_collection(data: collections.abc.Sequence[Any] | numpy.ndarray, wcs: astropy.wcs.WCS, labels: tuple[str, Ellipsis] = ('M', 'Z', 'P'), indices: tuple[int, Ellipsis] = (0, 1, 2), angles: tuple[astropy.units.Quantity, Ellipsis] = (-60 * u.deg, 0 * u.deg, 60 * u.deg)) → ndcube.NDCollection[source]#

Create an NDCollection of image cubes primarily used for solpolpy.

Parameters:

data (Sequence or numpy.ndarray) – Input cubes containing- - a sequence of NDCube-like objects with a .data attribute, or - a 3D NumPy array / FITS cube with shape (nz, ny, nx)
wcs (astropy.wcs.WCS) – WCS to assign to the output cubes.
labels (tuple[str, ...], optional) – Labels for the collection entries.
indices (tuple[int, ...], optional) – Indices selecting elements from data.
angles (tuple[astropy.units.Quantity, ...], optional) – Polarizer angles stored in the POLAR metadata.

Returns:

Collection of NDCube objects with aligned axes.

Return type:

NDCollection