punchbowl.auto.control.scheduler#

Functions#

generic_scheduler_flow_logic(→ int)

Implement the core logic of each scheduler flow.

Module Contents#

punchbowl.auto.control.scheduler.generic_scheduler_flow_logic(query_ready_files_func, construct_child_file_info, construct_child_flow_info, pipeline_config, update_input_file_state=True, new_input_file_state='progressed', session=None, reference_time: datetime.datetime | None = None, args_dictionary: dict = {}, children_are_one_to_one: bool = False, cap_planned_flows: bool = True) → int#

Implement the core logic of each scheduler flow.

Parameters:

  • query_ready_files_func – A function that returns a list of lists, where each of the inner lists is a group of files that will be the inputs to one flow

  • construct_child_file_info – A function that generates the child File entries for one group/flow

  • construct_child_flow_info – A function that generates the Flow entry for one group

  • pipeline_config – The config or config path

  • update_input_file_state – Whether to change the state of the input files

  • new_input_file_state – The new state to assign to each input file

  • session – A database Session

  • reference_time – Some time of observation time associated with this scheduling run. The meaning is defined by flow-specific functions passed into this function.

  • args_dictionary (dict) – Values in this dictionary are passed directly to the query_ready_files_func, query_ready_files_func, and construct_child_flow_info functions

  • children_are_one_to_one – By default, for each group of input files, it is assumed that all inputs together produce all the output files, and FileRelationships are generated accordingly. In a case where a batch of input files are to be processed in one flow, this assumption doesn’t hold. When this flag is set to True, it is assumed each input file connects to only one output file (at the corresponding position in the list of child File objects).