punchbowl.auto.control.launcher#
Functions#
|
|
|
|
|
|
|
|
|
Given a list of ready-to-launch flow_ids, this task creates flow runs in Prefect for them. |
|
|
|
The main launcher flow for Prefect, responsible for identifying flows, based on priority, |
Module Contents#
- punchbowl.auto.control.launcher.gather_planned_flows(session, weight_to_launch, max_flows_to_launch, flow_weights, flow_enabled, flow_batch_sizes)#
- punchbowl.auto.control.launcher.count_flows(session, weights)#
- punchbowl.auto.control.launcher.escalate_long_waiting_flows(session, pipeline_config)#
- punchbowl.auto.control.launcher.determine_launchable_flow_count(weight_planned, weight_running, max_weight_running, max_weight_to_launch, max_flows_to_launch)#
- async punchbowl.auto.control.launcher.launch_ready_flows(session: sqlalchemy.orm.Session, flow_info: list[list[punchbowl.auto.control.db.Flow]], tags_by_flow: dict[int, str], pipeline_config: dict) None#
Given a list of ready-to-launch flow_ids, this task creates flow runs in Prefect for them. These flow runs are automatically marked as scheduled in Prefect and will be picked up by a work queue and agent as soon as possible.
- Parameters:
session (sqlalchemy.orm.session.Session) – A SQLAlchemy session for database interactions
flow_info (List[int]) – A list of flow IDs from the punchpipe database identifying which flows to launch
- Return type:
A list of responses from Prefect about the flow runs that were created
- punchbowl.auto.control.launcher.load_flow_data(pipeline_config)#
- async punchbowl.auto.control.launcher.launcher(pipeline_config_path=None)#
- The main launcher flow for Prefect, responsible for identifying flows, based on priority,
that are ready to run and creating flow runs for them. It also escalates long-waiting flows’ priorities.
See EM 41 or the internal requirements document for more details
- Return type:
Nothing