lifecycle.api.TaskGroupingHook¶
- class hamilton.lifecycle.api.TaskGroupingHook¶
Implement this to run something after task grouping or task expansion. This will allow you to capture information about the tasks during Parallelize/Collect blocks in dynamic DAG execution.
- final post_task_expand(*, run_id: str, task_id: str, parameters: Dict[str, Any])¶
Hook that is called immediately after a task is expanded into parallelizable tasks. Note that this is only useful in dynamic execution.
- Parameters:
run_id – ID of the run, unique in scope of the driver.
task_id – ID of the task.
parameters – Parameters that are being passed to each of the expanded tasks.
- final post_task_group(*, run_id: str, task_ids: List[str])¶
Hook that is called immediately after a task group is created. Note that this is only useful in dynamic execution, although we reserve the right to add this back into the standard hamilton execution pattern.
- Parameters:
run_id – ID of the run, unique in scope of the driver.
task_ids – IDs of tasks that are in the group.
- abstractmethod run_after_task_expansion(*, run_id: str, task_id: str, parameters: Dict[str, Any], **future_kwargs)¶
Runs after task expansion in Parallelize/Collect blocks. This allows you to capture information about the task that was expanded.
- Parameters:
run_id – ID of the run, unique in scope of the driver.
task_id – ID of the task that was expanded.
parameters – Parameters that were passed to the task.
future_kwargs – Additional keyword arguments – this is kept for backwards compatibility.
- abstractmethod run_after_task_grouping(*, run_id: str, task_ids: List[str], **future_kwargs)¶
Runs after task grouping. This allows you to capture information about which tasks were created for a given run.
- Parameters:
run_id – ID of the run, unique in scope of the driver.
task_ids – List of tasks that were grouped together.
future_kwargs – Additional keyword arguments – this is kept for backwards compatibility.