Data quality¶
Apache Hamilton comes with data quality included out of the box. While you can read more about this in the API reference, we have a few examples to help get you started.
The following two examples showcase a similar workflow, one using the vanilla hamilton data quality decorator, and the other using the pandera integration. The goal of this is to show how to use runtime data quality checks in a larger, more complex ETL.
Async validators¶
For validation logic that requires async operations (e.g., async database queries or API calls), use AsyncDataValidator or AsyncBaseDefaultValidator from hamilton.data_quality.base. These define async def validate() and work with AsyncDriver. You can mix sync and async validators in a single @check_output_custom call.
See the check_output reference and data quality writeup for details and examples.