Batches¶
Batch executes independent requests on a bounded
thread pool and stores their results in input order.
client.zonal_stats.raster.batch(...) waits for completion and returns one.
Reading or exporting the results makes no additional requests.
Eager batches and streams share the same bounded executor: neither queues the
whole workload up front. Eager batches still retain all completed inputs and
results; streams release consumed results. With errors="raise", an eager batch
finishes all calculations before raising its first failure in input order, while
a stream stops scheduling work when that failure is encountered.
Both modes return BatchFailure for failed calculations under errors="return".
The failure retains .item and the original .error; to_df() keeps this same
failure object in its error column. Zonal batch inputs are always ZonalTask,
whether the source was supplied as a URL, a source list, or a STAC search.
batch
¶
Eager and streaming results over one bounded, ordered executor.
Classes:
| Name | Description |
|---|---|
Batch |
Compute all inputs and store results in input order. |
BatchFailure |
A batch failure retaining the input and original exception. |
BatchStream |
Single-pass, input-ordered results with at most max_workers pending inputs. |
Batch
¶
Batch(
inputs: Iterable[I],
compute: Callable[[I], T],
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise"] = "raise",
label: Callable[[I], Any] | None = None,
error_context: Callable[[I], dict[str, Any]]
| None = None,
)
Batch(
inputs: Iterable[I],
compute: Callable[[I], T],
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["return"],
label: Callable[[I], Any] | None = None,
error_context: Callable[[I], dict[str, Any]]
| None = None,
)
Batch(
inputs: Iterable[I],
compute: Callable[[I], T],
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise", "return"] = "raise",
label: Callable[[I], Any] | None = None,
error_context: Callable[[I], dict[str, Any]]
| None = None,
)
Batch(
inputs: Iterable[I],
compute: Callable[[I], T],
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise", "return"] = "raise",
label: Callable[[I], Any] | None = None,
error_context: Callable[[I], dict[str, Any]]
| None = None,
)
Bases: Generic[I, T]
Compute all inputs and store results in input order.
Construction waits for every computation to finish. At most max_workers
computations are pending at once; inputs are consumed as capacity becomes
available. compute must be thread-safe. Completed inputs and results are
retained, and reading the completed batch never starts more work.
With errors="raise" (the default), construction raises the first failed
input's exception after all inputs finish. With errors="return", BatchFailure objects
occupy their input positions. Only ordinary Exception subclasses are
captured; interrupts propagate.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
inputs
|
Iterable[I]
|
Inputs to consume and compute once each. |
required |
compute
|
Callable[[I], T]
|
Function called for each input. |
required |
max_workers
|
int
|
Maximum concurrent computations; defaults to eight. |
DEFAULT_MAX_WORKERS
|
errors
|
Literal['raise', 'return']
|
Raise the first error in input order, or retain errors as results. |
'raise'
|
label
|
Callable[[I], Any] | None
|
Optional input label for failed rows in |
None
|
error_context
|
Callable[[I], dict[str, Any]] | None
|
Optional additional fields for failed rows. |
None
|
Methods:
| Name | Description |
|---|---|
results |
Return a copy of the completed results in input order. |
to_df |
Export completed results, with an |
Attributes:
| Name | Type | Description |
|---|---|---|
max_workers |
|
|
errors |
|
|
inputs |
Sequence[I]
|
The inputs in result order, as a copy. |
to_df
¶
Export completed results, with an error column for failed rows.
Successful results use their to_dict() representation. Failed rows
retain their input label if supplied. Requires the optional pandas extra.
BatchFailure
¶
BatchStream
¶
BatchStream(
inputs: Iterable[I],
compute: Callable[[I], T],
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise"] = "raise",
)
BatchStream(
inputs: Iterable[I],
compute: Callable[[I], T],
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["return"],
)
BatchStream(
inputs: Iterable[I],
compute: Callable[[I], T],
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise", "return"] = "raise",
)
BatchStream(
inputs: Iterable[I],
compute: Callable[[I], T],
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise", "return"] = "raise",
)
Bases: Generic[I, T]
Single-pass, input-ordered results with at most max_workers pending inputs.
Work starts on iteration. Use as a context manager when stopping early; closing cancels queued work and waits for requests already in flight. Results are not retained. Keep the owning client open while consuming.
Methods:
| Name | Description |
|---|---|
close |
Stop consuming inputs and finish any in-flight requests. |
stream=True returns a single-pass BatchStream with bounded pending work.
Use its context manager when stopping early. prepare() returns a ZonalJob
whose request_count can be inspected before calling run(stream=True).
See STAC searches and large jobs.
zonal_job
¶
Prepared Cartesian workloads without materializing site-source pairs.
Classes:
| Name | Description |
|---|---|
StacItemLike |
A STAC item exposing its dictionary representation. |
StacSearchLike |
A search that yields STAC item dictionaries. |
ZonalSource |
A resolved URL and optional STAC provenance. |
ZonalTask |
One AOI-source pair, also retained on streamed failures. |
ZonalJob |
Resolved AOIs and sources; pairs are generated only as workers need them. |
Functions:
| Name | Description |
|---|---|
default_asset |
The key of the first asset with the |
source_from_item |
Resolve one STAC Item (mapping or |
resolve_sources |
|
Attributes:
| Name | Type | Description |
|---|---|---|
SourceInput |
TypeAlias
|
|
SourceInput
module-attribute
¶
SourceInput: TypeAlias = (
str | Mapping[str, Any] | StacItemLike | ZonalSource
)
StacItemLike
¶
StacSearchLike
¶
Bases: Protocol
A search that yields STAC item dictionaries.
Methods:
| Name | Description |
|---|---|
items_as_dicts |
|
ZonalSource
dataclass
¶
ZonalTask
dataclass
¶
ZonalTask(aoi: Any, label: Any, source: ZonalSource)
One AOI-source pair, also retained on streamed failures.
Attributes:
| Name | Type | Description |
|---|---|---|
aoi |
Any
|
|
label |
Any
|
|
source |
ZonalSource
|
|
ZonalJob
¶
ZonalJob(
aois: list[Any],
labels: list[Any],
sources: tuple[ZonalSource, ...],
compute: Callable[[ZonalTask], ZonalStatsResult],
)
Resolved AOIs and sources; pairs are generated only as workers need them.
Constructed by an endpoint with resolved sources and a bound computation; jobs only generate pairs and execute them.
Preparing fetches search results but sends no statistics requests. Running again repeats calculations; this is not a durable checkpoint or resume store.
Methods:
| Name | Description |
|---|---|
run |
Execute eagerly, or stream with bounded memory using stream=True. |
Attributes:
| Name | Type | Description |
|---|---|---|
sources |
|
|
request_count |
int
|
Exact number of AOI-source pairs (before retries). |
run
¶
run(
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise"] = "raise",
stream: Literal[False] = False,
) -> Batch[ZonalTask, ZonalStatsResult]
run(
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise"] = "raise",
stream: Literal[True],
) -> BatchStream[ZonalTask, ZonalStatsResult]
run(
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise"] = "raise",
stream: bool = False,
) -> (
Batch[ZonalTask, ZonalStatsResult]
| BatchStream[ZonalTask, ZonalStatsResult]
)
run(
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise", "return"] = "raise",
stream: Literal[False] = False,
) -> Batch[
ZonalTask, ZonalStatsResult | BatchFailure[ZonalTask]
]
run(
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise", "return"] = "raise",
stream: Literal[True],
) -> BatchStream[
ZonalTask, ZonalStatsResult | BatchFailure[ZonalTask]
]
run(
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise", "return"] = "raise",
stream: bool = False,
) -> (
Batch[
ZonalTask,
ZonalStatsResult | BatchFailure[ZonalTask],
]
| BatchStream[
ZonalTask,
ZonalStatsResult | BatchFailure[ZonalTask],
]
)
run(
*,
max_workers: int = DEFAULT_MAX_WORKERS,
errors: Literal["raise", "return"] = "raise",
stream: bool = False,
) -> (
Batch[
ZonalTask,
ZonalStatsResult | BatchFailure[ZonalTask],
]
| BatchStream[
ZonalTask,
ZonalStatsResult | BatchFailure[ZonalTask],
]
)
Execute eagerly, or stream with bounded memory using stream=True.
default_asset
¶
The key of the first asset with the data role, else the first key.
Many items list a thumbnail before their data, so taking the first key alone could compute statistics over a PNG.
source_from_item
¶
source_from_item(
item: Any, asset: str | None
) -> ZonalSource
Resolve one STAC Item (mapping or to_dict() object) to its asset URL.
resolve_sources
¶
resolve_sources(
url: str | None,
sources: Any,
search: Any,
asset: str | None,
) -> tuple[ZonalSource, ...]
The second type parameter is the yielded value: default execution returns
Batch[ZonalTask, ZonalStatsResult] (or BatchStream), while errors="return"
includes BatchFailure[ZonalTask] in that value type. The same rule applies to
standalone Batch and BatchStream construction. A variable error policy keeps
the union, since either behavior is possible.