Evaluation

The evaluator, event table validation, and metric base classes. Guide: Evaluation and metrics.

InventoryEvaluator

InventoryEvaluator()

Compute metrics from a run's event table.

fit selects and validates the event table rows (by default the run's scoring window); evaluate computes metrics, pooled into one row by default or grouped by event table columns such as ["unique_id"].

Example

evaluator = InventoryEvaluator().fit(result)          # the scoring window
evaluator.evaluate([fill_rate, avg_on_hand])           # one pooled row
evaluator.evaluate([fill_rate], groupby=["unique_id"]) # one row per SKU

fit

fit(
    simulation_result: Optional[SimulationResult] = None,
    event_frame: Optional[DataFrame] = None,
    *,
    window: Optional[str] = None,
) -> "InventoryEvaluator"

Select and validate the event table rows to evaluate.

Pass either a SimulationResult or an event table DataFrame. The rows are checked with validate_event_frame so metrics are computed from balanced books.

Parameters:

Name	Type	Description	Default
simulation_result	Optional[SimulationResult]	A result from SimulationEngine.run.	None
event_frame	Optional[DataFrame]	An event table DataFrame, for example one you saved or enriched with per-row cost rates.	None
window	Optional[str]	"scoring" (default, the window summary() uses), "warmup", "settlement", or "all", for a simulation_result and an event_frame alike.	None

Returns:

Type	Description
'InventoryEvaluator'	The fitted evaluator (self).

Raises:

Type	Description
ValueError	If both or neither inputs are given, the window is unknown, or the event table fails validation.

evaluate

evaluate(
    metrics: Iterable,
    groupby: Optional[Union[str, Sequence[str]]] = None,
    context: Optional[dict] = None,
) -> DataFrame

Compute metrics for each group of the fitted event table.

Parameters:

Name	Type	Description	Default
metrics	Iterable	Metric functions (metric(event_frame, context)) or objects with name and compute, such as BaseInventoryMetric subclasses. A function's __name__ becomes its column name.	required
groupby	Optional[Union[str, Sequence[str]]]	Event table columns to group by, such as ["unique_id"], or one column name, such as "unique_id" (the same as a one-element list); None or [] (default) gives one pooled row.	None
context	Optional[dict]	Options and rates for the metrics, for example cost rates, cost_components, include_partial_cycles, or periods_per_year.	None

Returns:

Type	Description
DataFrame	One row per group, with the group columns followed by one column per
DataFrame	metric.

Raises:

Type	Description
ValueError	If the evaluator is not fitted.

validate_event_frame

validate_event_frame(event_frame: DataFrame) -> DataFrame

Return a defensive copy after strict structural and physical validation.

Every row must satisfy the demand, on-hand, backlog, pipeline and inventory-position identities; stock cannot be short while the shelf still holds units, and a SKU cannot hold stock and backorders at once. Rows of consecutive periods of one SKU and policy must chain: each row starts with the previous row's ending on-hand, backorders and pipeline.

Event tables from runs with general process flows also carry process_inflow_units and process_outflow_units; when present (both are required together) they are validated as nonnegative flows and enter the physical inventory balance. Event tables from runs with supplier delivery outcomes carry supplier_shortfall_units; when present it is validated as a nonnegative flow and leaves the pipeline balance. When order_arrival_flag is present it must be boolean and set only on period rows that received stock. When target_missing_flag is present it must be boolean and set only on decision rows with no order and no target (order_quantity == 0, target_level missing).

BaseInventoryMetric

Bases: ABC

Base class for a named, configurable metric.

Subclass it, set name (the output column) and implement compute. InventoryEvaluator calls compute(event_frame, context) once per group. Plain functions with the signature metric(event_frame, context) work as metrics too; a class is useful when the metric has parameters.

Example

class ShareOfDaysBelow(BaseInventoryMetric):
    def __init__(self, level):
        self.level = level
        self.name = f"days_below_{level}"

    def compute(self, event_frame, context):
        rows = event_frame[event_frame["event_type"] == "period"]
        return float((rows["ending_on_hand"] < self.level).mean())

compute abstractmethod

compute(event_frame: DataFrame, context: dict) -> float

Compute the metric for one slice of the event table.

Parameters:

Name	Type	Description	Default
event_frame	DataFrame	The event table rows of one window and group.	required
context	dict	The context dict passed to InventoryEvaluator.evaluate.	required

Returns:

Type	Description
float	The metric value.

CoverageMetric

CoverageMetric(mode: str = 'forward')

Bases: BaseInventoryMetric

Mean inventory coverage in periods: stock divided by a demand rate.

mode="forward" divides ending stock by an expected demand rate, taken from an expected_demand_rate event table column or context["forward_demand_rate"]. mode="trailing" divides by each SKU's average realised demand in the slice. Rows with a zero rate are ignored. For slices with more than one SKU the result is the mean of the SKU-period ratios.

Parameters:

Name	Type	Description	Default
mode	str	"forward" or "trailing". The metric's name is coverage_<mode>.	'forward'

TotalCost

TotalCost(
    *,
    holding: Optional[float] = None,
    shortage: Optional[float] = None,
    backlog: Optional[float] = None,
    order_per_line: Optional[float] = None,
    order_per_unit: Optional[float] = None,
    purchase: Optional[float] = None,
    waste: Optional[float] = None,
    terminal_backlog: Optional[float] = None,
    terminal_pipeline: Optional[float] = None,
    salvage_on_hand: Optional[float] = None,
    salvage_pipeline: Optional[float] = None,
    name: str = "total_cost",
)

Bases: BaseInventoryMetric

Total cost of the components whose rates you give.

Each rate names a component and its value: TotalCost(holding=0.2, shortage=1.0) adds holding and shortage cost and nothing else. It gives the same number as total_cost with cost_components and the rates in context, which remains the way to use per-row rate columns.

Parameters:

Name	Type	Description	Default
holding	Optional[float]	Per unit on hand and per period.	None
shortage	Optional[float]	Per unit of demand not served.	None
backlog	Optional[float]	Per unit owed and per period.	None
order_per_line	Optional[float]	Fee per SKU order line; give with order_per_unit.	None
order_per_unit	Optional[float]	Cost per unit ordered; give with order_per_line.	None
purchase	Optional[float]	Per unit ordered.	None
waste	Optional[float]	Per unit expired or written off.	None
terminal_backlog	Optional[float]	Per unit still owed at the end.	None
terminal_pipeline	Optional[float]	Per unit still on order at the end.	None
salvage_on_hand	Optional[float]	Credit per unit on hand at the end; give with salvage_pipeline. Salvage is subtracted.	None
salvage_pipeline	Optional[float]	Credit per unit on order at the end.	None
name	str	Output column name (default "total_cost").	'total_cost'

Raises:

Type	Description
ValueError	If no rate is given, a rate is negative or not finite, or only one rate of a two-rate component is given.

Example

evaluator.evaluate([fill_rate, TotalCost(holding=0.2, shortage=1.0)])