topobench.evaluator package

Subpackages

• topobench.evaluator.metrics package
  • Submodules
  • topobench.evaluator.metrics.example module
    • ExampleRegressionMetric
      • ExampleRegressionMetric.compute()
      • ExampleRegressionMetric.full_state_update
      • ExampleRegressionMetric.higher_is_better
      • ExampleRegressionMetric.is_differentiable
      • ExampleRegressionMetric.sum_squared_error
      • ExampleRegressionMetric.total
      • ExampleRegressionMetric.update()
  • Module contents
    • ExampleRegressionMetric
      • ExampleRegressionMetric.compute()
      • ExampleRegressionMetric.full_state_update
      • ExampleRegressionMetric.higher_is_better
      • ExampleRegressionMetric.is_differentiable
      • ExampleRegressionMetric.sum_squared_error
      • ExampleRegressionMetric.total
      • ExampleRegressionMetric.update()

Submodules

topobench.evaluator.base module

Abstract class for the evaluator class.

class topobench.evaluator.base.AbstractEvaluator

Bases: ABC

Abstract class for the evaluator class.

abstract compute()

Compute the metrics.

abstract reset()

Reset the metrics.

abstract update(model_out: dict)

Update the metrics with the model output.

Parameters:

model_outdict

The model output.

topobench.evaluator.evaluator module

This module contains the Evaluator class that is responsible for computing the metrics.

class topobench.evaluator.evaluator.TBEvaluator(task, **kwargs)

Bases: AbstractEvaluator

Evaluator class that is responsible for computing the metrics.

Parameters:

taskstr

The task type. It can be either “classification” or “regression”.

**kwargsdict

Additional arguments for the class. The arguments depend on the task. In “classification” scenario, the following arguments are expected: - num_classes (int): The number of classes. - metrics (list[str]): A list of classification metrics to be computed. In “regression” scenario, the following arguments are expected: - metrics (list[str]): A list of regression metrics to be computed.

compute()

Compute the metrics.

Returns:

dict

Dictionary containing the computed metrics.

reset()

Reset the metrics.

This method should be called after each epoch.

update(model_out: dict)

Update the metrics with the model output.

Parameters:

model_outdict

The model output. It should contain the following keys: - logits : torch.Tensor The model predictions. - labels : torch.Tensor The ground truth labels.

Raises:

ValueError

If the task is not valid.

Module contents

Evaluators for model evaluation.

class topobench.evaluator.AbstractEvaluator

Bases: ABC

Abstract class for the evaluator class.

abstract compute()

Compute the metrics.

abstract reset()

Reset the metrics.

abstract update(model_out: dict)

Update the metrics with the model output.

Parameters:

model_outdict

The model output.

class topobench.evaluator.TBEvaluator(task, **kwargs)

Bases: AbstractEvaluator

Evaluator class that is responsible for computing the metrics.

Parameters:

taskstr

The task type. It can be either “classification” or “regression”.

**kwargsdict

Additional arguments for the class. The arguments depend on the task. In “classification” scenario, the following arguments are expected: - num_classes (int): The number of classes. - metrics (list[str]): A list of classification metrics to be computed. In “regression” scenario, the following arguments are expected: - metrics (list[str]): A list of regression metrics to be computed.

compute()

Compute the metrics.

Returns:

dict

Dictionary containing the computed metrics.

reset()

Reset the metrics.

This method should be called after each epoch.

update(model_out: dict)

Update the metrics with the model output.

Parameters:

model_outdict

The model output. It should contain the following keys: - logits : torch.Tensor The model predictions. - labels : torch.Tensor The ground truth labels.

Raises:

ValueError

If the task is not valid.