SimpleDet Docs

Evaluation

Evaluation is not a separate engine in this repo. It is a controlled reuse of the pipeline test path, which keeps dataset metadata, evaluator configuration, and output files aligned with the training run.

Overview Quickstart Core Concepts API Reference

Use this page to understand how testing, evaluator configuration, and output artifacts stay coupled to the pipeline.

Direct evaluation helper

For the project-layout workflow, the simplest entry point is the direct evaluation helper. It runs the same pipeline-backed test path without requiring a config file. 

from simpledet import run_evaluation
from simpledet.suite import build_detector

metrics = run_evaluation(
    dataset_root="/data/project",
    detector_spec=build_detector(
        "fcos",
        encoder="convnext_tiny.in12k_ft_in1k",
        num_classes=3,
        in_channels=3,
    ),
    categories=("object_a", "object_b", "object_c"),
    in_channels=3,
)

Legacy low-level entry point

Reuse the same detector spec and the same dataset metadata you trained with. That keeps class names, evaluator settings, and expected outputs consistent when you need full explicit control. 

from simpledet import evaluate
from simpledet.suite import build_detector

metrics = evaluate(
    build=True,
    detector_spec=build_detector(
        "fcos",
        encoder="convnext_tiny.in12k_ft_in1k",
        num_classes=3,
        in_channels=3,
    ),
    data_folder="/data/project",
    result_folder="results/project",
    annot_file_train="/data/project/annotations/train.json",
    annot_file_val="/data/project/annotations/val.json",
    annot_file_test="/data/project/annotations/test.json",
    tif_channels_to_load=[1, 1, 1],
    in_channels=3,
    categories=("object_a", "object_b", "object_c"),
)

Outputs

  • native-manifest.json
  • checkpoints/ for the retained checkpoint files
  • prediction records returned by the runtime

The native evaluation path returns prediction records directly and writes a JSON manifest for the run summary.

Conceptually, what evaluation means here

  • It is not an independent evaluator API.
  • It reuses pipeline.test().
  • It depends on the evaluator configuration loaded by the runtime config.
  • It exists so testing stays coupled to the same pipeline assumptions used during training.