SimpleDet Docs

Evaluation

Evaluation is not a separate engine in this repo. It is a controlled reuse of the pipeline test path, which keeps dataset metadata, evaluator configuration, and output files aligned with the training run.

Overview Quickstart Core Concepts API Reference

Use this page to understand how testing, evaluator configuration, and output artifacts stay coupled to the pipeline.

Direct evaluation helper

For the project-layout workflow, the simplest entry point is the direct evaluation helper. It runs the same pipeline-backed test path without requiring a config file. 

from simpledet import run_evaluation
from simpledet.suite import build_detector

metrics = run_evaluation(
    dataset_root="/data/project",
    detector_spec=build_detector(
        "fcos",
        encoder="convnext_tiny.in12k_ft_in1k",
        num_classes=3,
        in_channels=3,
    ),
    categories=("object_a", "object_b", "object_c"),
    in_channels=3,
)

Legacy low-level entry point

Reuse the same detector spec and the same dataset metadata you trained with. That keeps class names, evaluator settings, and expected outputs consistent when you need full explicit control. 

from simpledet import evaluate
from simpledet.suite import build_detector

metrics = evaluate(
    build=True,
    detector_spec=build_detector(
        "fcos",
        encoder="convnext_tiny.in12k_ft_in1k",
        num_classes=3,
        in_channels=3,
    ),
    data_folder="/data/project",
    result_folder="results/project",
    annot_file_train="/data/project/annotations/train.json",
    annot_file_val="/data/project/annotations/val.json",
    annot_file_test="/data/project/annotations/test.json",
    tif_channels_to_load=[1, 1, 1],
    in_channels=3,
    categories=("object_a", "object_b", "object_c"),
)

Outputs

  • native-manifest.json
  • native-metrics.json with mAP-style bbox summaries, per-class AP, recall, and COCO-style prediction export records
  • checkpoints/ for the retained checkpoint files
  • prediction records returned by the runtime

The native evaluation path returns prediction records and metrics directly, writes the metrics payload as a separate artifact, and writes a JSON manifest for the run summary.

Conceptually, what evaluation means here

  • It computes native bbox metrics from the test annotations and serialized predictions.
  • It reuses pipeline.test().
  • It converts SimpleDet xyxy boxes into COCO-style xywh export records.
  • It exists so testing stays coupled to the same pipeline assumptions used during training.