Detections

Base class

Detections API: abstract base class, low-level utilities, and auto-detect factory.

byotrack.api.detections.detections.draw_disk_2d(segmentation: ndarray, positions: ndarray, radii: ndarray, labels: ndarray, *, overwrite=True) → ndarray

Draw disks on a 2D segmentation mask.

Modify segmentation inplace.

Parameters:

• segmentation (np.ndarray) – 2D segmentation mask. Empty or pre-filled with other instances. Shape: (H, W), dtype: int

• positions (np.ndarray) – Position of the disks to draw. Shape: (N, 2), dtype: float

• radii (np.ndarray) – Radii of the disks to draw. Shape: (N, 2), dtype: float

• labels (np.ndarray) – Instance labels for each disk. As usual, an offset of 1 is applied, i.e. the ith disk is drawn with labels[i] + 1. Shape: (N,), dtype: int

• overwrite (bool) – Allow disk to overwrite the pre-filled segmentation. Default: True

Returns:

The modified segmentation.

Return type:

np.ndarray

byotrack.api.detections.detections.draw_disk_3d(segmentation: ndarray, positions: ndarray, radii: ndarray, labels: ndarray, *, overwrite=True) → ndarray

Draw disks on a 3D segmentation mask.

Modify segmentation inplace.

Parameters:

• segmentation (np.ndarray) – 3D segmentation mask. Empty or pre-filled with other instances. Shape: (D, H, W), dtype: int

• positions (np.ndarray) – Position of the disks to draw. Shape: (N, 3), dtype: float

• radii (np.ndarray) – Radii of the disks to draw. Shape: (N, 3), dtype: float

• labels (np.ndarray) – Instance labels for each disk. As usual, an offset of 1 is applied, i.e. the ith disk is drawn with labels[i] + 1. Shape: (N,), dtype: int

• overwrite (bool) – Allow disk to overwrite the pre-filled segmentation. Default: True

Returns:

The modified segmentation.

Return type:

np.ndarray

byotrack.api.detections.detections.fast_relabel(segmentation: ndarray, labels: ndarray) → None

Inplace fast relabel with given mapping.

It keeps the background as is (0 => 0) and assumes labeling from 0 to N-1 (offset of 1 in segmentation). In practice, it maps instance i to mapping[i-1] + 1.

Parameters:

• segmentation (np.ndarray) – Instance segmentation frame. Shape: ([D, ]H, W), dtype: int

• labels (np.ndarray) – Mapping to the targeted labels. Shape: (N,), dtype: int

byotrack.api.detections.detections.labels_of(segmentation: Tensor) → Tensor

byotrack.api.detections.detections.labels_of(segmentation: ndarray) → ndarray

Extract the sorted labels of an instance segmentation mask.

The background is excluded, and labels starts at 0 (offset of 1 with the segmentation mask where the background is 0 and the first label is 1).

Parameters:

segmentation (torch.Tensor | np.ndarray) – Segmentation mask Shape: ([D, ]H, W), dtype: int

Returns:

Sorted labels inside the segmentation mask.

Shape: (N,), dtype: int

Return type:

torch.Tensor | np.ndarray

byotrack.api.detections.detections.relabel_consecutive(segmentation: Tensor, *, inplace: bool = True) → Tensor

byotrack.api.detections.detections.relabel_consecutive(segmentation: ndarray, *, inplace: bool = True) → ndarray

Relabel a segmentation mask so that labels are consecutive.

For N instances, labels are 0 for background and [1:N] for each instance.

Parameters:

• segmentation (torch.Tensor | np.ndarray) – Segmentation mask Shape: ([D, ]H, W), dtype: int

• inplace (bool) – Modify in place. Default: True

Returns:

Relabeled segmentation (same object if inplace=True)

Return type:

torch.Tensor | np.ndarray

byotrack.api.detections.detections.compress(tensor: Tensor, level: int = 3) → Tensor

Compress a tensor using zstd.

byotrack.api.detections.detections.decompress(tensor: Tensor, dtype: dtype = torch.int32) → Tensor

Decompress a zstd-compressed tensor.

byotrack.api.detections.detections.cached(fn: Callable[[_D], torch.Tensor]) → Callable[[_D], torch.Tensor]

Enable detections properties caching.

Reads and writes self._cache (a dict[str, torch.Tensor]) using the wrapped method’s __name__ as the key. Caching is skipped when self._use_cache is False or if self has a backing attribute f'_{fn.__name__}' (i.e. the property is simply derived from a dedicated storage).

Note that segmentation is compressed before caching if self._compress is set.

Parameters:

fn – Method to wrap. Its __name__ is used as the cache key.

Returns:

Wrapped callable that checks / populates self._cache.

class byotrack.api.detections.detections.Detections(*, confidence: Tensor | None = None, labels: Tensor | None = None, cache: bool = True, compress: bool = False)

Bases: ABC

Abstract base class for frame-level detections.

Represents the set of detected objects in a single video frame. Three concrete implementations exist:

• PointDetections: center positions + optional spot radius

• BBoxDetections: axis-aligned bounding boxes

• SegmentationDetections: full instance-segmentation mask

All implementations expose the same properties: position, bbox, segmentation, confidence, length, dim, shape, mass.

The easiest way to create a Detections object from raw array-like data is via as_detections(), which automatically selects the right subclass based on the shape and dtype of the input.

Note

All positions/bounding boxes use the index coordinate system (not xyz). In ByoTrack the depth axis (k) comes before height (i, row) and width (j, column): (k, i, j) in 3-D, (i, j) in 2-D.

Note

Properties are derived lazily on first access and cached (controlled by the cache constructor argument). For example, PointDetections caches bbox and segmentation the first time they are requested; subsequent accesses return the cached tensor without recomputation.

Note

The segmentation tensor can additionally be stored in compressed form (ZSTD) to reduce memory usage, controlled by the compress constructor argument (defaults to the ZSTD_SEG environment variable).

length

Number of detections (N).

Type:

int

dim

Spatial dimension, i.e. 2 or 3.

Type:

int

shape

Image shape ([D, ]H, W).

Type:

tuple[int, …]

position

Center positions. Shape: (N, dim), dtype: float32.

Type:

torch.Tensor

bbox

Bounding boxes ([front, ]top, left, [depth, ]height, width), Shape: (N, 2*dim), dtype: int32.

Type:

torch.Tensor

segmentation

Instance segmentation mask. Labels are consecutive, from 1 to the length (N), where the detection i is labeled i+1. 0 is assigned to each background pixel. Shape: ([D, ]H, W), dtype: int32.

Type:

torch.Tensor

confidence

Per-detection confidence score. Defaults to all-ones. Shape: (N,), dtype: float32.

Type:

torch.Tensor

labels

Labels of the detections. Defaults to consecutive labels from 0 to N-1. If given, stored and used to draw labeled_segmentation. As 0 is the background, labels drawn on segmentation start at 1 (off-by-one). Shape: (N,), dtype: int32.

Type:

torch.Tensor

mass

Per-detection pixel count (or approximation). Shape: (N,), dtype: int32.

Type:

torch.Tensor

labeled_segmentation

Segmentation mask using the labels. As for the segmentation, detection i is labeled with labels[i] + 1. For SegmentationDetections, it provides a mapping to the original segmentation map which has been relabeled consecutively internally. Shape: ([D, ]H, W), dtype: int32.

Type:

torch.Tensor

metadata

Arbitrary per-detection tensors stored by external components (e.g. byotrack.FeaturesExtractor). Not persisted by save().

Type:

dict[str, torch.Tensor]

abstractmethod filter(kept: Tensor) → Detections

Filter the detections based on a boolean tensor.

Parameters:

kept (torch.Tensor) – Detection to keep. Shape: (N,), dtype: bool

Returns:

Filtered detections.

Return type:

byotrack.Detections

save(path: str | os.PathLike) → None

Save detections to a file using torch.save.

Parameters:

path (str | os.PathLike) – Output path (expected .pt extension).

static load(path: str | os.PathLike, *, cache: bool = True, compress: bool = False) → Detections

Load detections from a file written by save().

Dispatches to the appropriate subclass based on the "_type" key.

Parameters:

• path (str | os.PathLike) – Input path.

• cache (bool) – Cache lazily-computed properties. Default: True.

• compress (bool) – Compress the segmentation mask in memory using ZSTD. Defaults to the ZSTD_SEG environment variable value.

Returns:

The loaded detections object.

Return type:

Detections

static save_multi_frames_detections(detections_sequence: Sequence[Detections], path: str | os.PathLike) → None

Save a sequence of per-frame detections as {path}/0.pt, 1.pt, …

Parameters:

• detections_sequence (Sequence[Detections]) – Detections for each frame.

• path (str | os.PathLike) – Output folder (created if absent).

static load_multi_frames_detections(path: str | os.PathLike) → list[Detections]

Load a sequence of per-frame detections from a folder.

Expects files named 0.pt, 1.pt, …, N.pt in path.

Parameters:

path (str | os.PathLike) – Input folder.

Returns:

Detections for each frame (ordered by index).

Return type:

list[Detections]

byotrack.api.detections.detections.as_detections(data: Detections | ndarray | Tensor, **kwargs: Any) → Detections

Convert array-like data to the appropriate Detections subclass.

Heuristic rules:

1. Detections instance -> returned unchanged.

2. 2-D floating tensor/array of shape (N, 2) or (N, 3) -> PointDetections.

3. 2-D integer tensor/array of shape (N, 4) or (N, 6) -> BBoxDetections.

4. 2-D or 3-D integer tensor/array -> SegmentationDetections.

Parameters:

• data (DetectionsLike) – Input data.

• **kwargs – Forwarded to the chosen subclass constructor.

Returns:

Wrapped detections object.

Return type:

Detections

Raises:

ValueError – If the format cannot be determined.

Instance Segmentation

SegmentationDetections: detections represented by a full instance-segmentation mask.

class byotrack.api.detections.segmentation_detections.SegmentationDetections(segmentation: torch.Tensor, *, confidence: torch.Tensor | None = None, position_method: str | Callable = 'median', cache: bool = True, compress: bool = False)

Bases: Detections

Detections represented by a full instance-segmentation mask.

The primary data is a 2-D or 3-D integer tensor of shape ([D, ]H, W). Labels can be any non-negative integers (consecutive or not); 0 is always background. Non-consecutive labels are stored in labels so the original mapping is preserved; internally the mask is relabelled to the consecutive sequence 1, …, N for efficient computation.

All other properties (position, bbox, mass) are derived lazily.

segmentation

Instance segmentation mask. Labels are consecutive, from 1 to the length (N), where the detection i is labeled i+1. 0 is assigned to each background pixel. Shape: ([D, ]H, W), dtype: int32.

Type:

torch.Tensor

labels

Labels of the detections. Defaults to the labels of the input segmentation mask so that labeled_segmentation remap the input segmentation. As 0 is the background, labels drawn on segmentation start at 1 (off-by-one). Shape: (N,), dtype: int32.

Type:

torch.Tensor

filter(kept: Tensor) → SegmentationDetections

Filter the detections based on a boolean tensor.

Parameters:

kept (torch.Tensor) – Detection to keep. Shape: (N,), dtype: bool

Returns:

Filtered detections.

Return type:

byotrack.Detections

Bounding Box

BBoxDetections: detections represented as axis-aligned bounding boxes.

class byotrack.api.detections.bbox_detections.BBoxDetections(bbox: Tensor, *, confidence: Tensor | None = None, labels: Tensor | None = None, shape: tuple[int, ...] | None = None, cache: bool = True, compress: bool = False)

Bases: Detections

Detections represented as axis-aligned bounding boxes.

The primary data is a tensor of bounding boxes (N, 2*dim) in the format ([front, ]top, left, [depth, ]height, width) — i.e. start coordinates followed by sizes in index coordinates.

All other properties (position, segmentation, mass) are derived lazily.

Note

Bounding boxes use integer index coordinates. All box sizes must be strictly positive; a zero-size box raises ValueError at construction time.

bbox

Bounding boxes ([k, ]i, j, [dk, ], di, dj) of each detection in index coordinates. Shape: (N, 2*dim), dtype: int32.

Type:

torch.Tensor

filter(kept: Tensor) → BBoxDetections

Filter the detections based on a boolean tensor.

Parameters:

kept (torch.Tensor) – Detection to keep. Shape: (N,), dtype: bool

Returns:

Filtered detections.

Return type:

byotrack.Detections

Point

PointDetections: detections represented as center positions with optional spot radii.

class byotrack.api.detections.point_detections.PointDetections(position: Tensor, *, radius: float | Tensor = 2.0, confidence: Tensor | None = None, labels: Tensor | None = None, shape: tuple[int, ...] | None = None, cache: bool = True, compress: bool = False)

Bases: Detections

Detections represented as center positions with an optional spot radius.

The primary data is a tensor of center positions (N, dim). All other properties (bbox, segmentation, mass) are derived lazily from the positions and radius.

position

Position ([k, ]i, j) of each detection in index coordinates. Shape: (N, dim), dtype: float32.

Type:

torch.Tensor

radius

Per detection and axis spot radius for segmentation / bbox conversion. Shape: (N, dim), dtype: float32.

Type:

torch.Tensor

filter(kept: Tensor) → PointDetections

Filter the detections based on a boolean tensor.

Parameters:

kept (torch.Tensor) – Detection to keep. Shape: (N,), dtype: bool

Returns:

Filtered detections.

Return type:

byotrack.Detections

Statistics utils

Statistics utilities for sequences of detections.

Provides functions to estimate aggregate properties (mass, radius, nearest-neighbor distance, anisotropy) over a sequence of per-frame byotrack.Detections objects.

byotrack.api.detections.statistics.average_mass(detections_sequence: Sequence[byotrack.Detections]) → float

Return the average mass (pixel surface or volume) per detection over a sequence of frames.

Parameters:

detections_sequence (Sequence[byotrack.Detections]) – Sequence of per-frame detections.

Returns:

Average number of pixels (2D) or voxels (3D) per detection. Returns 0.0 if the

sequence is empty or contains no detections.

Return type:

float

byotrack.api.detections.statistics.average_radius(detections_sequence: Sequence[byotrack.Detections], anisotropy: tuple[float, float, float] = (1.0, 1.0, 1.0)) → float

Return the average radius of detections over a sequence of frames.

Assumes each detection is roughly spherical (3D) or circular (2D), and derives the radius from the average mass using the corresponding volume formula:

• 2D: mass = π * R² => R = sqrt(mass / π)

• 3D: mass = 4/3 * π * R³ => R = (3 * mass / (4 * π)) ** (1/3)

Parameters:

• detections_sequence (Sequence[byotrack.Detections]) – Sequence of per-frame detections.

• anisotropy (tuple[float, float, float]) – Anisotropy factors (ani_z, ani_y, ani_x) used to scale the average mass before computing the radius. These factors convert voxel coordinates to isotropic ones. Default: (1.0, 1.0, 1.0) (no scaling).

Returns:

Average detection radius in isotropic coordinates. Returns 0.0 if the sequence

is empty.

Return type:

float

byotrack.api.detections.statistics.average_min_dist(detections_sequence: Sequence[byotrack.Detections], anisotropy: tuple[float, float, float] = (1.0, 1.0, 1.0)) → float

Return the average minimal distance between two detections in the same frame.

For each frame with at least two detections, computes the per-detection minimum distance to its nearest neighbor, then takes the median across all detections in that frame (to reduce the influence of outliers). The result is averaged over all eligible frames.

Parameters:

• detections_sequence (Sequence[byotrack.Detections]) – Sequence of per-frame detections. Frames with fewer than two detections are ignored.

• anisotropy (tuple[float, float, float]) – Anisotropy factors (ani_z, ani_y, ani_x) used to scale detection positions before computing distances. Only the last dim elements of the tuple are applied. Default: (1.0, 1.0, 1.0) (no scaling).

Returns:

Average (over frames) of the per-frame median nearest-neighbor distance.

Returns 0.0 if no frame contains at least two detections.

Return type:

float

byotrack.api.detections.statistics.anisotropy(detections_sequence: Sequence[byotrack.Detections], *, only_depth=True) → tuple[float, float, float]

Return the average anisotropy found in the detections based on their size.

It makes the assumption that objects do not have a preferential direction and therefore their average size should be isotrope.

The anisotropy is defined as the scaling factors (ani_z, ani_y, ani_x) to scale original coordinates to isotrope ones.

This always takes the last dimension (X) as reference and therefore anisotropy = (ani_z, ani_y, 1).

If only_depth is set to true, the two last dimensions (YX) are used as references and only the axis Z is anisotrope: anisotropy = (ani_z, 1, 1).