Cell Tracking Challenge (CTC)

Wrappers around CTC metrics [10]

We currently only wrap the CTC software providing SEG, DET and TRA metrics.

In the future, we may wrap the Fiji plugin to access BIO metrics.

class byotrack.metrics.ctc.CTCSoftwareRunner(ctc_software: str | PathLike)

Bases: object

Runs CTC software (TRA/DET/SEG) [10]

It wraps the TRA/DET/SEG softwares that are distributed at https://celltrackingchallenge.net/evaluation-methodology It is up to the user to download the correct executable and ensure that it works.

These softwares take as inputs the path to the folder of the dataset (with ground-truth and results tracks), the sequence id to evaluate and the number of digits used to encode time in file names. (3 or 4)

They output a string with the measure score, that we parse.

Note

This code requires CTC softwares as extra dependencies. They can be downloaded at https://celltrackingchallenge.net/evaluation-methodology.

ctc_software

Path to the ctc software folder. It should be the root folder containing Win/Linux/Mac subfolders with their executables.

Type:: pathlib.Path

system

The user system in the CTC nomenclature. One of Linux, Mac or Win.

Type:: str

last_log

Logs of the last computed metrics

Type:: str

run(metric: str, dataset: str | PathLike, seq=1, n_digit=4) → float

Run the CTC software for the given metric and dataset

Parameters:

metric (str) – The metric to evaluate. One of (TRA, DET, SEG).
dataset (Union[str, os.PathLike]) – Path to the dataset to evaluate.
seq (int) – Sequence to evaluate inside the dataset. CTC softwares will compare {dataset}/{seq:02}_RES with {dataset}/{seq:02}_GT Default: 1
n_digit (int) – Number of digits used to encode time in file names. It is dataset dependant, but in ByoTrack, by default we use 4 digits. Default: 4

Returns:

The evaluated metric

Return type:

float

class byotrack.metrics.ctc.CTCMetrics(ctc_software: str | PathLike)

Bases: CTCSoftwareRunner

Wrapper around CTC metrics. [10]

It wraps the TRA/DET/SEG softwares that are distributed at https://celltrackingchallenge.net/evaluation-methodology It is up to the user to download the correct executable and ensure that it works.

To compute a metric, the code creates a temporary folder, where it saves the ground-truth tracks alongside the predicted tracks. Then it runs the required CTC software, and finally remove the temporary folder. See CTCSoftwareRunner.

Note

This code requires CTC softwares as extra dependencies. They can be downloaded at https://celltrackingchallenge.net/evaluation-methodology.

We provide two distincts methods to evaluate the metrics. One based on tracks, the other on detections:

1. compute_detection_metrics compute the DET and SEG using the results of a segmentation algorithm (no tracks information is required). Note that TRA cannot be computed in this case. 2. compute_tracking_metrics compute the metrics for tracks. One can compute the DET and SEG for the tracks, but these will not match with the ones computed for detections. Indeed, tracks filter out some detections and may add points to tracks to fill missing detections. Usually, this leads to better DET/SEG metrics as it reduces false positive and false negative.

It will store the logs of the last called metric in self.last_log

compute_tracking_metric(metric: str, tracks: Collection[Track], ground_truth_tracks: str | PathLike | Collection[Track], *, detections_sequence: Sequence[Detections] = (), ground_truth_detections_sequence: Sequence[Detections] = (), **kwargs) → float

Compute the given metric for the given tracks and ground truthes.

It will create a temporary folder, to store the tracks and ground-truthes in the right format and then execute the CTC software. The temporary folder is removed at the end.

See run to simply compute the metric of an already existing folder.

Note

In CTC, matching of predicted tracks with GT ones is done based on a kind of IOU. Therefore, it may be useful to provide the detections_sequence associated with the tracks. You may also tweak the default_radius arguments (in kwargs). (See ctc.save_tracks)

Note

This function computes the tracking metrics, which are different than those computed with compute_detection_metric for SEG and DET. For tracking metrics, we export the segmentation of tracks, rather than the detections_sequence. Tracks segmentation differs from the predicted segmentation, as the tracks processus filters out detections and may predict tracks position for missing ones. (It usually improves the segmentation metrics, as it reduces false positive and false negative)

Parameters:

metric (str) – The metric to evaluate. One of (TRA, DET, SEG).
tracks (Collection[byotrack.Track]) – Predicted tracks to evaluate.
ground_truth_tracks (Union[str, os.PathLike, Collection[byotrack.Track]]) – Ground truth data. It is either a path to the GT tracks folder, which will be copied in our temporary folder. Or it is a list of ByoTrack.Track, that will be saved in the temporary folder.
detections_sequence (Sequence[byotrack.Detections]) – Optional detections, used when saving the tracks. Default: () # No detections and tracks segmentations will be disk of radius default_radius.
ground_truth_detections_sequence (Sequence[byotrack.Detections]) – Optional detections for ground-truth. When saving the GT tracks, these detections are used. See ctc.save_tracks. Default: () # No detections and tracks segmentations will be disk of radius default_radius.
**kwargs – Additional arguments to provide to the ctc.save_tracks function. ‘shape’: Provide the shape of saved image. It is mandatory if no detections is provided. ‘default_radius’: Radius of the disk drawn for tracks that have no detections. ‘last’: Last frame to consider. (Typically to shorten the sequences, or if no object is tracked on the last frames, this will enforce the creation of empty tiff files)

compute_detection_metric(metric: str, detections_sequence: Sequence[Detections], ground_truth_detections_sequence: str | PathLike | Sequence[Detections]) → float

Compute the given metric for the given detections and ground truthes.

It will create a temporary folder, to store the detections and ground-truthes in the right format and then execute the CTC software. The temporary folder is removed at the end.

See run to simply compute the metric of an already existing folder.

Note

This function truly computes the detections metrics, which are different than those computed with compute_tracking_metric for SEG and DET. For tracking metrics, we export the segmentation of tracks, rather than the detections_sequence. Tracks segmentation differs from the predicted segmentation, as the tracks processus filters out detections and may predict tracks position for missing ones. (It usually improves the segmentation metrics, as it reduces false positive and false negative)

Parameters:

metric (str) – The metric to evaluate. One of (TRA, DET, SEG).
tracks (Collection[byotrack.Track]) – Predicted tracks to evaluate.
ground_truth_tracks (Union[str, os.PathLike, Collection[byotrack.Track]]) – Ground truth data. It is either a path to the GT tracks folder (), which will be copied in our temporary folder. Or it is a list of ByoTrack.Track, that will be saved in the temporary folder.
detections_sequence (Sequence[byotrack.Detections]) – Optional detections, used when saving the tracks. Default: () # No detections and tracks segmentations will be disk of radius default_radius.
ground_truth_detections_sequence (Sequence[byotrack.Detections]) – Optional detections for ground-truth. When saving the GT tracks, these detections are used. See ctc.save_tracks. Default: () # No detections and tracks segmentations will be disk of radius default_radius.

static copy_ground_truth(ground_truth_path: str | PathLike, target_path: str | PathLike, as_seg=False) → None

Copy the ground-truth to the target

It will copy any files matching maskT.tif, res_track.txt, man_trackT.tif, man_segT.tif, man_track.txt. Nomenclature of files is adapted to match ground truth one. Also, files with 3 digits to encode the time information, will be saved with 4 digits.

Note

It is probably more permissive than the software, allowing to convert from/to SEG to/from TRA annotations. It also supports converting results into annotations (SEG and TRA). In some cases, it may produce invalid data for CTC softwares, for instance converting SEG GT into TRA ones only works if SEG GT are fully annotated and it only allows to run DET measures, as the metadata txt file would be missing.

Parameters:

ground_truth_path (Union[str, os.PathLike]) – Path to ground truthes. We support any directory containing potential files to copy. (In particular it can be 01_GT/SEG or 01_GT/TRA)
target_path (Union[str, os.PathLike]) – Path where to copy files.
as_seg (bool) – Save target_files as man_segT.tif instead of man_trackT.tif Default: False

class byotrack.metrics.ctc.BioMetrics(fiji_path: str | PathLike)

Bases: object

Wrapper around the CTC “Biological measures” Fiji plugin [10]

It allows the computations of BIO metrics for the Cell Tracking Challenge [10].

Note

This implementation requires Fiji to be installed (https://imagej.net/downloads) with the CTC plugins (https://github.com/CellTrackingChallenge/fiji-plugins)

runner

Fiji runner

Type:: byotrack.fiji.FijiRunner

last_metrics

Sub metrics information for the last computed BIO It consists of (“CT”, “TF”, “BC(i)”, “CCA”). Their average gives the BIO metric.

Type:: Tuple[float, float, float, float]

run(dataset: str | PathLike, seq=1, n_digit=4) → float

Run the CTC “Biological measures” plugin on the given dataset.

The dataset should already have results stored in it. It expects the CTC format.

Parameters:

metric (str) – The metric to evaluate. One of (TRA, DET, SEG).
dataset (Union[str, os.PathLike]) – Path to the dataset to evaluate.
seq (int) – Sequence to evaluate inside the dataset. The plugin will compare {dataset}/{seq:02}_RES with {dataset}/{seq:02}_GT Default: 1
n_digit (int) – Number of digits used to encode time in file names. It is dataset dependant, but in ByoTrack, by default we use 4 digits. Default: 4

Returns:

The evaluated metric

Return type:

float

compute(tracks: Collection[Track], ground_truth_tracks: str | PathLike | Collection[Track], *, detections_sequence: Sequence[Detections] = (), ground_truth_detections_sequence: Sequence[Detections] = (), **kwargs) → float

Compute BIO metric for the given tracks and ground truthes.

It will create a temporary folder, to store the tracks and ground-truthes in the right format and then execute the CTC plugins. The temporary folder is removed at the end.

See run to simply compute the metric of an already existing folder.

Note

In CTC, matching of predicted tracks with GT ones is done based on a kind of IOU. Therefore, it may be useful to provide the detections_sequence associated with the tracks. You may also tweak the default_radius arguments (in kwargs). (See ctc.save_tracks)

Parameters:

tracks (Collection[byotrack.Track]) – Predicted tracks to evaluate.
ground_truth_tracks (Union[str, os.PathLike, Collection[byotrack.Track]]) – Ground truth data. It is either a path to the GT tracks folder, which will be copied in our temporary folder. Or it is a list of ByoTrack.Track, that will be saved in the temporary folder.
detections_sequence (Sequence[byotrack.Detections]) – Optional detections, used when saving the tracks. Default: () # No detections and tracks segmentations will be disk of radius default_radius.
ground_truth_detections_sequence (Sequence[byotrack.Detections]) – Optional detections for ground-truth. When saving the GT tracks, these detections are used. See ctc.save_tracks. Default: () # No detections and tracks segmentations will be disk of radius default_radius.
**kwargs – Additional arguments to provide to the ctc.save_tracks function. ‘shape’: Provide the shape of saved image. It is mandatory if no detections is provided. ‘default_radius’: Radius of the disk drawn for tracks that have no detections. ‘last’: Last frame to consider. (Typically to shorten the sequences, or if no object is tracked on the last frames, this will enforce the creation of empty tiff files)