TrackOnSTra Linker

Online implementation of TrackAstra.

byotrack.implementation.linker.frame_by_frame.trackonstra.build_cost_dict(nodes: list[dict[str, Any]], weights: Iterable[tuple[tuple[int, int], float]]) → dict[tuple[int, int, int], dict[int, float]]

Build the cost dictionary from Trackastra data format.

It converts the feasible tracking graph predicted by trackastra into a mapping from edge to cost. Where an edge is a link between two detections (node). It converts the probability weight into a cost = -log weight.

Parameters:

nodes (list[dict[str, Any]]) – Nodes of the graph (detections), with their features Format: { “id”: int, “coords”: (float, float), “time”: int, “label”: int}
weights (Iterable[tuple[tuple[int, int], float]]) – Considered edges, with their weight Format: [((node_id, node_id_2), weight), …]

Returns:

Cost dictionary of the feasible tracking graph. For every (frame1, node1, frame2) there is a dictionary with key node2 and the value the cost of the edge.

Return type:

dict[tuple[int, int, int], dict[int, float]]

class byotrack.implementation.linker.frame_by_frame.trackonstra.TrackastraFlex(transformer: TrackingTransformer, train_args: dict[str, Any], delta_t: int = 4, intra_weight: float = 0, device=None)

Bases: Trackastra

Trackastra with ability to modify delta_t.

predict_with_gap(imgs: ndarray, masks: ndarray, edge_threshold: float = 0.05, n_workers: int = 0) → dict: Same function as the original _predict but it calls predict_windows with the model delta_t.

class byotrack.implementation.linker.frame_by_frame.trackonstra.TrackOnStraParameters(*, association_threshold: float = 0.05, positional_cutoff: float = 256.0, n_valid=2, n_gap=3, association_method: str | AssociationMethod = AssociationMethod.OPT_SMOOTH, anisotropy: tuple[float, float, float] = (1.0, 1.0, 1.0), split_factor: float = 0.0, merge_factor: float = 0.0)

Bases: FrameByFrameLinkerParameters

Parameters of TrackOnStraLinker.

association_threshold

Minimum probability to consider a link. By default, we use the value from Trackstra. Default: 0.05

Type:: float

positional_cutoff

Defines an Euclidean threshold on links. We use the default value provided by Trackastra. Tuning it improves performance. Default: 256.0

Type:: float

n_valid

Number associated detections required to validate the track after its creation. Default: 2

Type:: int

n_gap

Number of consecutive frames without association before the track termination. Default: 3

Type:: int

association_method

The frame-by-frame association to use. See AssociationMethod. It can be provided as a string. (Choice: GREEDY, OPT_HARD, OPT_SMOOTH, SPARSE_OPT_HARD, SPARSE_OPT_SMOOTH) Default: OPT_SMOOTH

Type:: AssociationMethod

anisotropy

Anisotropy of images (Ratio of the pixel sizes for each axis, depth first). This will be used to scale distances. Default: (1., 1., 1.)

Type:: tuple[float, float, float]

split_factor

Allow splitting of tracks, using a second association step. The association threshold in this case is split_factor * association_threshold. Default: 0.0 (No splits)

Type:: float

merge_factor

Allow merging of tracks, using a second association step. The association threshold in this case is merge_factor * association_threshold. Default: 0.0 (No merges)

Type:: float

class byotrack.implementation.linker.frame_by_frame.trackonstra.TrackOnStraLinker(specs: TrackOnStraParameters, model: TrackastraFlex | None = None, optflow: byotrack.OpticalFlow | None = None, features_extractor: byotrack.FeaturesExtractor | None = None)

Bases: FrameByFrameLinker

Online TrackAstra.

It uses a trained TrackAstra model to predict linking costs. But it replaces the graph optimization from TrackAstra by our online FrameByFrame linking. This allows a simple support for false negative detections that TrackAstra do not support by itself.

Warning: This implementation is not yet Online. Indeed the linker has to be setup with: the full video and detections_sequence before being usable.

Note

This implementation requires trackastra. (pip install trackastra) trackastra is only available for python >= 3.10

See FrameByFrameLinker for the other attributes.

specs

Parameters specifications of the algorithm. See TrackOnStraParameters.

Type:: TrackOnStraParameters

model

Model of Trackastra used to compute the association costs.

Type:: TrackastraFlex

cost_dict

Cost dictionary of the feasible tracking graph.

Type:: dict[tuple[int, int, int], dict[int, float]

setup(video: Sequence[np.ndarray] | np.ndarray, detections_sequence: Sequence[byotrack.Detections]) → None

Offline setup of the linker by computing all the linking costs with Trackastra.

Linking costs are stored into cost_dict.

This function needs to be called on each video for the linking to be able to run.

Parameters:

video (Sequence[np.ndarray] | np.ndarray) – Sequence of T frames (array). Each frame is expected to have a shape ([D, ]H, W, C)
detections_sequence (Sequence[byotrack.Detections]) – Detections for each frame Detections is expected for each frame of the video, in the same order. (Note that for a given frame, the Detections can be empty)

motion_model() → None

Optional modelisation of motion for tracks.

It can be used to update some internal state of the tracker after the optical flow computation and before the distance computation.

post_association(frame: np.ndarray, detections: byotrack.Detections, active_mask: torch.Tensor) → None

Update the internal state of the tracker after update_active_tracks.

It should update any internal model/data. It is also responsible to register the position of each active track in all_positions for the current time frame.

Parameters:

frame (np.ndarray) – The current frame of the video Shape: (H, W, C), dtype: float
detections (byotrack.Detections) – Detections for the given frame
active_mask (torch.Tensor) – Boolean tensor indicating True for still active tracks Shape: (N_tracks), dtype: bool

cost(frame: np.ndarray, detections: byotrack.Detections) → tuple[torch.Tensor, float]

Compute the association cost between active tracks and detections.

It also returns the threshold to use (Depending on the dist you use, association_threshold could be related to a more meaning full quantity than the cost itself). For instance, when using a squared Euclidean distance, the association threshold could be express as the distance in pixel, and this function could square it. For likelihood association, you could provide the association threshold as a probability and use -log(threshold) as the true threshold. (See KalmanLinker and NearestNeighborLinker)

Parameters:

frame (np.ndarray) – The current frame of the video Shape: (H, W, C), dtype: float
detections (byotrack.Detections) – Detections for the given frame

Returns:

The cost matrix between active tracks and detections: Shape: (n_tracks, n_dets), dtype: float
float: The association threshold to use.: It can be different than self.association_threshold depending on the dist build here

Return type:

torch.Tensor