Nearest Neighbor Linker

class byotrack.implementation.linker.frame_by_frame.nearest_neighbor.NearestNeighborParameters(association_threshold: float = 5.0, *, n_valid=3, n_gap=3, association_method: str | AssociationMethod = AssociationMethod.OPT_SMOOTH, anisotropy: tuple[float, float, float] = (1.0, 1.0, 1.0), ema=0.0, fill_gap=False, split_factor: float = 0.0, merge_factor: float = 0.0)

Bases: FrameByFrameLinkerParameters

Parameters of NearestNeighborLinker.

Note

The merging and splitting features is still experimental.

association_threshold

This is the main hyperparameter, it defines the threshold on the distance used not to link tracks with detections. It prevents to link with false positive detections. Default: 5 pixels

Type:: float

n_valid

Number associated detections required to validate the track after its creation. Default: 3

Type:: int

n_gap

Number of consecutive frames without association before the track termination. Default: 3

Type:: int

association_method

The frame-by-frame association to use. See AssociationMethod. It can be provided as a string. (Choice: GREEDY, OPT_HARD, OPT_SMOOTH, SPARSE_OPT_HARD, SPARSE_OPT_SMOOTH) Default: OPT_SMOOTH

Type:: AssociationMethod

anisotropy

Anisotropy of images (Ratio of the pixel sizes for each axis, depth first). This will be used to scale distances. Default: (1., 1., 1.)

Type:: tuple[float, float, float]

ema

Optional exponential moving average to reduce detection noise. Detection positions are smoothed using this EMA. Should be smaller than 1. It use: x_{t+1} = ema x_{t} + (1 - ema) det(t) As motion is not modeled, EMA may introduce lag that will hinder tracking. It is more effective with optical flow to compensate motions, in this case, a typical value is 0.5, to average the previous position with the current measured one. For more advanced modelisation, see KalmanLinker. Default: 0.0 (No EMA)

Type:: float

fill_gap

Fill the gap of missed detections using a forward optical flow propagation (Only when optical flow is provided). We advise to rather use a ForwardBackward interpolation using the same optical flow: it will produce smoother interpolations. Default: False

Type:: bool

split_factor

Allow splitting of tracks, using a second association step. The association threshold in this case is split_factor * association_threshold. Default: 0.0 (No splits)

Type:: float

merge_factor

Allow merging of tracks, using a second association step. The association threshold in this case is merge_factor * association_threshold. Default: 0.0 (No merges)

Type:: float

class byotrack.implementation.linker.frame_by_frame.nearest_neighbor.NearestNeighborLinker(specs: NearestNeighborParameters, optflow: byotrack.OpticalFlow | None = None, features_extractor: byotrack.FeaturesExtractor | None = None, *, save_all=False)

Bases: FrameByFrameLinker

Frame by frame linker by associating with the closest detections.

Motion is not modeled, but if an optical flow method is provided, it will be used to compensate motion online. Matching is done from a simple Euclidean distance. This can be easily changed by inheriting this class and overriding the cost method.

See FrameByFrameLinker for the other attributes.

specs

Parameters specifications of the algorithm. See NearestNeighborParameters.

Type:: NearestNeighborParameters

active_positions

The positions of actives tracks, if undetected it is estimated by optical flow propagation, or simply propagated if no optical flow is given. Shape: (N, D), dtype: float32

Type:: torch.Tensor

reset(dim=2) → None

Reset the linking algorithm.

Flush all data stored from a previous linking and prepare a new linking.

Parameters:: dim (int) – The dimension of the data. Default: 2

motion_model() → None

Optional modelisation of motion for tracks.

It can be used to update some internal state of the tracker after the optical flow computation and before the distance computation.

cost(frame: np.ndarray, detections: byotrack.Detections) → tuple[torch.Tensor, float]

Compute the association cost between active tracks and detections.

It also returns the threshold to use (Depending on the dist you use, association_threshold could be related to a more meaning full quantity than the cost itself). For instance, when using a squared Euclidean distance, the association threshold could be express as the distance in pixel, and this function could square it. For likelihood association, you could provide the association threshold as a probability and use -log(threshold) as the true threshold. (See KalmanLinker and NearestNeighborLinker)

Parameters:

frame (np.ndarray) – The current frame of the video Shape: (H, W, C), dtype: float
detections (byotrack.Detections) – Detections for the given frame

Returns:

The cost matrix between active tracks and detections: Shape: (n_tracks, n_dets), dtype: float
float: The association threshold to use.: It can be different than self.association_threshold depending on the dist build here

Return type:

torch.Tensor

post_association(frame: np.ndarray, detections: byotrack.Detections, active_mask: torch.Tensor) → None

Update the internal state of the tracker after update_active_tracks.

It should update any internal model/data. It is also responsible to register the position of each active track in all_positions for the current time frame.

Parameters:

frame (np.ndarray) – The current frame of the video Shape: (H, W, C), dtype: float
detections (byotrack.Detections) – Detections for the given frame
active_mask (torch.Tensor) – Boolean tensor indicating True for still active tracks Shape: (N_tracks), dtype: bool