Nearest Neighbor Linker

class byotrack.implementation.linker.frame_by_frame.nearest_neighbor.NearestNeighborParameters(association_threshold: float = 5.0, *, n_valid=3, n_gap=3, association_method: str | AssociationMethod = AssociationMethod.OPT_SMOOTH, ema=0.0, fill_gap=False)

Bases: FrameByFrameLinkerParameters

Parameters of NearestNeighborLinker

association_threshold

This is the main hyperparameter, it defines the threshold on the distance used not to link tracks with detections. It prevents to link with false positive detections. Default: 5 pixels

Type:

float

n_valid

Number of frames with a correct association required to validate the track at its creation. Default: 3

Type:

int

n_gap

Number of frames with no association before the track termination. Default: 3

Type:

int

association_method

The frame-by-frame association to use. See AssociationMethod. It can be provided as a string. (Choice: GREEDY, OPT_HARD, OPT_SMOOTH) Default: OPT_SMOOTH

Type:

AssociationMethod

ema

Optional exponential moving average to reduce detection noise. Detection positions are smoothed using this EMA. Should be smaller than 1. It use: x_{t+1} = ema x_{t} + (1 - ema) det(t) As motion is not modeled, EMA may introduce lag that will hinder tracking. It is more effective with optical flow to compensate motions, in this case, a typical value is 0.5, to average the previous position with the current measured one. For more advanced modelisation, see KalmanLinker. Default: 0.0 (No EMA)

Type:

float

fill_gap

Fill the gap of missed detections using a forward optical flow propagation (Only when optical flow is provided). We advise to rather use a ForwardBackward interpolation using the same optical flow: it will produce smoother interpolations. Default: False

Type:

bool

class byotrack.implementation.linker.frame_by_frame.nearest_neighbor.NearestNeighborLinker(specs: NearestNeighborParameters, optflow: OpticalFlow | None = None, features_extractor: FeaturesExtractor | None = None, save_all=False)

Bases: FrameByFrameLinker

Frame by frame linker by associating with the closest detections

Motion is not modeled, but if an optical flow method is provided, it will be used to compensate motion online. Matching is done from a simple Euclidean distance. This can be easily changed by inheriting this class and overriding the cost method.

See FrameByFrameLinker for the other attributes.

specs

Parameters specifications of the algorithm. See NearestNeighborParameters.

Type:

NearestNeighborParameters

active_positions

The positions of actives tracks, if undetected it is estimated by optical flow propagation. Shape: (N, D), dtype: float32

Type:

Optional[torch.Tensor]

reset() → None

Reset the linking algorithm

Flush all data stored from a previous linking and prepare a new linking

motion_model() → None

Optional modelisation of motion for tracks

It can be used to update some internal state of the tracker after the optical flow computation and before the distance computation.

cost(_: ndarray, detections: Detections) → Tuple[Tensor, float]

Compute the association cost between active tracks and detections

It also returns the threshold to use (Depending on the dist you use, association_threshold could be related to a more meaning full quantity than the cost itself). For instance, when using a squared Euclidean distance, the association threshold could be express as the distance in pixel, and this function could square it. For likelihood association, you could provide the association threshold as a probability and use -log(threshold) as the true threshold. (See KalmanLinker and NearestNeighborLinker)

Parameters:

  • frame (np.ndarray) – The current frame of the video Shape: (H, W, C), dtype: float

  • detections (byotrack.Detections) – Detections for the given frame

Returns:

The cost matrix between active tracks and detections

Shape: (n_tracks, n_dets), dtype: float32

float: The association threshold to use.

It can be different than self.association_threshold depeding on the dist build here

Return type:

torch.Tensor

post_association(_: ndarray, detections: Detections, links: Tensor)

Update the tracks and the internal variables of the tracker

It should call the update method of each active tracks and update any internal model/data. It should also create new track handlers for each extra detection. Finally, it is also responsible to register the position of each active track in all_positions for the current time frame.

Parameters:

  • frame (np.ndarray) – The current frame of the video Shape: (H, W, C), dtype: float

  • detections (byotrack.Detections) – Detections for the given frame

  • links (torch.Tensor) – The links made between active tracks and the detections Shape: (L, 2), dtype: int32