Nearest Neighbor Linker

class byotrack.implementation.linker.frame_by_frame.nearest_neighbor.NearestNeighborParameters(association_threshold: float = -1.0, *, n_valid=3, n_gap=3, association_method: str | AssociationMethod = AssociationMethod.SPARSE_OPT_SMOOTH, anisotropy: tuple[float, float, float] = (1.0, 1.0, 1.0), ema=0.0, fill_gap=False, split_factor: float = 0.0, merge_factor: float = 0.0)

Bases: FrameByFrameLinkerParameters

Parameters of NearestNeighborLinker.

Note

Most parameters can be estimated automatically from the detections using estimate.

association_threshold

This is the main hyperparameter, it defines the threshold on the distance used not to link tracks with detections. A low threshold will typically reduce wrong assignments and ID-switches, but may increase track fragmentation. Higher values will reduce track fragmentation, but miss-detected tracks may be linked to a wrong detection. Default: -1.0 (to be estimated, see estimate.)

Type:: float

n_valid

Number of detections required to validate the track after its creation. If a track is missed during its first n_valid frames, it is dropped. This provides robustness to false positive detections. With no false positives, it can be set to 1 (a detection always belongs to a track). Highers values allow to remove non time-consistent false positives, but may prune real tracks that have been miss-detected. Default: 3

Type:: int

n_gap

Number of consecutive frames without any association (miss-detected) before the track termination. This provides robustness to false negative detections. Without any false negatives, it can be set to 0. Higher values allow to support larger gaps in the track, but may lead to wrong assignments. Default: 3

Type:: int

association_method

The frame-by-frame association to use. See AssociationMethod. It can be provided as a string. (Choice: GREEDY, OPT_HARD, OPT_SMOOTH, SPARSE_OPT_HARD, SPARSE_OPT_SMOOTH) Default: SPARSE_OPT_SMOOTH

Type:: AssociationMethod

anisotropy

Anisotropy of images (Ratio of the pixel sizes for each axis, depth first). This will be used to scale distances. Default: (1., 1., 1.)

Type:: tuple[float, float, float]

ema

Optional exponential moving average to reduce detection noise. Detection positions are smoothed using this EMA. Should be smaller than 1. It use: x_{t+1} = ema x_{t} + (1 - ema) det(t) As motion is not modeled, EMA may introduce lag that will hinder tracking. It is more effective with optical flow to compensate motions, in this case, a typical value is 0.5, to average the previous position with the current measured one. For more advanced modelisation, see KalmanLinker. Default: 0.0 (No EMA)

Type:: float

fill_gap

Fill the gap of missed detections using a forward optical flow propagation (Only when optical flow is provided). We advise to rather use a ForwardBackward interpolation using the same optical flow: it will produce smoother interpolations. Default: False

Type:: bool

split_factor

Allow splitting of tracks, using a second association step. The association threshold in this case is split_factor * association_threshold. Default: 0.0 (No splits)

Type:: float

merge_factor

Allow merging of tracks, using a second association step. The association threshold in this case is merge_factor * association_threshold. Default: 0.0 (No merges)

Type:: float

check(): Check the specification for invalid values.

estimate(detections_sequence) → NearestNeighborParameters

Estimate parameters from the given detections.

Estimation is triggered by providing negative dummy values for positive parameters. The dummy values are then replaced by their estimate.

Estimators:

association_threshold: max(3 * statistics.average_radius, statistics.average_min_dist)
anisotropy: Computed from statistics.anisotropy.
split_factor: 1.0 if the number of detection increase by more than 30% over the full sequence.
merge_factor: 1.0 if the number of detection decrease by more than 30% over the full sequence.

Parameters:: detections_sequence (Sequence[byotrack.Detections]) – Detections for the current sequence.
Returns:: self with updated parameters.
Return type:: NearestNeighborParameters

class byotrack.implementation.linker.frame_by_frame.nearest_neighbor.NearestNeighborLinker(specs: NearestNeighborParameters, optflow: byotrack.OpticalFlow | None = None, features_extractor: byotrack.FeaturesExtractor | None = None, *, save_all=False)

Bases: FrameByFrameLinker

Frame by frame linker by associating with the closest detections.

Motion is not modeled, but if an optical flow method is provided, it will be used to compensate motion online. Matching is done from a simple Euclidean distance. This can be easily changed by inheriting this class and overriding the cost method.

See FrameByFrameLinker for the other attributes.

specs

Parameters specifications of the algorithm. See NearestNeighborParameters.

Type:: NearestNeighborParameters

active_positions

The positions of actives tracks, if undetected it is estimated by optical flow propagation, or simply propagated if no optical flow is given. Shape: (N, D), dtype: float32

Type:: torch.Tensor

reset(dim=2) → None

Reset the linking algorithm.

Flush all data stored from a previous linking and prepare a new linking.

Parameters:: dim (int) – The dimension of the data. Default: 2

motion_model() → None

Optional modelisation of motion for tracks.

It can be used to update some internal state of the tracker after the optical flow computation and before the distance computation.

cost(frame: np.ndarray | None, detections: byotrack.Detections) → tuple[torch.Tensor, float]

Compute the association cost between active tracks and detections.

It also returns the threshold to use (Depending on the dist you use, association_threshold could be related to a more meaning full quantity than the cost itself). For instance, when using a squared Euclidean distance, the association threshold could be express as the distance in pixel, and this function could square it. For likelihood association, you could provide the association threshold as a probability and use -log(threshold) as the true threshold. (See KalmanLinker and NearestNeighborLinker)

Parameters:

frame (np.ndarray | None) – The current optional frame of the video Shape: (H, W, C), dtype: float
detections (byotrack.Detections) – Detections for the given frame

Returns:

The cost matrix between active tracks and detections: Shape: (n_tracks, n_dets), dtype: float
float: The association threshold to use.: It can be different than self.association_threshold depending on the dist build here

Return type:

torch.Tensor

post_association(frame: np.ndarray | None, detections: byotrack.Detections, active_mask: torch.Tensor) → None

Update the internal state of the tracker after update_active_tracks.

It should update any internal model/data. It is also responsible to register the position of each active track in all_positions for the current time frame.

Parameters:

frame (np.ndarray | None) – The optional current frame of the video Shape: (H, W, C), dtype: float
detections (byotrack.Detections) – Detections for the given frame
active_mask (torch.Tensor) – Boolean tensor indicating True for still active tracks Shape: (N_tracks), dtype: bool