ByoTrack ViZ
We add some visualization tools. They rely on matplotlib and opencv-python.
Some of these tools support 3D data, but we still advise to export your results to other dedicated software for 3D visualization.
Note
We developed these tools under linux using opencv + gtk backend. It seems that the zoom/drag/quit operations are not automatically supported on other platforms.
- byotrack.visualize.display_lifetime(tracks: Collection[Track])
Display the lifetime of tracks
Active tracks are in white. (Tracks on x-axis, Frames on y-axis)
- Parameters:
tracks (Collection[byotrack.Track]) – Tracks
- byotrack.visualize.temporal_projection(tracks_tensor: Tensor, colors: Sequence[Tuple[int, int, int]] = ((255, 255, 255),), background: ndarray | None = None, color_by_time=False) ndarray
Project all given tracks into a single image
A track is displayed as the line of its consecutive positions. Track’s undefined positions are not displayed.
3d tracks and images are not supported
- Parameters:
tracks_tensor (torch.Tensor) – Tracks tensor (See byotrack.Track.tensorize) Positions of each track at each frame. (NaN if not defined). Shape: (T, N, 2)
colors (Sequence[Tuple[int, int, int]]) – Color sequence. When we color each track independently, each track is associated with a color (ith track with ith color, it loops if more tracks than colors). For time coloring, each frame has its own color (same across all tracks). Default: ((255, 255, 255),) (Everything is white)
background (Optional[np.ndarray]) – Optional frame to display behind the tracks. If not given, we set the background as 0, with the smallest size to contain all tracks. Default: None Shape: (H, W[, C])
color_by_time (bool) – If set to True, each frame has its own color. Otherwise, each track has its own color. Color by time is much slower (cv2 do not allow complex multilines draw) Default: False
- Returns:
- Projection image
Shape: (H, W, 3), dtype: np.uint8
- Return type:
np.ndarray
- class byotrack.visualize.InteractiveVisualizer(video: Sequence[ndarray] | ndarray = (), detections_sequence: Sequence[Detections] = (), tracks: Collection[Track] = ())
Bases:
objectInteractive visualization with opencv
Supports 3D stacks (but you will only see one plane at a time in 2D).
- Keys:
space: Pause/Unpause the video
w/x: Move backward/forward in the video (when paused)
b/n: Move backward/forward in depth (Z axis)
d: Switch detections display mode (Not displayed, Mask, Segmentation) if available
t: Switch on/off the display of tracks if available
v: Switch on/off the display of the video
- video
Optional video to display. Shape: (T, [D, ], H, W, C) Default: () (no frames)
- Type:
Sequence[np.ndarray] | np.ndarray
- detections_sequence
Optional detections to display Default: () (no detections)
- Type:
Sequence[byotrack.Detections]
- tracks
Optional tracks to display Default: () (no tracks)
- Type:
Collection[byotrack.Tracks]
- tracks_colors
Colors to use for tracks Colors are picked circularly given their position in the tracks list. By default we use the hsv colormaps from matplotlib
- draw_segmentation(frame: ndarray) ndarray
Draw the segmentation on the frame
It will draw the segmentation for frame_id and stack_id.
- Parameters:
frame (np.ndarray) – The 2D frame to draw on (will not be modified) Shape: (H, W), dtype: np.uint8
- Returns:
The resulting frame with the segmentation drawned on
- Return type:
np.ndarray
- draw_tracks(frame: ndarray) ndarray
Draw the tracks on the frame
It will draw the tracks for frame_id and stack_id.
- Parameters:
frame (np.ndarray) – The 2D frame to draw on (will not be modified) Shape: (H, W), dtype: np.uint8
- Returns:
The resulting frame with the drawn tracks
- Return type:
np.ndarray
- class byotrack.visualize.InteractiveFlowVisualizer(video: Sequence[ndarray] | ndarray, optflow: OpticalFlow, grid_step=20, precompute=False)
Bases:
objectInteractive optical flow visualization using a moving grid above the video
The video is displayed with a uniform grid (control points) above that moves according to the computed flow map. If the optical flow algorithm is good, the points should moves along the video.
It only supports 2D videos.
- Keys:
space: Pause/Unpause the video
w/x: Move backward/forward in the video (when paused)
g: Reset the control points as a grid
Note
To prevent lagging, even when precompute is False, optical flow map are saved and never computed twice. You should not use this function with too large videos. (Use slicing to reduce the size of a video)
- video
The video of interest (2D) Shape: (T, H, W, C)
- Type:
Sequence[np.ndarray] | np.ndarray
- optflow
Optical flow algorithm to visualize
- Type:
byotrack.OpticalFlow
- points
Control points (grid) Shape: (N, 2), dtype: float64
- Type:
np.ndarray
- precompute
Compute all the flows first (bi directionnal) to prevent lagging Default: False
- Type:
- tracks_colors
Colors to use for tracks Colors are picked circularly given the track identifier. By default we use the hsv colormaps from matplotlib
- byotrack.visualize.flow_to_rgb(flow: ndarray, clip: float | None = None) ndarray
Converts a 2D optical flow into a RBG map
The angle is converted into a color, the magnitude into luminosity. (Follow what is done by OpenCV for visualization)
See get_flow_wheel to get the color wheel for each direction.
- Parameters:
flow (np.ndarray) – Flow to convert to rgb Shape: (2, H, W)
clip (Optional[float]) – Clip the magnitude to this maximum value Default: None
- Returns:
- RGB flow visualization
Shape: (H, W, 3)
- Return type:
np.ndarray
- byotrack.visualize.get_flow_wheel(size=128, mask=True) ndarray
Build the color wheel that is used for flow_to_rgb