Welcome to ByoTrack’s documentation!
ByoTrack is a Python library that enables tracking of biological object in videos (2D or 3D). Many bioimage informatics tools already implement their own tracking tools (Icy, ImageJ, TrackMate…) but most of them are implemented in Java which makes it difficult for non-Java developers to experiment with the code. It is also difficult to integrate deep learning algorithms (mainly developed in Python) into these software.
We provide a unified python API for tracking that can be easily extended with new (and old) algorithms. We also provide implementations of well-known algorithms following our API. ByoTrack is based on numpy, pytorch and numba allowing fast computations with the access to the full python ecosystem.
Note
This project and documentation is under active development.
Cite Us
@article{hanson2024automatic,
title={Automatic monitoring of neural activity with single-cell resolution in behaving Hydra},
author={Hanson, Alison and Reme, Raphael and Telerman, Noah and Yamamoto, Wataru and Olivo-Marin, Jean-Christophe and Lagache, Thibault and Yuste, Rafael},
journal={Scientific Reports},
volume={14},
number={1},
pages={5083},
year={2024},
publisher={Nature Publishing Group UK London}
}
References
[1] F. De Chaumont, S. Dallongeville, N. Chenouard, et al., “Icy: an open bioimage informatics platform for extended reproducible research”, Nature methods, vol. 9, no. 7, pp. 690-696, 2012.
[2] J.-C. Olivo-Marin, “Extraction of spots in biological images using multiscale products”, Pattern Recognition, vol. 35, no. 9, pp. 1989-1996, 2002.
[3] U. Schmidt, M. Weigert, C. Broaddus, and G. Myers, “Cell detection with star-convex polygons”, in Medical Image Computing and Computer Assisted Intervention-MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part II 11. Springer, 2018, pp. 265-273.
[4] N. Chenouard, I. Bloch, and J.-C. Olivo-Marin, “Multiple hypothesis tracking for cluttered biological image sequences”, IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 11, pp. 2736-3750, 2013.
[5] T. Lagache, A. Hanson, J. Perez-Ortega, et al., “Tracking calcium dynamics from individual neurons in behaving animals”, PLoS computational biology, vol. 17, pp. e1009432, 10 2021.
[6] J. Schindelin, I. Arganda-Carreras, E. Frise, et al., “Fiji: an open-source platform for biological-image analysis”, Nature Methods, 9(7), 676-682, 2012.
[7] K. Jaqaman, D. Loerke, M. Mettlen, et al., “Robust single-particle tracking in live-cell time-lapse sequences.”, Nature Methods, 5(8), 695-702, 2008.
[8] J.-Y. Tinevez, N. Perry, J. Schindelin, et al., “TrackMate: An open and extensible platform for single-particle tracking.”, Methods, 115, 80-90, 2017.
[9] R. Reme, A. Newson, E. Angelini, J.-C. Olivo-Marin and T. Lagache, “Particle tracking in biological images with optical-flow enhanced kalman filtering”, in International Symposium on Biomedical Imaging (ISBI2024).
[10] M. Maška, V. Ulman, D. Svoboda, P. Matula, et al., “A benchmark for comparison of cell tracking algorithms”, in Bioinformatics, 2014.
[11] R. Reme, A. Newson, E. Angelini, J.-C. Olivo-Marin and T. Lagache, “SINETRA: a Versatile Framework for Evaluating Single Neuron Tracking in Behaving Animals”, arXiv preprint arXiv:2411.09462, 2024.
Contents
Getting started
API
- Video
- Video
- Video Reader
slice_length()MetaVideoReaderVideoReaderVideoReader.supported_extensionsVideoReader.pathVideoReader.releasedVideoReader.fpsVideoReader.shapeVideoReader.channelsVideoReader.lengthVideoReader.frame_idVideoReader.release()VideoReader.grab()VideoReader.retrieve()VideoReader.read()VideoReader.seek()VideoReader.tell()VideoReader.open()
OpenCVVideoReaderPILVideoReaderTiffVideoReaderFrameTiffLoaderpil_loader()MultiFrameReader
- Video transforms
- Detections
relabel_consecutive()DetectionsDetections.dataDetections.lengthDetections.dimDetections.frame_idDetections.shapeDetections.positionDetections.bboxDetections.segmentationDetections.confidenceDetections.massDetections.use_median_positionDetections.save()Detections.load()Detections.save_multi_frames_detections()Detections.load_multi_frames_detections()
- Tracks
- Tracker
- Detector
- Linker
- Refiner
- Optical Flow
- Parameters
API implementations
- Detectors
- Linkers
- Frame By Frame Linkers
AssociationMethodTrackHandlerTrackHandler.n_validTrackHandler.n_gapTrackHandler.startTrackHandler.identifierTrackHandler.track_stateTrackHandler.last_associationTrackHandler.detection_idsTrackHandler.track_idsTrackHandler.merge_idTrackHandler.parent_idTrackHandler.is_splitTrackHandler.TrackStateTrackHandler.update()TrackHandler.register_track_id()
OnlineFlowExtractorFrameByFrameLinkerParametersFrameByFrameLinkerFrameByFrameLinker.specsFrameByFrameLinker.optflowFrameByFrameLinker.features_extractorFrameByFrameLinker.save_allFrameByFrameLinker.frame_idFrameByFrameLinker.inactive_tracksFrameByFrameLinker.active_tracksFrameByFrameLinker.all_positionsFrameByFrameLinker.split_linksFrameByFrameLinker.merge_linksFrameByFrameLinker.reset()FrameByFrameLinker.collect()FrameByFrameLinker.motion_model()FrameByFrameLinker.cost()FrameByFrameLinker.post_association()FrameByFrameLinker.update_active_tracks()FrameByFrameLinker.update_detections()FrameByFrameLinker.associate()FrameByFrameLinker.update()
greedy_assignment_solver()
- Nearest Neighbor Linker
NearestNeighborParametersNearestNeighborParameters.association_thresholdNearestNeighborParameters.n_validNearestNeighborParameters.n_gapNearestNeighborParameters.association_methodNearestNeighborParameters.anisotropyNearestNeighborParameters.emaNearestNeighborParameters.fill_gapNearestNeighborParameters.split_factorNearestNeighborParameters.merge_factor
NearestNeighborLinker
- Kalman Linker
CostTrackBuildingKalmanLinkerParametersKalmanLinkerParameters.association_thresholdKalmanLinkerParameters.detection_stdKalmanLinkerParameters.process_stdKalmanLinkerParameters.kalman_orderKalmanLinkerParameters.n_validKalmanLinkerParameters.n_gapKalmanLinkerParameters.association_methodKalmanLinkerParameters.anisotropyKalmanLinkerParameters.cost_methodKalmanLinkerParameters.track_buildingKalmanLinkerParameters.split_factorKalmanLinkerParameters.merge_factorKalmanLinkerParameters.online_process_std
KalmanLinker
- Kalman and Optical Flow Tracking
KOFTLinkerParametersKOFTLinkerParameters.association_thresholdKOFTLinkerParameters.detection_stdKOFTLinkerParameters.flow_stdKOFTLinkerParameters.process_stdKOFTLinkerParameters.kalman_orderKOFTLinkerParameters.n_validKOFTLinkerParameters.n_gapKOFTLinkerParameters.association_methodKOFTLinkerParameters.anisotropyKOFTLinkerParameters.cost_methodKOFTLinkerParameters.track_buildingKOFTLinkerParameters.split_factorKOFTLinkerParameters.merge_factorKOFTLinkerParameters.extract_flows_on_detectionsKOFTLinkerParameters.always_measure_velocityKOFTLinkerParameters.online_process_std
KOFTLinker
- Icy EMHT
MotionEMHTParametersEMHTParameters.detections_fprEMHTParameters.detections_fnrEMHTParameters.expected_track_lengthEMHTParameters.expected_initial_particlesEMHTParameters.expected_new_particlesEMHTParameters.existence_probEMHTParameters.termination_probEMHTParameters.motionEMHTParameters.use_most_likely_modelEMHTParameters.inertiaEMHTParameters.gate_factorEMHTParameters.tree_depthEMHTParameters.to_xml()
IcyEMHTLinker
- TrackMate
TrackMateParametersTrackMateParameters.allow_gap_closingTrackMateParameters.allow_track_splittingTrackMateParameters.allow_track_mergingTrackMateParameters.alternative_linking_cost_factorTrackMateParameters.blocking_valueTrackMateParameters.cutoff_percentileTrackMateParameters.max_frame_gapTrackMateParameters.linking_max_distanceTrackMateParameters.gap_closing_max_distanceTrackMateParameters.merging_max_distanceTrackMateParameters.splitting_max_distanceTrackMateParameters.kalman_search_radiusTrackMateParameters.save()
TrackMateLinker
- Frame By Frame Linkers
- Refiners
- Optical Flows
Datasets
Icy support
Fiji support
Visualization
- ByoTrack ViZ
display_lifetime()temporal_projection()InteractiveVisualizerInteractiveFlowVisualizerInteractiveFlowVisualizer.videoInteractiveFlowVisualizer.optflowInteractiveFlowVisualizer.grid_stepInteractiveFlowVisualizer.pointsInteractiveFlowVisualizer.precomputeInteractiveFlowVisualizer.tracks_colorsInteractiveFlowVisualizer.scaleInteractiveFlowVisualizer.run()InteractiveFlowVisualizer.handle_actions()
flow_to_rgb()get_flow_wheel()