Passionate about precision medicine and advancing the healthcare industry?
Recent advancements in underlying technology have finally made it possible for AI to impact clinical care in a meaningful way. Tempus' proprietary platform connects an entire ecosystem of real-world evidence to deliver real-time, actionable insights to physicians, providing critical information about the right treatments for the right patients, at the right time.
Focus: Contribute to the design of the core architecture of LMMs and building of the infrastructure for training it at scale.
Responsibilities: You will contribute to the following activities:
-
Design and definition of the architecture of the LMMs, considering different fusion strategies and modality-specific processing.
-
Implement, refine, benchmark and optimize model architectures using deep learning frameworks such as PyTorch or TensorFlow.
-
Develop and manage the end-to-end training pipelines, including data loading, preprocessing, and model training. Architect and deploy distributed training workflows, optimizing for performance across cloud GPU fleets.
-
Implement distributed training strategies to handle large-scale datasets and models.
-
Design and implement methods to fuse knowledge with the multimodal representations within the LMM.
-
Experiment with different approaches to enhance the model's understanding and reasoning abilities through knowledge integration.
-
Monitor and debug training processes, identifying and resolving performance bottlenecks.
-
Collaborate with the knowledge integration engineer to ensure the architecture can accommodate knowledge injection mechanisms.
Skills needed:
-
Deep understanding of deep learning principles and architectures (especially transformers).
-
Extensive experience with multimodal machine learning concepts and techniques (for example, different fusion methods for text and images). Solid understanding of optimization techniques for large-scale models.
-
Strong proficiency in Python and deep learning frameworks (PyTorch/TensorFlow) and model management libraries like HF Transformers.
-
Experience with training large multimodal models with distributed training frameworks (for example, Horovod, MosaicML) and GPU fleet management.
-
Strong understanding of knowledge representation concepts (for example, knowledge graphs, ontologies).
-
Experience with distributed training frameworks and cloud computing platforms (for example, GCP, Azure).
#LI-SH1
