Architect- AI Observability Lead Full-time

Overview

We are seeking an experienced AI Observability Lead to join our team and spearhead our efforts in monitoring, analyzing, and improving the performance and reliability of our AI systems, with a focus on Machine Learning, Computer Vision, and Generative AI. This role is critical in ensuring the transparency, explainability, and trustworthiness of our AI models and applications, as well as mentoring and developing team members.

Responsibilities

1. Develop and implement comprehensive AI observability strategies and frameworks across the organization, specifically tailored for ML, Computer Vision, and Generative AI systems.
2. Design and maintain monitoring systems to track AI model performance, data quality, and system health in real-time for various AI domains.
3. Create dashboards and visualization tools to provide clear insights into AI system behavior and performance metrics, with specific attention to the unique challenges of Computer Vision and Generative AI.
4. Collaborate with data scientists, ML engineers, and software developers to integrate observability practices into the AI development lifecycle across different AI domains.
5. Implement logging and tracing mechanisms to capture relevant data for debugging and auditing AI systems, including large language models and image generation systems.
6. Develop and maintain alert systems to quickly identify and respond to anomalies or degradations in AI model performance across ML, Computer Vision, and Generative AI applications.
7. Lead root cause analysis efforts for AI system failures or unexpected behaviors in various AI domains.
8. Establish best practices for AI explainability and interpretability, ensuring compliance with relevant regulations and ethical guidelines, particularly for Generative AI systems.
9. Stay up-to-date with the latest advancements in AI observability tools and techniques, particularly those relevant to ML, Computer Vision, and Generative AI, and evaluate their potential application within the organization.
10. Provide training, mentorship, and guidance to team members on AI observability best practices and tools, fostering their professional growth and expertise in various AI domains.
11. Collaborate with stakeholders to define and track relevant KPIs for AI system performance and reliability across different AI applications.
12. Contribute to the development of AI governance policies and procedures related to observability and transparency, with consideration for the specific challenges of Computer Vision and Generative AI.
13. Mentor and develop junior team members, fostering their skills in AI observability and domain-specific knowledge in ML, Computer Vision, and Generative AI.

Qualifications

• Master’s degree in Computer Science, Data Science, or a related field; Ph.D. preferred.
• 7+ years of experience in machine learning, data science, or software engineering roles, with significant exposure to Computer Vision and Generative AI projects.
• 4+ years of experience specifically focused on AI/ML observability, monitoring, or related fields.
• Strong programming skills in languages such as Python, Java, or Go.
• Extensive experience with AI/ML frameworks (e.g., TensorFlow, PyTorch, scikit-learn) and cloud platforms (e.g., AWS, Azure, GCP).
• Proficiency in data visualization tools and techniques (e.g., Grafana, Tableau, D3.js).
• Deep understanding of AI/ML concepts, including model training, evaluation, and deployment, with specific knowledge of Computer Vision and Generative AI techniques.
• Experience with monitoring and observability tools such as Prometheus, ELK stack, or similar systems.
• Excellent problem-solving skills and ability to troubleshoot complex AI systems across various domains.
• Proven track record of mentoring and developing junior team members in AI-related roles.