Architect- Gen AI Full-time

Overview

Develop cutting-edge architectural strategies for Gen AI components and platforms, leveraging advanced techniques such as chunking, Retrieval-Augmented Generation (RAG), Ai agents, and embeddings. Balance build versus buy decisions, ensuring alignment with SaaS models and decision trees, particularly for the PepGenX platform. Emphasize low coupling and cohesive model development.

Responsibilities

1. Develop cutting-edge architectural strategies for Gen AI components and platforms, leveraging advanced techniques such as chunking, Retrieval-Augmented Generation (RAG), Ai agents, and embeddings. Balance build versus buy decisions, ensuring alignment with SaaS models and decision trees, particularly for the PepGenX platform. Emphasize low coupling and cohesive model development.
2. Lead working sessions for Arch Alignment, pattern library development, GEN AI Tools Data Architect alignment, tag new components to reuse (components reuse strategy), patterns of the usecases, reuse components ( save efforts, time money).
3. Lead the implementation of LLM operations, focusing on optimizing model performance, scalability, and efficiency. Design and implement LLM agentic processes to create autonomous AI systems capable of complex decision-making and task execution.
4. Work closely with data scientists and AI professionals to identify and pilot innovative use cases that drive digital transformation. Assess the feasibility of these use cases, aligning them with business objectives, ROI and leveraging advanced AI techniques.
5. Gather inputs from various stakeholders to align technical implementations with current and future requirements. Develop processes and products based on these inputs, incorporating state-of-the-art AI methodologies.
6. Define AI architecture and select suitable technologies, with a focus on integrating RAG systems, embedding models, and advanced LLM frameworks. Decide on optimal deployment models, ensuring seamless integration with existing data management and analytics tools.
7. Audit AI tools and practices, focusing on continuous improvement of LLM ops and agentic processes. Collaborate with security and risk leaders to mitigate risks such as data poisoning and model theft, ensuring ethical AI implementation.
8. Stay updated on AI regulations and map them to best practices in AI architecture and pipeline planning. Develop expertise in ML and deep learning workflow architectures, with a focus on chunking strategies, embedding pipelines, and RAG system implementation.
9. Apply advanced software engineering and DevOps principles, utilizing tools like Git, Kubernetes, and CI/CD for efficient LLM ops. Collaborate across teams to ensure AI platforms meet both business and technical requirements.
10. Spearhead the exploration and application of cutting-edge Large Language Models (LLMs) and Generative AI, including multi-modal capabilities and agentic processes. Oversee MLOps, automating ML pipelines from training to deployment with a focus on RAG and embedding optimization.
11. Engage in sophisticated model development from ideation to deployment, leveraging advanced chunking and RAG techniques. Effectively communicate complex analysis results to business partners and executives.
12. Proactively reduce biases in model predictions, focusing on fair and inclusive AI systems through advanced debiasing techniques in embeddings and LLM training.
13. Design efficient data pipelines to support large language model training and inference, with a focus on optimizing chunking strategies and embedding generation for RAG systems.

Qualifications

1. Proven track record in shipping products and developing state-of-the-art Gen AI product architecture.
2. 10-15 years of experience with a strong balance of business acumen and technical expertise in AI.
3. 5+ years in building and releasing NLP/AI software, with specific experience in RAG systems and embedding models.
4. Demonstrated experience in delivering Gen AI products, including Multi-modal LLMs, Foundation models, and agentic AI systems.
5. Deep familiarity with cloud technologies, especially Azure, and experience deploying models for large-scale inference using advanced LLM ops techniques.
6. Proficiency in PyTorch, TensorFlow, Kubernetes, Docker, LlamaIndex, LangChain, LLM, SLM, LAM, and cloud platforms, with a focus on implementing RAG and embedding pipelines.
7. Excellent communication and interpersonal skills, with a strong design capability and ability to articulate complex AI concepts to diverse audiences.
8. Hands-on experience with chunking strategies, RAG implementation, and optimizing embedding models for various AI applications.