Ops Tech AI & Automation Principal Engineering Manager Full-time

About Citi:
Citi, the leading global bank, has approximately 200 million customer accounts and does business in more than 160 countries and jurisdictions. Citi provides consumers, corporations, governments, and institutions with a broad range of financial products and services, including consumer banking and credit, corporate and investment banking, securities brokerage, transaction services, and wealth management.

As a bank with a brain and a soul, Citi creates economic value that is systemically responsible and in our clients’ best interests. As a financial institution that touches every region of the world and every sector that shapes your daily life, our Enterprise Operations & Technology teams are charged with a mission that rivals any large tech company. Our technology solutions are the foundations of everything we do from keeping the bank safe, managing global resources, and providing the technical tools our workers need to be successful to designing our digital architecture and ensuring our platforms provide a first-class customer experience. We reimagine client and partner experiences to deliver excellence through secure, reliable, and efficient services.

Our commitment to diversity includes a workforce that represents the clients we serve from all walks of life, backgrounds, and origins. We foster an environment where the best people want to work. We value and demand respect for others, promote individuals based on merit, and ensure opportunities for personal development are widely available to all. Ideal candidates are innovators with well-rounded backgrounds who bring their authentic selves to work and complement our culture of delivering results with pride. If you are a problem solver who seeks passion in your work, come join us. We’ll enable growth and progress together.

About Our Team:

As a Ops Tech AI & Automation Principal Engineering Manager, you will be a part of the Ops Tech AI & Automation team. Your primary focus will be to ensure quality standards are met within existing and planned frameworks. This will involve rigorous testing, debugging, and refining of the software to ensure its efficiency and reliability. You will also be responsible for researching and implementing new technologies and methodologies to improve the functionality and user experience of our AI systems. You will also be required to focus on the development, research, and reengineering of Generative AI technologies. This involves leading a variety of engineering activities, including the design, development, and deployment of software in coordination with the Technology team.

The team you will be working with will consist of both employee and contractor resources based in the US and offshore. This diverse team will bring together a wealth of knowledge and experience in AI development, and you will have the opportunity to collaborate and learn from these talented individuals. You will also be expected to stay up-to-date with the latest advancements in AI and machine learning, and apply this knowledge to the continuous improvement and reengineering of our systems. This is a dynamic and fast-paced role that will require a high level of technical expertise and a passion for AI technology.

Responsibilities:

• Manage at least one product line from design to implementation and ongoing performance according to business need
• Manage all technology standards across functional area, ensuring development, understanding and adherence around the globe
• Set goals within functional area that are delivered to appropriate Operations & Technology organizations
• Contribute to the IT Project Leadership function to harmonize with overall business / technology objectives and ensure teams within the functional area are adequately staffed and possess the required skills sets to carry out their roles
• Provide an environment where Subject Matter Experts can work efficiently and delivery effectively
• Manage execution of IT strategy and roadmap for assigned technology area
• Apply an in-depth understanding of concepts and procedures within engineering and basic knowledge of those elements in other areas
• Responsible for training and coaching team members, performance management activities, monitoring quantity and quality of work, and addressing development needs for assigned personnel
• Assist with budget oversight, policy formulation and planning and ensure that development requests are in line with business requirements
• Make evaluative judgments based on information analysis in complicated or unique situations; utilizing multiple sources of information
• Negotiate with senior leaders (across functions) and communicate and negotiate with external parties
• Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm’s reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency, as well as effectively supervise the activity of others and create accountability with those who fail to maintain these standards.
• Lead the development of Generative AI models, ensuring they are designed and implemented in line with business objectives and quality standards. This includes working closely with the modelling team to ensure the accuracy and efficiency of these models.
• Validate all AI models before deployment, conducting rigorous testing and debugging to ensure their reliability and performance. This includes ensuring that all models meet the necessary quality standards and comply with all relevant regulations.
• Establish and enforce guardrails for AI development, ensuring that all activities are conducted within defined parameters. This includes setting boundaries for model development and deployment to ensure the integrity and security of our systems.
• Oversee the integration of AI models into existing systems, ensuring seamless functionality and user experience. This includes working closely with the technology team to ensure successful integration.
• Collaborate with the business team to ensure that Ground Truth is captured for all Generative AI models. This involves ensuring that all models are based on accurate and reliable data, and that they accurately reflect the realities of the business environment.
• Foster a collaborative environment, working closely with the modelling and business teams to ensure the successful development and deployment of Generative AI models. This includes facilitating communication and collaboration between these teams, and ensuring that all team members are aligned with the overall AI development strategy.

Qualifications:

• 6+ years of relevant experience in an Engineering role
• Experience working in Financial Services or a large complex and/or global environment
• Project Management experience
• Proven hands-on experience with Large Language Models (LLMs) in previous projects, particularly in product development using LLMs, demonstrating a strong understanding of their application and functionality.
• Experience in Chat, IVR, and Banking sectors is highly desirable, showcasing the ability to apply AI technologies in diverse contexts.
• Proficiency in Speech to Text technologies such as Whisper and Google Text-to-Speech, demonstrating a strong understanding of natural language processing.
• Hands-on experience with Vector Database & Graph DB technologies such as Pinecone, Milvus, and Neo4j, indicating a strong understanding of database management and data modeling.
• Solid understanding and practical experience using Transformer and Neural Network models, demonstrating a strong foundation in machine learning and deep learning techniques.
• Hands-on experience using Open AI APIs and Open Source LLMs such as Llama2, Mistral, Mixtral, etc., showcasing the ability to leverage open-source tools and resources in AI development.
• Solid understanding of frameworks like Langchain and Haystack, and the ability to use Agentic Workflows within these frameworks, indicating a strong ability to work within established AI development frameworks.
• Hands-on experience in creating Embeddings using MPNET and Ada models, demonstrating a strong understanding of word embedding techniques in natural language processing.
• Ability to use advanced techniques like Hybrid Dimensionality Reduction (HyDE), Maximal Marginal Relevance (MMR), and LLM reranking for effective semantic search, showcasing a strong understanding of advanced AI techniques and their application in improving search functionality.
• Comprehensive knowledge of design metrics, analytics tools, benchmarking activities and related reporting to identify best practices
• Demonstrated analytic/diagnostic skills
• Ability to work in a matrix environment and partner with virtual teams
• Ability to work independently, multi-task, and take ownership of various parts of a project or initiative
• Ability to work under pressure and manage to tight deadlines or unexpected changes in expectations or requirements
• Proven track record of operational process change and improvement
• Consistently demonstrates clear and concise written and verbal communication

Education:

• Bachelor’s degree/University degree or equivalent experience
• Master’s degree preferred

Job Family Group:
Technology

Job Family:
Systems & Engineering

Time Type:
Full time

Primary Location:
Irving Texas United States

Primary Location Full Time Salary Range:
$125,760.00 – $188,640.00

In addition to salary, Citi’s offerings may also include, for eligible employees, discretionary and formulaic incentive and retention awards. Citi offers competitive employee benefits, including: medical, dental & vision coverage; 401(k); life, accident, and disability insurance; and wellness programs. Citi also offers paid time off packages, including planned time off (vacation), unplanned time off (sick leave), and paid holidays. For additional information regarding Citi employee benefits, please visit citibenefits.com. Available offerings may vary by jurisdiction, job level, and date of hire.

Anticipated Posting Close Date:
Sep 30, 2024

Citi is an equal opportunity and affirmative action employer.

Qualified applicants will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.

Citigroup Inc. and its subsidiaries (“Citi”) invite all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View the “EEO is the Law” poster. View the EEO is the Law Supplement.

View the EEO Policy Statement.

View the Pay Transparency Posting