Production Systems Engineer, AI Systems Full-time

Meta is seeking a Systems Engineer to join our Release to Production (RTP) team working on AI/ML initiatives supporting large scale AI Training and Inference. Our servers and data centers are the foundation upon which our rapidly scaling infrastructure operates efficiently to deliver our innovative services. The RTP team is responsible for the end-to-end Hardware Lifecycle of all Meta servers including prototyping of experimental HW, pre-production hands-on system and hardware debugging and stress testing, enabling production-ready system monitoring, automated provisioning and automated remediation of issues. RTP team also helps in exploring, developing and productizing high-performance software and hardware technologies for AI at datacenter scale. RTP Engineers have a large swath of XFN partners they work closely with e.g. HW/SW co-design teams, hardware designers, networking teams, system manufacturers, component vendors, capacity engineering, production engineering, production services, and data center operations teams to enable new systems that will be deployed in our production data centers. We are looking for a candidate who can support scale up and scale out network technologies (e.g. NICs) for Meta AI systems that are powering Meta’s tremendous leaps in the AI space. The ideal candidate is knowledgeable about network technologies and has hands-on experience supporting them through at least a couple of hardware/software (firmware, driver) lifecycle phases: design+bring up, server integration, system validation, supporting customer deployment, production issue triage, rolling out new features in FW/Driver.

Production Systems Engineer, AI Systems Responsibilities

• Support new AI platform introduction in to Meta fleet by driving scale up (e.g. NVlink, XGMI) and scale out (e.g. NICs) interface. integration
• Contribute to new feature/technology development/validation across hardware/software stack.
• Contribute to enabling hacks for future technology explorations in AI space such as memory, network and storage interdependencies in the context of AI workloads.
• Proactively create experiments and tooling to detect and diagnose hardware/firmware/software health issues.
• Troubleshoot, diagnose and root cause of system failures and isolate the components/failure scenarios while working with internal & external partners.
• Develop visibility through data visualization and implement systemic solutions to hardware health issues.
• Leverage production experience to drive external and internal teams to continuously improve product quality.

Minimum Qualifications

• Currently has, or is in the process of obtaining a Bachelor’s degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience. Degree must be completed prior to joining Meta.
• 3+ years of work experience in one or more domains such as: Network ASIC development (Silicon design or bringup or characterization), board development, network product deployment and customer support (Switches, NICs), Interconnect Technologies (e.g. Optics, DAC).
• Knowledge of server architecture and components.
• Experience working with Linux.
• Knowledge of TCP/IP and experience using iperf.
• Hands on troubleshooting and debug experience.

Preferred Qualifications

• Experience working with full server system, including PCIe.
• Experience working with RDMA.
• Experience working with large scale deployments.
• 2+ years experience scripting automation in Python or PHP or Perl.

Start preparing
Learn about how to prepare for your interview with our interview guide, tips, and interactive experiences.
Visit interview prep