Senior Data Engineer / AI Engineer, Network Performance Automation Section Full-time

Job Description:

About Organization

E2E Performance & Capacity Division is tasked with monitoring End-to-End network performance and/or capacity KPIs, filing business requirements, and aligning with Network Functions/Applications owners.

In the role of Data Engineer within the Network Performance Automation section, the individual will technically lead the team in developing automated data pipelines, cleaning, and processing data, and providing insights for automation projects. The work environment prioritizes innovation and efficiency, leveraging AI and data engineering to streamline and optimize processes.

The role also involves guiding the team and overseeing the successful completion of automation and AI projects. Responsibilities include coordinating tasks, managing project timelines, and ensuring high-quality deliverables. Leadership and expertise are critical in driving automation initiatives and contributing to AI-based solutions.

Job duties

• Data Engineering: Design, build, and maintain scalable data pipelines for automation projects using Python. Create robust systems that extract, clean, and process data from multiple sources to support dashboards and reports.
• Software Development: Utilize software lifecycle mainly in developing data pipelines. Develop automation scripts to streamline data analysis and data processing tasks. Ensure that the codebase is clean, maintainable, and scalable, meeting project deadlines and quality standards.
• AI Engineering: Contribute to developing AI and machine learning solutions & techniques to enhance automation projects. Work with AI-based models and use cases to optimize data processing, forecasting, and correlative analysis.
• Data Storage and Management: Implement and manage data storage and retrieval systems to support batch analytics and reporting. Ensure data governance, quality, accuracy, and security while maintaining compliance with company standards.
• Team Leadership: Lead the team and manage ongoing data engineering and AI projects. Coordinate project tasks, set priorities, and ensure the successful completion of project milestones. Mentor and guide team members to foster a collaborative and high-performing environment.
• Collaboration and Communication: Collaborate with cross-functional teams to understand business requirements and translate them into data engineering and AI-based automation solutions. Communicate technical concepts and results to stakeholders clearly and effectively.
• Management Reporting: Prepare and deliver reports to management on ongoing projects and activities. Provide updates on project timelines, achievements, and any issues that require escalation. Ensure reports are clear, concise, and tailored to the audience’s needs.
• Continuous Improvement: Proactively identify opportunities to improve automation and AI processes. Stay updated with industry trends in data engineering and AI to ensure the team’s solutions remain innovative and effective.

Minimum Qualification

• Education: Bachelor’s Degree (BS) in Computer Science, Data Engineering, or a related field.
• Experience: 8+ years of experience in software design/development and data engineering, with a strong focus on Python programming. Familiarity with building dashboards using different frameworks like Streamlit.
• AI and Machine Learning: Proven experience with AI and machine learning technologies, building pipelines for AI models.
• Data Visualization: Proficiency in data analysis and data visualization tools such as Pandas, NumPy, and Matplotlib, with strong Python skills.
• Database Knowledge: Understanding of OLAP, OLTP and document databases and SQL, with the ability to manage data storage and retrieval efficiently.
• Communication Skills: Strong verbal and written communication skills, with the ability to present complex information to stakeholders in an understandable way.
• Collaboration: Ability to work collaboratively in a cross-functional team environment.
• Problem-Solving Skills: Excellent problem-solving and analytical skills to resolve complex issues effectively.
• Project Management and Team Leadership: Experience in project management for leading automation and AI projects, with knowledge of agile methodologies like Scrum. Proven ability to guide and direct a team toward project milestones.
• Data Processing Frameworks: Proficiency with Apache Spark or Trino for large-scale data processing and workflow orchestration with Apache Airflow. Experience with Pandas for data manipulation and exploratory data analysis.
• ETL Tools: Experience with orchestration tools like Airflow or Apache NiFi for data integration and ETL processes. Ability to design and maintain complex data workflows.
• Project Management and Automation Tools: Familiarity with project management tools like Jira and automation platforms like Power Automate. Knowledge of Jupyter for data manipulation using Python libraries.

Preferred Qualifications

• Experience with AI-based Solutions: In-depth knowledge of AI applications and solutions, with hands-on experience in implementing AI and machine learning models. A background in developing AI-based projects and leveraging AI techniques for automation and data engineering. – including open-source LLM (Large Language Models) and AI forecasting models.-
• Experience with BigQuery: Familiarity with Google BigQuery for large-scale data processing and analytics.
• Familiarity with Cloud Platforms: Experience with cloud-based data platforms like AWS, GCP, or Azure, including deploying and managing data engineering resources in the cloud.
• Big Data Expertise: Knowledge of big data concepts, tools, and technologies, including data storage, processing, and analysis at scale.
• Web Development Skills: Understanding of web design principles, with some experience in JS/HTML/CSS for building user interfaces or dashboards.
• Domo Experience: Experience with Domo for data visualization and business intelligence, including creating dashboards and reports.

Languages:
English (Overall – 3 – Advanced)