Job Description

• Design, build, and maintain scalable ML infrastructure and automated machine learning (ML) pipelines, with a strong focus on PySpark and Databricks.
• Develop and implement Point-in-Time (PIT) feature engineering pipelines and feedback loops for continuous model improvement.
• Manage end-to-end model monitoring systems to detect data drift, model drift, and data quality issues.
• Oversee the full ML model lifecycle using MLflow (including Tracking, Registry, and Deployment).
• Build and maintain large-scale data processing pipelines for feature engineering and ensure data quality.
• Manage Feature Store operations, including feature versioning and governance.
• Run large-scale hyperparameter tuning using tools like Hyperopt and Ray Tune.
• Collaborate closely with Data Scientists to productionize and deploy new machine learning models.
• Ensure ML systems follow best practices for CI/CD automation (preferred).
• Utilize advanced Databricks features (Jobs, Workflows, Feature Store, Unity Catalog) for robust ML system management.
• (Preferred) Work with Spark Structured Streaming and model calibration techniques such as Isotonic Regression.

5+ years in MLOps, ML Engineering, or Data Engineering with a strong ML focus.
Expertise in PySpark, Databricks, MLflow, Feature Store, hyperparameter tuning, and model monitoring.

Job Tags

Similar Jobs