What You’ll Work On

You’ll review ML challenges involving:

• Experiment design and model selection
• Small and synthetic datasets
• Data quality and preprocessing
• Distribution shift and data contamination
• Label noise and feature leakage
• Model evaluation and metric selection
• Hyperparameter tuning
• Train / validation / test methodology
• Reproducibility and deterministic pipelines
• Statistical significance of model improvements

What We’re Looking For

• 3+ years of hands-on applied machine learning experience
• Strong experience with:
• ML experiment design
• Model selection
• Hyperparameter tuning
• Model evaluation
• Data preprocessing and validation
• Strong understanding of train, validation, and test splits
• Ability to identify:
• Data leakage
• Label noise
• Distribution shift
• Spurious correlations
• Feature leakage
• Data contamination
• Experience evaluating whether performance improvements are statistically meaningful
• Strong understanding of ML evaluation metrics
• Experience debugging ML workloads across CPU and GPU environments
• Ability to analyze technical problems and provide clear written feedback

Nice to Have

• Experience with Kaggle or ML competitions
• Designing benchmark datasets or ML challenges
• Background in data-centric AI
• Experience with synthetic data generation
• Familiarity with statistical testing
• Experience with ML evaluation pipelines
• Developing ML curricula or technical assessments
• Understanding of common ML failure modes

What You’ll Be Responsible For

• Reviewing ML challenges for design and solvability
• Evaluating datasets for meaningful signals
• Identifying shortcuts in synthetic datasets
• Determining ML problem diagnostics
• Reviewing evaluation metrics
• Detecting evaluation flaws
• Verifying reproducibility
• Assessing challenge difficulty
• Providing recommendations for improvements

Engagement

Work Type: Remote
Engagement: Part-time, project-based consulting
Focus: Applied machine learning, experiment design, data quality, and model evaluation

This role is ideal for ML engineers passionate about debugging experiments, understanding model dynamics, and designing rigorous ML experiments.