ci: pin MLflow for protobuf compatibility AB#5480878

[ranadeepsingh]

Related Issues/PRs

• AB#5480878: Fix SynapseML Python CI mlflow protobuf incompatibility
• Blocks full Azure validation of fix: correct LightGBM improvement tolerance semantics #2578 and ci: guard Databricks T4 GPU pool migration AB#5478524 #2579
• Reproduced on scheduled master build 228845851

What changes are proposed in this pull request?

Pin the shared Python test environment to mlflow==2.21.3.

Six Python test jobs currently fail during collection because the cached unpinned MLflow package imports
google.protobuf.service from an incompatible protobuf release. The same failure occurs on master, so it
is not caused by #2579.

MLflow 2.21.3:

• is already used by SynapseML's Databricks test cluster dependencies;
• constrains protobuf to a compatible major version;
• includes generated protobuf code that handles protobuf 5; and
• changes the environment.yml hash, forcing Azure Pipelines to build a fresh conda cache instead of
  restoring the incompatible artifact.

How is this patch tested?

• I have written tests or performed focused validation and confirmed the proposed change works.

• Loaded environment.yml and verified the exact MLflow pin.

• Imported mlflow.protos.service_pb2 from the MLflow 2.21.3 wheel with protobuf 5.29.6.

• Confirmed the conda cache input hash changes from
  53693f7fca9c7552136553c3373c989575ab9bca9408dc9240812455a2448031 to
  e3e7eeb36600cc82c712e67fa477036ffcb4bb803c8b6a162eb1314c14b121f1.

• black --check --extend-exclude 'docs/' .

• Full Azure Pipelines build 228877170 passed, including every Python test job and the Databricks GPU job.

Does this PR change any dependencies?

• No. You can skip this section.
• Yes. The shared CI environment now pins MLflow to 2.21.3 for protobuf compatibility and reproducibility.

Does this PR add a new feature? If so, have you added samples on website?

• No. You can skip this section.
• Yes. Make sure you have added samples following below steps.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[github-actions]

Hey @ranadeepsingh 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

• fix: Fix LightGBM crashes with empty partitions
• feat: Make HTTP on Spark back-offs configurable
• docs: Update Spark Serving usage
• build: Add codecov support
• perf: improve LightGBM memory usage
• refactor: make python code generation rely on classes
• style: Remove nulls from CNTKModel
• test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

[ranadeepsingh]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

Pins MLflow in the shared Python CI conda environment to address protobuf compatibility issues impacting Python test collection, improving reproducibility of CI runs.

Changes:

• Pin mlflow to 2.21.3 in environment.yml (pip dependencies) to avoid incompatible protobuf imports.

Show a summary per file

File	Description
environment.yml	Pins mlflow==2.21.3 in the shared CI Python environment to stabilize protobuf compatibility.

Review details

• Files reviewed: 1/1 changed files
• Comments generated: 0
• Review effort level: Low

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.78%. Comparing base (b350135) to head (14c45d1).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2580      +/-   ##
==========================================
- Coverage   84.79%   84.78%   -0.01%     
==========================================
  Files         334      334              
  Lines       17801    17801              
  Branches     1619     1619              
==========================================
- Hits        15094    15093       -1     
- Misses       2707     2708       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.