refactor: Vendoring Protobuf #1869
Conversation
|
|
||
|
|
||
| def _build_rewriter(top_level_names: Iterable[str]): | ||
| """Build a per-file rewriter for the given top-level package names. |
There was a problem hiding this comment.
Is this used when rewriting protobuf imports inside the worker? Or is it rewriting the protobuf imports within the protobuf module itself?
| init.write_text( | ||
| "# Copyright (c) Microsoft Corporation. All rights reserved.\n" | ||
| "# Licensed under the MIT License.\n" | ||
| "# Vendored namespace package. See ../README.md.\n", |
There was a problem hiding this comment.
Can you update the README / some other documentation with this change?
| # pure-Python implementation here so that any vendored protobuf module | ||
| # imported transitively never tries to load a C extension. This must run | ||
| # before any submodule import that touches protobuf. | ||
| os.environ.setdefault("PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION", "python") |
There was a problem hiding this comment.
Are we setting this variable by default for production environments? We only need this setting when performing the build, right? We can add this as a pipeline variable instead
| # assigned onto a vendored field (e.g. ``RpcRetryOptions.delay_interval``) | ||
| # must therefore come from the same vendored protobuf — a ``Duration`` | ||
| # imported from the customer-facing ``google.protobuf`` namespace lives in | ||
| # a different descriptor pool and the assignment would fail at runtime. |
There was a problem hiding this comment.
nit: lengthy comment. We can just say that the worker no longer imports from google.protobuf directly now & references _vendored.google.protobuf now
| # to the worker's vendored copy, not whatever the customer ships in | ||
| # their `.python_packages`. The proxy worker (Python 3.13+) runs out | ||
| # of process and is unaffected, so we only rewrite for the in-process | ||
| # worker. |
There was a problem hiding this comment.
nit: proxy worker is unaffected for a different reason & azure functions worker is not "in-proc"
| # their `.python_packages`. The proxy worker (Python 3.13+) runs out | ||
| # of process and is unaffected, so we only rewrite for the in-process | ||
| # worker. | ||
| rewrite_protobuf = WORKER_DIR == "azure_functions_worker" |
There was a problem hiding this comment.
Should this be a global variable, similar to WORKER_DIR?
| regardless of any version the customer ships in ``.python_packages``. | ||
| Safe to re-run; the script is idempotent. | ||
|
|
||
| Skipped for the proxy worker (Python >= 3.13) which is out-of-process |
There was a problem hiding this comment.
nit: out-of-process is not the reason 3.13 is unaffected
| if WORKER_DIR != "azure_functions_worker": | ||
| print( | ||
| f"Skipping vendor_deps for {WORKER_DIR} " | ||
| "(only required for in-process worker)." |
2 participants
Description
Fixes #
Pull Request Checklist
Host-Worker Contract
Worker Execution Logic
If yes, please answer the following:
Python Version Coverage
Programming Model Compatibility (for Python 3.13+)