Skip to content

Cloudflare reg origin prod deploy #408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jeffreyaven merged 7 commits into from
Jun 26, 2026
Merged

Cloudflare reg origin prod deploy #408

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
67be046
Add Cloudflare Worker origin (green) + dual-path CI
jeffreyaven Jun 26, 2026
9106cf8
Merge pull request #406 from stackql/feature/provider-updates
jeffreyaven Jun 26, 2026
b273ca3
Fix R2 sync region: force AWS_REGION=auto for Cloudflare endpoint
jeffreyaven Jun 26, 2026
402fb27
CI: gate heavy analysis workflows on PR only, not on branch push
jeffreyaven Jun 26, 2026
696fed4
ci updates
jeffreyaven Jun 26, 2026
ad96630
CI: bump GO_VERSION to stable for core builds (needs go >= 1.25.3)
jeffreyaven Jun 26, 2026
912c10f
Merge pull request #407 from stackql/fix/r2-sync-region
jeffreyaven Jun 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 9 additions & 10 deletions .github/workflows/aot.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,14 @@ on:
- main
- dev
push:
branches:
- main
- dev
# branch pushes (merges) are not re-tested here; the pull_request run above is
# the merge gate. Tag pushes still trigger explicit on-demand analysis.
tags:
- aot*

env:
IS_TAG: ${{ github.ref_type == 'tag' }}
GO_VERSION: '~1.22'
GO_VERSION: 'stable' # stackql-core (built from main) tracks recent Go; stable always satisfies its go.mod
STACKQL_CORE_REPOSITORY: ${{ vars.STACKQL_CORE_REPOSITORY != '' && vars.STACKQL_CORE_REPOSITORY || 'stackql/stackql' }}
STACKQL_CORE_REF: ${{ vars.STACKQL_CORE_REF != '' && vars.STACKQL_CORE_REF || 'main' }}
STACKQL_ANY_SDK_REPOSITORY: ${{ vars.STACKQL_ANY_SDK_REPOSITORY != '' && vars.STACKQL_ANY_SDK_REPOSITORY || 'stackql/any-sdk' }}
Expand Down Expand Up @@ -42,37 +41,37 @@ jobs:
REG_DENO_DEPLOY_API_PROD: stackql-registry

steps:
- uses: actions/checkout@v4.1.1
- uses: actions/checkout@v7
name: "[SETUP] checkout repo"
with:
fetch-depth: 0

- name: Set up Go 1.x
uses: actions/setup-go@v5.0.0
uses: actions/setup-go@v6
with:
go-version: ${{ env.GO_VERSION }}
check-latest: true
cache: true
id: go

- name: Download core
uses: actions/checkout@v4.1.1
uses: actions/checkout@v7
with:
repository: ${{ env.STACKQL_CORE_REPOSITORY }}
ref: ${{ env.STACKQL_CORE_REF }}
token: ${{ secrets.CI_STACKQL_PACKAGE_DOWNLOAD_TOKEN }}
path: stackql-core

- name: Download any-sdk
uses: actions/checkout@v4.1.1
uses: actions/checkout@v7
with:
repository: ${{ env.STACKQL_ANY_SDK_REPOSITORY }}
ref: ${{ env.STACKQL_ANY_SDK_REF }}
token: ${{ secrets.CI_STACKQL_PACKAGE_DOWNLOAD_TOKEN }}
path: stackql-any-sdk

- name: Setup Python
uses: actions/setup-python@v5.0.0
uses: actions/setup-python@v6
with:
python-version: '3.12'

Expand Down Expand Up @@ -143,7 +142,7 @@ jobs:
fi

- name: Upload AOT analysis logs
uses: actions/upload-artifact@v4.3.1
uses: actions/upload-artifact@v7
if: always()
with:
name: aot_analysis_logs_${{ github.event.repository.name }}_${{ github.run_id }}
Expand Down
71 changes: 69 additions & 2 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
REG_DENO_DEPLOY_API_PROD: stackql-registry

steps:
- uses: actions/checkout@v4.1.1
- uses: actions/checkout@v7
name: "[SETUP] checkout repo"
with:
fetch-depth: 0
Expand Down Expand Up @@ -67,7 +67,7 @@ jobs:

- name: '[PACKAGE] set up golang'
if: env.NUM_PROVIDERS > 0
uses: actions/setup-go@v5.0.0
uses: actions/setup-go@v6
with:
go-version: ^1.19
check-latest: true
Expand Down Expand Up @@ -162,6 +162,73 @@ jobs:
run: |
python scripts/deploy/pull-additional-docs-from-artifact-repo.py

#
# Cloudflare (green) dual-publish. Runs here, BEFORE clean-deploy-dir.py
# flattens/destroys the working tree (which removes the origin/ Worker
# source). The full docs tree (changed providers + everything pulled from
# the artifact repo, plus the freshly generated providers.yaml) lives at
# ${REG_WEBSITE_DIR}/${REG_PROVIDER_PATH} at this point, byte-identical to
# what the Deno origin is about to deploy. Same push/branch gating as the
# Deno steps, so blue and green stay in sync during the transition window.
#

- name: "[DEPLOY-CF] install worker deps"
if: env.REG_EVENT == 'push'
run: |
cd origin && npm install

- name: "[DEPLOY-CF] sync docs to R2 (dev)"
if: env.REG_TARGET_BRANCH == 'dev' && env.REG_EVENT == 'push'
env:
AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
AWS_SESSION_TOKEN: ""
AWS_DEFAULT_REGION: auto
AWS_REGION: auto
R2_BUCKET: stackql-provider-registry-dev
run: |
# version-pinned .tgz are immutable -> --size-only keeps R2 ops low and skips re-uploads
aws s3 sync "${REG_WEBSITE_DIR}/${REG_PROVIDER_PATH}" "s3://${R2_BUCKET}/${REG_PROVIDER_PATH}" \
--endpoint-url "https://${{ secrets.CLOUDFLARE_ACCOUNT_ID }}.r2.cloudflarestorage.com" \
--region auto --size-only --delete --no-progress
# providers.yaml can change without changing size -> always overwrite it
aws s3 cp "${REG_WEBSITE_DIR}/${REG_PROVIDER_PATH}/providers.yaml" "s3://${R2_BUCKET}/${REG_PROVIDER_PATH}/providers.yaml" \
--endpoint-url "https://${{ secrets.CLOUDFLARE_ACCOUNT_ID }}.r2.cloudflarestorage.com" \
--region auto --no-progress

- name: "[DEPLOY-CF] sync docs to R2 (prod)"
if: env.REG_TARGET_BRANCH == 'main' && env.REG_EVENT == 'push'
env:
AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
AWS_SESSION_TOKEN: ""
AWS_DEFAULT_REGION: auto
AWS_REGION: auto
R2_BUCKET: stackql-provider-registry
run: |
aws s3 sync "${REG_WEBSITE_DIR}/${REG_PROVIDER_PATH}" "s3://${R2_BUCKET}/${REG_PROVIDER_PATH}" \
--endpoint-url "https://${{ secrets.CLOUDFLARE_ACCOUNT_ID }}.r2.cloudflarestorage.com" \
--size-only --delete --no-progress
aws s3 cp "${REG_WEBSITE_DIR}/${REG_PROVIDER_PATH}/providers.yaml" "s3://${R2_BUCKET}/${REG_PROVIDER_PATH}/providers.yaml" \
--endpoint-url "https://${{ secrets.CLOUDFLARE_ACCOUNT_ID }}.r2.cloudflarestorage.com" \
--no-progress

- name: "[DEPLOY-CF] deploy worker (dev)"
if: env.REG_TARGET_BRANCH == 'dev' && env.REG_EVENT == 'push'
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
run: |
cd origin && npx wrangler deploy --env dev

- name: "[DEPLOY-CF] deploy worker (prod)"
if: env.REG_TARGET_BRANCH == 'main' && env.REG_EVENT == 'push'
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
run: |
cd origin && npx wrangler deploy --env production

- name: "[DEPLOY] install deno"
if: env.REG_EVENT == 'push'
uses: denoland/setup-deno@main
Expand Down
31 changes: 15 additions & 16 deletions .github/workflows/regression.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,16 @@ on:
- main
- dev
push:
branches:
- main
- dev
# branch pushes (merges) are not re-tested here; the pull_request run above is
# the merge gate. Tag pushes still trigger explicit on-demand analysis.
tags:
- robot*
- regression*
- integration*

env:
IS_TAG: ${{ github.ref_type == 'tag' }}
GO_VERSION: '~1.22'
GO_VERSION: 'stable' # stackql-core (built from main) tracks recent Go; stable always satisfies its go.mod
STACKQL_CORE_REPOSITORY: ${{ vars.STACKQL_CORE_REPOSITORY != '' && vars.STACKQL_CORE_REPOSITORY || 'stackql/stackql' }}
STACKQL_CORE_REF: ${{ vars.STACKQL_CORE_REF != '' && vars.STACKQL_CORE_REF || 'main' }}
STACKQL_ANY_SDK_REPOSITORY: ${{ vars.STACKQL_ANY_SDK_REPOSITORY != '' && vars.STACKQL_ANY_SDK_REPOSITORY || 'stackql/any-sdk' }}
Expand All @@ -32,15 +31,15 @@ jobs:
steps:

- name: Check out code into the Go module directory
uses: actions/checkout@v4.1.1
uses: actions/checkout@v7
with:
repository: ${{ env.STACKQL_CORE_REPOSITORY }}
ref: ${{ env.STACKQL_CORE_REF }}
token: ${{ secrets.CI_STACKQL_PACKAGE_DOWNLOAD_TOKEN }}
path: stackql-core-pkg

- name: Setup Python
uses: actions/setup-python@v5.0.0
uses: actions/setup-python@v6
with:
cache: pip
python-version: '3.12'
Expand All @@ -61,7 +60,7 @@ jobs:
cicd/util/01-build-robot-lib.sh

- name: Upload python package artifact
uses: actions/upload-artifact@v4.3.1
uses: actions/upload-artifact@v7
with:
name: python-package-dist-folder
path: stackql-core-pkg/test/dist
Expand All @@ -86,37 +85,37 @@ jobs:
REG_DENO_DEPLOY_API_PROD: stackql-registry

steps:
- uses: actions/checkout@v4.1.1
- uses: actions/checkout@v7
name: "[SETUP] checkout repo"
with:
fetch-depth: 0

- name: Set up Go 1.x
uses: actions/setup-go@v5.0.0
uses: actions/setup-go@v6
with:
go-version: ${{ env.GO_VERSION }}
check-latest: true
cache: true
id: go

- name: Download core
uses: actions/checkout@v4.1.1
uses: actions/checkout@v7
with:
repository: ${{ env.STACKQL_CORE_REPOSITORY }}
ref: ${{ env.STACKQL_CORE_REF }}
token: ${{ secrets.CI_STACKQL_PACKAGE_DOWNLOAD_TOKEN }}
path: stackql-core

- name: Download any-sdk
uses: actions/checkout@v4.1.1
uses: actions/checkout@v7
with:
repository: ${{ env.STACKQL_ANY_SDK_REPOSITORY }}
ref: ${{ env.STACKQL_ANY_SDK_REF }}
token: ${{ secrets.CI_STACKQL_PACKAGE_DOWNLOAD_TOKEN }}
path: stackql-any-sdk

- name: Setup Python
uses: actions/setup-python@v5.0.0
uses: actions/setup-python@v6
with:
python-version: '3.12'

Expand Down Expand Up @@ -276,7 +275,7 @@ jobs:
python3 scripts/cicd/python/robot-parse.py --robot-output-file stackql-core/test/robot/reports/output.xml > stackql-core/test/robot/reports/proxied_parsed_output.json

- name: Upload core traffic lights
uses: actions/upload-artifact@v4.3.1
uses: actions/upload-artifact@v7
if: success()
with:
name: proxied-core-traffic-lights
Expand Down Expand Up @@ -315,7 +314,7 @@ jobs:
python3 scripts/cicd/python/robot-parse.py --robot-output-file test/robot/reports/mocked/output.xml > test/robot/reports/mocked/parsed_output.json

- name: Upload local registry mocked traffic lights
uses: actions/upload-artifact@v4.3.1
uses: actions/upload-artifact@v7
if: success()
with:
name: local-registry-mocked-traffic-lights
Expand Down Expand Up @@ -352,7 +351,7 @@ jobs:
python3 scripts/cicd/python/robot-parse.py --robot-output-file test/robot/reports/readonly/output.xml > test/robot/reports/readonly/parsed_output.json

- name: Upload readonly traffic lights
uses: actions/upload-artifact@v4.3.1
uses: actions/upload-artifact@v7
if: success()
with:
name: local-registry-readonly-traffic-lights
Expand Down Expand Up @@ -381,7 +380,7 @@ jobs:
python3 scripts/cicd/python/robot-parse.py --robot-output-file test/robot/reports/readwrite/output.xml > test/robot/reports/readwrite/parsed_output.json

- name: Upload readonly traffic lights
uses: actions/upload-artifact@v4.3.1
uses: actions/upload-artifact@v7
if: success()
with:
name: local-registry-readwrite-traffic-lights
Expand Down
5 changes: 5 additions & 0 deletions origin/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
node_modules/
package-lock.json
.wrangler/
.dev.vars
dist/
Loading
Loading