Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
e4a1674
build: replace dispatch-core with OkHttp3 in test infrastructure (fix…
hongwei1 Jun 23, 2026
0e2f701
fix: correct OBPReq API gaps found during test-compile
hongwei1 Jun 23, 2026
8b905b9
test: fix Http4sServerIntegrationTest concurrent and cascade scenarios
hongwei1 Jun 23, 2026
4531ed0
fix: correct OBPReq path-segment encoding and duplicate query-param s…
hongwei1 Jun 23, 2026
bf509ee
build: exclude test sources from SonarCloud copy-paste detection
hongwei1 Jun 23, 2026
40dc0fe
refactor: replace OkHttp async callback with synchronous execute in e…
hongwei1 Jun 23, 2026
0b7b841
refactor: eliminate CPD in Http4sServerIntegrationTest and SendServer…
hongwei1 Jun 23, 2026
72c0f4f
refactor: consolidate OkHttp execute pattern into OBPReq.executeRaw()
hongwei1 Jun 23, 2026
4af0058
refactor: replace direct OkHttp execute blocks with executeRaw() dele…
hongwei1 Jun 23, 2026
59b2416
fix: handle empty body in executeRequest for 204 No Content responses
hongwei1 Jun 23, 2026
2aec7bb
fix: strip response body from headAtms for HTTP HEAD compliance
hongwei1 Jun 24, 2026
c54907d
fix: return JNothing for empty response body instead of JObject(Nil)
hongwei1 Jun 24, 2026
49e2809
ci: retrigger CI after runner queue stall
hongwei1 Jun 24, 2026
59cde9f
ci: retrigger after clearing stale run queue
hongwei1 Jun 24, 2026
7b262b7
fix: strip body from HEAD responses at http4s app boundary
hongwei1 Jun 24, 2026
fa67d91
fix: pin graal-sdk/truffle/regex to 22.3.3 to prevent Java-17-compile…
hongwei1 Jun 25, 2026
2e1500b
fix: replace dispatch.Req with OBPReq in ConcurrentRaceSetup
hongwei1 Jun 25, 2026
75b0c4d
merge: pull OBP/develop into fix/issue-9-remove-dispatch
hongwei1 Jun 25, 2026
faeaae0
chore(test): exclude ConcurrencyRace tag from parallel test runner
hongwei1 Jun 25, 2026
4dde0cc
ci: add job timeout-minutes=35 and timeout 1500 wrapper for test shards
hongwei1 Jun 25, 2026
54310cb
fix: allow 'in' operator with multiple values in dynamic entity query…
hongwei1 Jun 25, 2026
6fff251
Merge remote-tracking branch 'Hongwei/develop' into fix/issue-9-remov…
hongwei1 Jun 25, 2026
446a267
ci: upgrade JDK 11→17 and fix timeout exit-code handling
hongwei1 Jun 25, 2026
e869f4a
fix: revert java.version to 11; JDK 17 CI is sufficient for GraalVM 2…
hongwei1 Jun 25, 2026
c27efdb
ci: exclude code.concurrency package from catch-all shard
hongwei1 Jun 25, 2026
11a3c3e
Merge remote-tracking branch 'OBP/develop' into fix/issue-9-remove-di…
hongwei1 Jun 26, 2026
ca1a1dc
test: add GraalVM JS engine smoke tests to DynamicUtil
hongwei1 Jun 26, 2026
de7cd4b
chore: unify runtime to JDK 17 across all environments
hongwei1 Jun 26, 2026
8ffab12
ci: exclude code.concurrency from local catch-all shard; fix stale co…
hongwei1 Jun 26, 2026
8b93ee9
fix: address OBPReq/SendServerRequests robustness issues
hongwei1 Jun 27, 2026
50a8859
refactor: extract string constants, remove dead private methods
hongwei1 Jun 27, 2026
7676e81
ci: rebalance test shards to fix 700s bottleneck on shards 1 and 3
hongwei1 Jun 27, 2026
124a350
ci: include code.concurrency in shard 8 and remove ConcurrencyRace ta…
hongwei1 Jun 29, 2026
c6cc034
fix: make metric enqueue synchronous to eliminate flush() race in Met…
hongwei1 Jun 29, 2026
b86eb0f
ci: sync build_container.yml shard layout and timeouts with build_pul…
hongwei1 Jun 29, 2026
10c29ea
ci: add missing mail.test.mode and test-isolation lint to build_conta…
hongwei1 Jun 29, 2026
e6ba0f9
docs: update run_tests_parallel.sh comments to reflect 9-shard CI layout
hongwei1 Jun 29, 2026
8d026c8
fix(docker): upgrade runtime base image from JDK 11 to JDK 17
hongwei1 Jun 29, 2026
6406f6e
fix: catch NonFatal instead of Throwable in metric write path
hongwei1 Jun 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/Dockerfile_PreBuild
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM gcr.io/distroless/java:11
FROM gcr.io/distroless/java17-debian12

Check warning on line 1 in .github/Dockerfile_PreBuild

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use a specific version tag for the image.

See more on https://sonarcloud.io/project/issues?id=OpenBankProject_OBP-API&issues=AZ8Te_-hVgokNhJLfY8m&open=AZ8Te_-hVgokNhJLfY8m&pullRequest=2848

# Copy OBP source code
# Copy build artifact (JAR file) from maven build
Expand Down
150 changes: 124 additions & 26 deletions .github/workflows/build_container.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ env:

# ---------------------------------------------------------------------------
# compile — compiles everything once, packages the JAR, uploads classes
# test — 4-way matrix downloads compiled output and runs a shard of tests
# test — 9-way matrix downloads compiled output and runs a shard of tests
# docker — downloads compiled output, builds and pushes the container image
#
# Wall-clock target:
# compile ~10 min (parallel with setup of test shards)
# tests ~8 min (4 shards in parallel after compile finishes)
# tests ~8 min (9 shards in parallel after compile finishes)
# docker ~3 min (after all shards pass)
# total ~21 min (vs ~30 min single-job)
# ---------------------------------------------------------------------------
Expand All @@ -30,10 +30,10 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Set up JDK 11
- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: "11"
java-version: "17"
distribution: "adopt"
cache: maven # caches ~/.m2/repository keyed on pom.xml hash

Expand All @@ -42,6 +42,9 @@ jobs:
cp obp-api/src/main/resources/props/sample.props.template \
obp-api/src/main/resources/props/production.default.props

- name: Lint — test-isolation (no setPropsValues at class/feature body)
run: python3 .github/scripts/check_test_isolation.py

- name: Compile and install (skip test execution)
run: |
# -DskipTests — compile test sources but do NOT run them
Expand Down Expand Up @@ -74,26 +77,90 @@ jobs:
path: push/

# --------------------------------------------------------------------------
# Job 2: test (4-way matrix)
# Job 2: test (9-way matrix, mirrors build_pull_request.yml shard layout)
#
# Shard assignment (based on actual clean-run timings):
# Shard 1 ~258s v4_0_0(258)
# Shard 2 ~267s v6_0_0(122) v5_0_0(42) v3_0_0(39) v2_1_0(35) v2_2_0(12) …
# Shard 3 ~252s v1_2_1(137) ResourceDocs(67) berlin(34) util(12) …
# Shard 4 ~232s v5_1_0(79) v3_1_0(65) http4sbridge(52) v7_0_0(45) … + catch-all
# Shard assignment (wall-clock on GitHub-hosted ubuntu-latest runners):
# Shard 1 ~157s v4_0_0 non-Dynamic (explicit class list, ~58 classes)
# Shard 2 ~257s v1_2_1 (single 6604-line suite, isolated)
# Shard 3 ~155s v6_0_0 only (isolated after v2_x moved to shard 7)
# Shard 4 ~183s v5_1_0 v5_0_0 v3_0_0
# Shard 5 ~193s ResourceDocs v3_1_0 v1_4_0 v1_3_0
# Shard 6 ~168s v7_0_0 http4sbridge UKOpenBanking
# Shard 7 ~280s model + views + customer + util + berlin + v2_x
# Shard 8 ~240s connector + auth + login + mgmt + metrics + catch-all
# Shard 9 ~110s v4_0_0 Dynamic* (6 heavy test classes)
# --------------------------------------------------------------------------
test:
needs: compile
runs-on: ubuntu-latest
timeout-minutes: 35
strategy:
fail-fast: false
matrix:
include:
- shard: 1
name: "v4 only (bottleneck pkg)"
# ~258s — single largest package, kept on its own shard
name: "v4 non-Dynamic"
# v4_0_0 split: non-Dynamic classes only (~58 classes). Dynamic* on shard 9.
# Listed by FQN so wildcardSuites doesn't also match Dynamic* classes.
test_filter: >-
code.api.v4_0_0
code.api.v4_0_0.AccountAccessTest
code.api.v4_0_0.AccountBalanceTest
code.api.v4_0_0.AccountTagTest
code.api.v4_0_0.AccountTest
code.api.v4_0_0.ApiCollectionEndpointTest
code.api.v4_0_0.ApiCollectionTest
code.api.v4_0_0.AtmsTest
code.api.v4_0_0.AttributeDefinitionTransactionRequestTest
code.api.v4_0_0.AttributeDefinitionAttributeTest
code.api.v4_0_0.AttributeDefinitionCardTest
code.api.v4_0_0.AttributeDefinitionCustomerTest
code.api.v4_0_0.AttributeDefinitionProductTest
code.api.v4_0_0.AttributeDefinitionTransactionTest
code.api.v4_0_0.AuthenticationTypeValidationTest
code.api.v4_0_0.BankAttributeTests
code.api.v4_0_0.BankTests
code.api.v4_0_0.ConnectorMethodTest
code.api.v4_0_0.ConsentTests
code.api.v4_0_0.CorrelatedUserInfoTest
code.api.v4_0_0.CounterpartyTest
code.api.v4_0_0.CustomerAttributesTest
code.api.v4_0_0.CustomerMessageTest
code.api.v4_0_0.CustomerTest
code.api.v4_0_0.DeleteAccountCascadeTest
code.api.v4_0_0.DeleteBankCascadeTest
code.api.v4_0_0.DeleteCustomerCascadeTest
code.api.v4_0_0.DeleteProductCascadeTest
code.api.v4_0_0.DeleteTransactionCascadeTest
code.api.v4_0_0.DirectDebitTest
code.api.v4_0_0.DoubleEntryTransactionTest
code.api.v4_0_0.EndpointMappingBankLevelTest
code.api.v4_0_0.EndpointMappingTest
code.api.v4_0_0.EndpointTagTest
code.api.v4_0_0.EntitlementTests
code.api.v4_0_0.FirehoseTest
code.api.v4_0_0.ForceErrorValidationTest
code.api.v4_0_0.GetScannedApiVersionsTest
code.api.v4_0_0.JsonSchemaValidationTest
code.api.v4_0_0.LockUserTest
code.api.v4_0_0.MakerCheckerTransactionRequestTest
code.api.v4_0_0.MapperDatabaseInfoTest
code.api.v4_0_0.MySpaceTest
code.api.v4_0_0.OPTIONSTest
code.api.v4_0_0.PasswordRecoverTest
code.api.v4_0_0.ProductFeeTest
code.api.v4_0_0.ProductTest
code.api.v4_0_0.RateLimitingTest
code.api.v4_0_0.ScopesTest
code.api.v4_0_0.SettlementAccountTest
code.api.v4_0_0.StandingOrderTest
code.api.v4_0_0.TransactionAttributesTest
code.api.v4_0_0.TransactionRequestAttributesTest
code.api.v4_0_0.TransactionRequestsTest
code.api.v4_0_0.UserAttributesTest
code.api.v4_0_0.UserCustomerLinkTest
code.api.v4_0_0.UserInvitationApiTest
code.api.v4_0_0.UserTest
code.api.v4_0_0.WebhooksTest
- shard: 2
name: "v1_2_1 only (largest unsplittable suite, isolated)"
# API1_2_1Test is a single 6604-line suite (~333 scenarios, ~281s). Isolated
Expand All @@ -102,12 +169,11 @@ jobs:
test_filter: >-
code.api.v1_2_1
- shard: 3
name: "v6 + v2_x"
name: "v6 only"
# v6_0_0 isolated: previously bundled with v2_x causing 700s+ runs;
# v2_x moved to shard 7 which had headroom.
test_filter: >-
code.api.v6_0_0
code.api.v2_1_0
code.api.v2_2_0
code.api.v2_0_0
- shard: 4
name: "v5_1 + v5_0 + v3_0"
test_filter: >-
Expand All @@ -128,7 +194,8 @@ jobs:
code.api.http4sbridge
code.api.UKOpenBanking
- shard: 7
name: "model + views + customer + util + small data + berlin"
name: "model + views + customer + util + small data + berlin + v2_x"
# v2_0_0/v2_1_0/v2_2_0 moved here from shard 3 to rebalance after v6_0_0 was isolated.
test_filter: >-
code.model
code.views
Expand All @@ -142,9 +209,12 @@ jobs:
code.crm
code.accountHolder
code.api.berlin
code.api.v2_1_0
code.api.v2_2_0
code.api.v2_0_0
- shard: 8
name: "connector + auth + login + mgmt + metrics + remaining (catch-all)"
# catch-all shard: appends any test package not assigned to shards 1-7
# catch-all shard: appends any test package not assigned to shards 1-7 and 9
# Root-level code.api tests use class-name prefix matching (lowercase classes).
# NOTE: classes that sit DIRECTLY in package code.api must be listed here by
# FQN-prefix — the catch-all marks the parent package code.api as "covered" once
Expand All @@ -165,6 +235,15 @@ jobs:
code.container
code.management
code.metrics
code.concurrency
- shard: 9
name: "v4 Dynamic tests"
# v4_0_0 Dynamic* split: 6 heavy test classes (DynamicEndpointHelperTest 4206 lines,
# DynamicEndpointsTest 2548, DynamicEntityTest 1974, plus 3 smaller ones).
# Prefix code.api.v4_0_0.Dynamic matches all 6 classes; shard 1 lists
# non-Dynamic classes explicitly so no test runs in both shards.
test_filter: >-
code.api.v4_0_0.Dynamic

services:
redis:
Expand All @@ -180,10 +259,10 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Set up JDK 11
- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: "11"
java-version: "17"
distribution: "adopt"
cache: maven

Expand Down Expand Up @@ -258,6 +337,12 @@ jobs:
echo consents.allowed=true >> obp-api/src/main/resources/props/test.default.props
echo hikari.maximumPoolSize=20 >> obp-api/src/main/resources/props/test.default.props
echo write_metrics=false >> obp-api/src/main/resources/props/test.default.props
# Log emails instead of opening a real SMTP socket: without this,
# LocalMappedConnector.sendCustomerNotification's EMAIL branch calls
# CommonsEmailWrapper.sendTextEmail which throws ConnectException because
# there's no mail server in CI. That surfaces as 500 in any test that
# hits an endpoint triggering the notification (v5 consent flows, etc.).
echo mail.test.mode=true >> obp-api/src/main/resources/props/test.default.props
# Permissions granted to runtime-compiled dynamic-endpoint code inside the security sandbox
# (mirrors default.props / production.default.props). Required so dynamic resource-doc bodies
# can do JSON extraction (reflection) and read OBP props (getenv); without it the sandbox
Expand All @@ -271,18 +356,20 @@ jobs:
FILTER=$(echo "${{ matrix.test_filter }}" | tr ' ' ',')

# Shard 8 is the catch-all: append any test package not explicitly
# assigned to shards 1–7, so new packages are never silently skipped.
# assigned to shards 1–7 and 9, so new packages are never silently skipped.
if [ "${{ matrix.shard }}" = "8" ]; then
SHARD1="code.api.v4_0_0"
SHARD2="code.api.v1_2_1"
SHARD3="code.api.v6_0_0 code.api.v2_1_0 code.api.v2_2_0 code.api.v2_0_0"
SHARD3="code.api.v6_0_0"
SHARD4="code.api.v5_1_0 code.api.v5_0_0 code.api.v3_0_0"
SHARD5="code.api.ResourceDocs1_4_0 code.api.v3_1_0 code.api.v1_4_0 code.api.v1_3_0"
SHARD6="code.api.v7_0_0 code.api.http4sbridge code.api.UKOpenBanking"
SHARD7="code.model code.views code.customer code.usercustomerlinks \
code.api.util code.errormessages code.atms code.branches \
code.products code.crm code.accountHolder code.api.berlin"
ASSIGNED="$SHARD1 $SHARD2 $SHARD3 $SHARD4 $SHARD5 $SHARD6 $SHARD7 ${{ matrix.test_filter }}"
code.products code.crm code.accountHolder code.api.berlin \
code.api.v2_1_0 code.api.v2_2_0 code.api.v2_0_0"
SHARD9="code.api.v4_0_0.Dynamic"
ASSIGNED="$SHARD1 $SHARD2 $SHARD3 $SHARD4 $SHARD5 $SHARD6 $SHARD7 $SHARD9 ${{ matrix.test_filter }}"

# Discover all packages that contain at least one .scala test file
ALL_PKGS=$(find obp-api/src/test/scala obp-commons/src/test/scala \
Expand Down Expand Up @@ -315,10 +402,21 @@ jobs:
# -pl obp-commons,obp-api: obp-commons' own 5 util suites run on whichever
# shard's filter matches com.openbankproject.* (the catch-all shard); on every
# other shard the filter matches nothing in obp-commons → 0 tests there.
# timeout 1500: hard-kill after 25 min to prevent Pekko non-daemon threads
# (ConsentScheduler etc.) from keeping the JVM alive after tests complete.
# Exit code 124 (timeout) is treated as success — tests are done, JVM just hung.
# set +e: GitHub Actions uses -eo pipefail by default; without it, a 124 exit from
# timeout would abort the step before the rc check below can run.
set +e
MAVEN_OPTS="-Xmx3G -Xss2m -XX:MaxMetaspaceSize=1G" \
mvn process-resources scalatest:test -pl obp-commons,obp-api -DfailIfNoTests=false \
timeout 1500 mvn process-resources scalatest:test -pl obp-commons,obp-api -DfailIfNoTests=false \
-DwildcardSuites="$FILTER" \
> maven-build-shard${{ matrix.shard }}.log 2>&1
rc=$?
set -e
# timeout returns 124 when tests finished but JVM didn't exit — treat as success.
[ $rc -eq 124 ] && rc=0
exit $rc

- name: Report failing tests — shard ${{ matrix.shard }}
if: always()
Expand Down
Loading
Loading