[BugFix] Detect BIGINT overflow in SUM and fix wrong AVG on the Calcite engine

[ahkcs]

Description

SUM/AVG over a BIGINT (long) column near 2^63 silently produced wrong results on the Calcite engine:

• SUM(long) — the enumerable accumulator is a plain long, so a running sum past 2^63 wrapped to a negative value (e.g. SUM(UserID) returned -5594372458244005145 with HTTP 200).
• AVG(long) — AVG reduces to SUM(field)/COUNT(field); the intermediate long sum wrapped the same way, so AVG returned a wrong (often negative) result.

Fix

Applied in the SQL-plugin lowering so the DSL-pushdown and in-memory (no-pushdown) paths behave identically:

• SUM(long) — the aggregate argument (a bare BIGINT column) is summed in DECIMAL so it cannot wrap, and the result is narrowed back to BIGINT via a new CHECKED_LONG_NARROW UDF. Narrowing raises a client error (4xx) when the sum clearly overflowed 2^63, and otherwise saturates. The output type stays bigint.
• AVG(long) — the bare BIGINT argument is averaged in DOUBLE, which holds the true average without an intermediate long wrap. The DOUBLE output type is unchanged.

Only bare BIGINT column references are widened; expressions such as sum(field + 1) are left untouched so PPLAggregateConvertRule can still rewrite them to pushdown-friendly form. Narrower integer sums (byte/short/int) already widen to a BIGINT accumulator and cannot overflow, so they are unchanged.

Behavior notes (intentional trade-offs)

• Boundary: OpenSearch computes pushed-down sums in double, and (double) Long.MAX_VALUE rounds to exactly 2^63, indistinguishable from a small overflow. To avoid erroring on a legitimate near-Long.MAX_VALUE sum, only magnitudes strictly beyond 2^63 are treated as overflow; a genuine overflow lands far outside that boundary, and the in-memory path detects it exactly.
• AVG pushdown: casting the AVG argument to DOUBLE keeps native avg pushdown in most cases, but an avg with a preceding sort can fall back to an in-memory sum/count reduction instead of a pushed-down native avg. The result stays correct; only that narrow case loses aggregate pushdown.

Before / After

Query	Before	After
stats sum(long_field) (overflow)	wrong negative value, HTTP 200	HTTP 400
stats avg(long_field) (overflow)	wrong negative value	correct double
stats sum(long_field) (in range)	correct	correct (bigint)
stats sum(Long.MAX)	correct	correct (no false error)

Testing

• CalcitePPLAggregationIT.testSumAvgLongOverflow — overflow → 4xx, avg correct, in-range and single-Long.MAX sums; runs on both the pushdown path and the no-pushdown path (via CalciteNoPushdownIT).
• REST yaml test issues/5164_agg.yml.
• Regenerated the affected explain fixtures under expectedOutput/calcite/ and expectedOutput/calcite_no_pushdown/.

Related

Addresses the aggregate half of #5164. Stacked on #5604 (reuses its ArithmeticException → 4xx routing); the diff will shrink once #5604 merges.

Check List

• New functionality includes testing.
• New functionality has been documented.
• Commits are signed per the DCO using --signoff.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 0a077e0)

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 No security concerns identified

✅ No TODO sections

🔀 No multiple PR themes

⚡ Recommended focus areas for review Possible Issue The narrow method casts the input to Number and calls doubleValue() without verifying the input is actually a Number. If the UDF is invoked with a non-numeric type (e.g., a string or null that bypasses NullPolicy.ANY), this will throw a ClassCastException at runtime instead of a controlled error. The method should validate the input type or document that it assumes the caller enforces numeric operands. public static long narrow(Object value) { double magnitude = ((Number) value).doubleValue(); Possible Issue The castBigintSumResults method checks aggExprList.size() != aggRexList.size() and returns aggRexList unchanged if they differ. However, if the sizes differ, iterating over both lists in the loop (line 1849) will either skip elements or throw an IndexOutOfBoundsException. The early return prevents the exception, but the condition suggests a logic error: if the lists are expected to align, a size mismatch indicates a bug elsewhere. If they can legitimately differ, the method should handle partial processing or document why returning the original list is correct. if (aggExprList.size() != aggRexList.size()) { return aggRexList;

[github-actions]

Persistent review updated to latest commit f01ed11

[github-actions]

PR Code Suggestions ✨

Latest suggestions up to 0a077e0

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Add null value handling The method doesn't handle null values, which could cause a NullPointerException when calling doubleValue() on a null Number. Add a null check at the beginning of the method to return a default value or throw a more descriptive exception. core/src/main/java/org/opensearch/sql/expression/function/udf/CheckedLongNarrowFunction.java [55-69] public static long narrow(Object value) { + if (value == null) { + throw new IllegalArgumentException("Cannot narrow null value to BIGINT"); + } double magnitude = ((Number) value).doubleValue(); if (magnitude > TWO_POW_63 || magnitude < -TWO_POW_63) { throw new ArithmeticException("BIGINT overflow in SUM"); } if (value instanceof BigDecimal decimal) { return decimal .min(BigDecimal.valueOf(Long.MAX_VALUE)) .max(BigDecimal.valueOf(Long.MIN_VALUE)) .longValue(); } return (long) magnitude; } Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies that the narrow method doesn't handle null values, which could cause a NullPointerException. Adding a null check improves robustness, though the actual impact depends on whether nulls can reach this method in practice (the UDF uses NullPolicy.ANY which may handle nulls before reaching this method).	Medium

---

Previous suggestions

Suggestions up to commit 31bbdf2

Category	Suggestion	Impact
Possible issue	Add null value handling The narrow method doesn't handle null values, which could cause a NullPointerException when calling doubleValue() on a null Number. Add a null check at the beginning of the method to return a default value or throw a more descriptive exception. core/src/main/java/org/opensearch/sql/expression/function/udf/CheckedLongNarrowFunction.java [55-69] public static long narrow(Object value) { + if (value == null) { + throw new IllegalArgumentException("Cannot narrow null value to BIGINT"); + } double magnitude = ((Number) value).doubleValue(); if (magnitude > TWO_POW_63 || magnitude < -TWO_POW_63) { throw new ArithmeticException("BIGINT overflow in SUM"); } if (value instanceof BigDecimal decimal) { return decimal .min(BigDecimal.valueOf(Long.MAX_VALUE)) .max(BigDecimal.valueOf(Long.MIN_VALUE)) .longValue(); } return (long) magnitude; } Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies that the narrow method lacks null handling, which could cause a NullPointerException. However, the score is moderate because the context of how this method is called (within aggregate operations) may already guarantee non-null values, making this a defensive programming improvement rather than a critical bug fix.	Medium

Suggestions up to commit 4d2c6fc

Category	Suggestion	Impact
Possible issue	Fix asymmetric overflow detection The overflow check uses magnitude > TWO_POW_63 which will never trigger for Long.MIN_VALUE (-2^63) since its magnitude is exactly 2^63, not greater. This asymmetry means negative overflows beyond -2^63 won't be detected. Use Math.abs(magnitude) > TWO_POW_63 or separate checks for positive/negative bounds. core/src/main/java/org/opensearch/sql/expression/function/udf/CheckedLongNarrowFunction.java [55-69] public static long narrow(Object value) { double magnitude = ((Number) value).doubleValue(); - if (magnitude > TWO_POW_63 || magnitude < -TWO_POW_63) { + if (Math.abs(magnitude) > TWO_POW_63) { throw new ArithmeticException("BIGINT overflow in SUM"); } if (value instanceof BigDecimal decimal) { return decimal .min(BigDecimal.valueOf(Long.MAX_VALUE)) .max(BigDecimal.valueOf(Long.MIN_VALUE)) .longValue(); } return (long) magnitude; } Suggestion importance[1-10]: 2 __ Why: The suggestion misunderstands the overflow check logic. The condition magnitude > TWO_POW_63 || magnitude < -TWO_POW_63 correctly checks both positive and negative bounds since magnitude is a signed double. The value -TWO_POW_63 is exactly -2^63, and values less than this (more negative) will be caught. The check is symmetric and correct as written.	Low

Suggestions up to commit f01ed11

Category	Suggestion	Impact
General	Strengthen type matching to avoid false positives The method assumes that any DECIMAL with scale 0 and max precision is a widened BIGINT sum, but this could incorrectly match user-defined DECIMAL columns that happen to have the same characteristics. Consider adding additional context checks (e.g., verifying the aggregate function is SUM) to avoid false positives. core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java [1870-1874] private static boolean isWidenedBigintSumType(RelDataType type) { + // Only match the specific DECIMAL type created by SUM(bigint) widening return type.getSqlTypeName() == SqlTypeName.DECIMAL && type.getScale() == 0 - && type.getPrecision() == OpenSearchTypeSystem.MAX_PRECISION; + && type.getPrecision() == OpenSearchTypeSystem.MAX_PRECISION + && !type.isNullable(); // Widened types preserve nullability; add more checks if needed } Suggestion importance[1-10]: 3 __ Why: The suggestion raises a valid concern about potential false positives, but the proposed solution (checking !type.isNullable()) is incorrect because widened types do preserve nullability (as the comment in the suggestion itself notes). The method is called only within castBigintSumResults, which already verifies the aggregate function is SUM, providing sufficient context. The concern is minor and the fix is not appropriate.	Low

[penghuo]

call-out alg difference between pushdown enabled vs disabled?

Two docs indexed: v = 4611686018427387904 (2^62) and v = 1. True sum = 4611686018427387905 (fits BIGINT exactly).

The no-pushdown PPL path (via widenBigintColumnToDecimal + CHECKED_LONG_NARROW) returns the exact bigint. The pushdown path silently loses precision because OpenSearch computes the sum in double, and 2^62 + 1 isn't representable

[ahkcs]

Added this distinction to the overflow documentation.

[github-actions]

Persistent review updated to latest commit 4d2c6fc

[github-actions]

Persistent review updated to latest commit 31bbdf2

[github-actions]

Persistent review updated to latest commit 0a077e0

[penghuo]

#5604 use Math.addExact to detect overflow. Why Sum choose another approach?

[ahkcs]

#5604 handles scalar arithmetic such as a + b. Calcite represents that operation as a visible RexCall(PLUS, a, b), so we can replace it with CHECKED_PLUS, which executes using Math.addExact.

SUM is represented as an AggregateCall; its repeated additions happen internally inside Calcite’s aggregate accumulator, so there are no individual PLUS nodes for #5604’s rewrite to replace. Using Math.addExact would require a custom aggregate implementation, would not apply to native OpenSearch pushdown, and could report an order-dependent intermediate overflow even when the final mathematical sum fits.

Therefore, the no-pushdown path accumulates BIGINT values in a wider DECIMAL, then checks and narrows the final result back to BIGINT. Keeping the standard SUM operator also preserves the existing pushdown optimizations.