Decouple Calcite PPL planning from ExprType#5633
Conversation
PR Reviewer Guide 🔍(Review updated until commit 689405c)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to 689405c Explore these optional code suggestions:
Previous suggestionsSuggestions up to commit 1e91aa8
Suggestions up to commit dc13ee5
Suggestions up to commit 4aebb3d
Suggestions up to commit 4aebb3d
Suggestions up to commit 3fa07c2
|
82f5607 to
a385878
Compare
|
Persistent review updated to latest commit a385878 |
a385878 to
3fa07c2
Compare
|
Persistent review updated to latest commit 3fa07c2 |
9675dd1 to
4aebb3d
Compare
|
Persistent review updated to latest commit 4aebb3d |
1 similar comment
|
Persistent review updated to latest commit 4aebb3d |
Rewrite the Calcite-side PPL planning surface so RelNode/RexNode code, coercion, type checking, and UDF implementations operate on RelDataType instead of bouncing through the v2 ExprType system. UDT identity at planning time: - UDTs (ExprDateType, ExprTimeType, ExprTimeStampType, ExprIPType, ExprBinaryType) are recognised via instanceof rather than getExprType(). Subclasses are preserved through createTypeWithNullability / createTypeWithCharsetAndCollation via a cloneWith hook on ExprSqlType / ExprJavaType so the instanceof checks survive type-factory operations. Calcite-side rewrites: - CoercionUtils: new RelDataType-typed common-type resolver with an internal CoercionTag widening DAG that mirrors v2 semantics. Widening produces UDT-normalized temporal types and always makes the cast target nullable so safe cast + primitive aggregators do not NPE on parse failure. - PPLTypeChecker: signatures expressed as List<List<RelDataType>>; adds renderTypeName() for error messages; folds DECIMAL to DOUBLE. isComparable() accepts UDT temporal vs standard temporal of same kind. - PPLOperandTypes: exposes RelDataType signature constants (DATE_UDT, INTEGER_T, ...). - PPLFuncImpTable.requiresNumericArgument: uses SqlTypeUtil.isNumeric and SqlTypeName.ANY for the "unknown-type" check. - ExtendedRexBuilder, AddSubDate/Extract/Format/LastDay/PeriodName/ TimestampAdd/TimestampDiff/Weekday/Span/WidthBucket and the ip UDFs branch on UDT classes and pass RelDataType through their implementors. - visitCast in CalciteRexNodeVisitor maps AST DataType directly to RelDataType, removing the DataType.getCoreType() round-trip. - CurrentFunction / FormatFunction take an internal Kind / boolean discriminator rather than an ExprCoreType. - DatetimeExtension switches from ExprUDT enum to type-class checks. Signed-off-by: Peng Huo <penghuo@gmail.com>
4aebb3d to
dc13ee5
Compare
|
Persistent review updated to latest commit dc13ee5 |
…eVisitor churn typesMatch compared UDTs via getClass(), but addCharsetAndCollation strips concrete subclass identity from VARCHAR-backed UDTs (ExprDateType, ExprTimeType, ExprTimeStampType, ExprBinaryType) — they all collapse to ExprSqlType. That let wrapUDT accept mismatched UDTs at the signature gate: `cidrmatch(date_field, "1.2.3.4/24")` matched the [BINARY_UDT, STRING_T] signature and crashed at runtime trying to parse a date as an IP. Compare via the ExprUDT tag instead, matching what PPLComparableTypeChecker.isComparable already does. Also revert CalciteRexNodeVisitor to upstream/main — the earlier inlined visitCast switch duplicated convertExprTypeToRelDataType, and the visitBetween comment tweak had no code change. Signed-off-by: Peng Huo <penghuo@gmail.com>
|
Persistent review updated to latest commit 1e91aa8 |
Signed-off-by: Peng Huo <penghuo@gmail.com>
|
Persistent review updated to latest commit 689405c |
Description
Rewrite the Calcite-side PPL planning surface so
RelNode/RexNodecode, coercion, type checking, and UDF implementations operate onRelDataTypedirectly instead of round-tripping through the v2ExprTypesystem. UDTs (ExprDateType,ExprTimeType,ExprTimeStampType,ExprIPType,ExprBinaryType) remain in the logical plan unchanged — only the planning-time abstraction is decoupled.Scope: PPL planning surface only. No changes to runtime UDF execution or the storage engine. IP/BINARY UDTs and temporal UDTs remain in logical plans; this PR only stops passing them through the
ExprTypeconversion layer at planning time.Key rewrites:
CoercionUtils— newRelDataType-typed common-type resolver with an internalCoercionTagwidening DAG that mirrors v2 semantics (STRING → TIMESTAMP direct edge, UNDEFINED → any-concrete at unit cost). Widening normalizes temporal types to their UDT equivalents; cast targets are always made nullable so safe cast + primitive aggregators do not NPE on parse failure.PPLTypeChecker— signatures expressed asList<List<RelDataType>>; addsrenderTypeName()for error messages; foldsDECIMAL→DOUBLE.isComparable()accepts UDT temporal vs standard temporal of the same kind.PPLOperandTypes— exposesRelDataTypesignature constants (DATE_UDT,INTEGER_T, …).PPLFuncImpTable.requiresNumericArgument— usesSqlTypeUtil.isNumericandSqlTypeName.ANYfor the "unknown-type" check.ExtendedRexBuilder,AddSubDate/Extract/Format/LastDay/PeriodName/TimestampAdd/TimestampDiff/Weekday/Span/WidthBucketand the IP UDFs — branch on UDT classes and passRelDataTypethrough their implementors.visitCastinCalciteRexNodeVisitor— maps ASTDataTypedirectly toRelDataType, removing theDataType.getCoreType()round-trip.CurrentFunction/FormatFunction— take an internalKind/booleandiscriminator rather than anExprCoreType.DatetimeExtension— switches fromExprUDTenum to type-class checks.UDT identity at planning time is now determined via
instanceof. Subclasses are preserved throughcreateTypeWithNullability/createTypeWithCharsetAndCollationvia acloneWithhook onExprSqlType/ExprJavaTypeso theinstanceofchecks survive type-factory operations.Related Issues
N/A
Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.