Skip to content

Adds Iterable<DataRowSchema>.toDataFrame() overload#1907

Open
Jolanrensen wants to merge 2 commits into
masterfrom
DataRowSchema-Iterable-toDataFrame
Open

Adds Iterable<DataRowSchema>.toDataFrame() overload#1907
Jolanrensen wants to merge 2 commits into
masterfrom
DataRowSchema-Iterable-toDataFrame

Conversation

@Jolanrensen

Copy link
Copy Markdown
Collaborator

Fixes #1880

Added Iterable.toDataFrame() overload for DataRowSchema types,
compiler plugin test in :samples, because the compiler plugin is live there.

This fixes the compiler plugin creating a new type immediately when turning a list of @DataSchema annotated classes into a dataframe.

@Jolanrensen Jolanrensen added Compiler plugin Anything related to the DataFrame Compiler Plugin API If it touches our API labels Jun 22, 2026
@Jolanrensen

Copy link
Copy Markdown
Collaborator Author
image As expected, this directly touches the kotlin-spark dev example :D (where this limitation was detected) I'll fix the example right away

@Jolanrensen

Jolanrensen commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator Author

A workaround for #1908 could be to write the function like this:

@JvmName("dataRowSchemaIterableToDataFrame")
@Suppress("INCORRECT_LEFT_COMPONENT_OF_INTERSECTION")
public inline fun <reified T : DataRowSchema> Iterable<T>.toDataFrame(): DataFrame<T & Any> =
    @DisableInterpretation
    toDataFrame {
        properties()
    }

But notebooks don't like T & Any and it could better be solved from the compiler plugin side I believe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

API If it touches our API Compiler plugin Anything related to the DataFrame Compiler Plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid breaking existing code with Iterable<DataSchema>.toDataFrame() calls when enabling the plugin

1 participant