feat: add expression array_size #1122
Conversation
| } | ||
| } | ||
| let sizes_array = Int32Array::from(builder.finish()); | ||
| Ok(ColumnarValue::Array(Arc::new(sizes_array))) |
There was a problem hiding this comment.
Is there a more efficient way to do this?
spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
Outdated
Show resolved
Hide resolved
viirya
changed the title
spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
Outdated
Show resolved
Hide resolved
|
Why not to use an It looks like it does the same. It may be done in a similar way, like for the |
Good input! will look into it! |
|
Note that DataFusion |
| @@ -2220,6 +2220,16 @@ object QueryPlanSerde extends Logging with ShimQueryPlanSerde with CometExprShim | |||
| expr.children(2)) | |||
| None | |||
| } | |||
| case Size(child, _) if child.dataType.isInstanceOf[ArrayType] => | |||
There was a problem hiding this comment.
This either needs to handle when legacySizeOfNull is true or pattern match to only false
|
Have to wait for this to get into the next version |
Thanks @Groennbeck. The next DataFusion release will likely be released in the next couple of weeks. |
|
@Groennbeck DataFusion 44.0.0 is now released |
Nice! will update this later today. |
|
Hi @Groennbeck are you still planning on update this PR? |
Which issue does this PR close?
Closes #.
Rationale for this change
To add support for expression: array_size
What changes are included in this PR?
How are these changes tested?
Tested by adding spark plan that calculated array_size