Releases: pola-rs/polars
Python Polars 0.20.23
🏆 Highlights
- Add a low-friction
sql
method for DataFrame and LazyFrame (#15783)
🚀 Performance improvements
- Don't rechunk in parallel collection (#15907)
- Improve non-trivial list aggregations (#15888)
- Ensure we hit specialized gather for binary/strings (#15886)
- Limit the cache size for
to_datetime
(#15826) - skip initial null items and don't recompute
slope
ininterpolate
(#15819)
✨ Enhancements
- don't require pyarrow for converting pandas to Polars if all columns have simple numpy-backed datatypes (#15933)
- Add option to disable globbing in csv (#15930)
- Add option to disable globbing in parquet (#15928)
- Expressify
dt.round
(#15861) - Improve error messages in context stack (#15881)
- Add dynamic literals to ensure schema correctness (#15832)
- add timestamp time travel in delta scan/read (#15813)
🐞 Bug fixes
- Set default limit for String column display to 30 and fix edge cases (#15934)
- Change recognition of numba ufunc (#15916)
- series.search_sorted could support more types of input (#15940)
- Remove ffspec from parquet reader (#15927)
- avoid WRITE+EXEC for CPUID check (#15912)
- fix inconsistent decimal formatting (#15457)
- Preserve NULLs for
is_not_nan
(#15889) - double projection check should only take the upstream projections into account (#15901)
- Ensure we don't create invalid frames when combining unit lit + … (#15903)
- Clear cached rename schema (#15902)
- Fix OOB in struct lit/agg aggregation (#15891)
- Refine interaction of "schema_overrides" with
read_excel
when using "calamine" engine (#15827) - Don't modify user-supplied
storage_options
dict (take a shallow-copy) (#15859) - create (q)cut labels in fixed order (#15843)
- Tag
shrink_dtype
as non-streaming (#15828)
📖 Documentation
- improve graphviz install documentation/error message (#15791)
- Extend docstring examples for asof_join (#15810)
📦 Build system
- Don't import jemalloc (#15942)
- Use default allocator for lts-cpu (#15941)
- replace all macos-latest referrals with macos-13 (#15926)
- pin mimalloc and macos-13 (#15925)
- use jemalloc in lts-cpu (#15913)
- Update Cargo.lock (#15865)
- Bump
ruff
version and improvemake clean
on the Python side (#15858) - Exclude
rust-toolchain.toml
from wheels (#15840)
🛠️ Other improvements
- Replace copy/paste import handling with
import_optional
utility function (#15906) - Reorganize from_iter and dispatch to collect_ca when possible (#15904)
- More PyO3 0.21 bound APIs (#15872)
- Improve type-coercion (#15879)
- Move type coercion to IR conversion phase (#15868)
- Fix Python test coverage upload (#15853)
- More upgrades to PyO3 0.21 Bound<> APIs (#15790)
- Use uv for installing Python dependencies in CI (#15848)
- Update benchmark tests (#15825)
Thank you to all our contributors for making this release possible!
@JulianCologne, @MarcoGorelli, @NedJWestern, @NexVeridian, @alexander-beedie, @deanm0000, @dependabot, @dependabot[bot], @ion-elgreco, @itamarst, @jr200, @nameexhaustion, @orlp, @reswqa, @ritchie46 and @stinodego
Python Polars 0.20.22
🚀 Performance improvements
- Improved type-inference for
read_excel
andread_ods
, use calamine engine forread_ods
(#15808) - Fix quadratic in binview growable same source (#15734)
- use two binary searches for equality mask when data is sorted (#15702)
- improve filter parallelism (#15686)
✨ Enhancements
- Minor type-inference update for
read_database
(#15809) - Improved type-inference for
read_excel
andread_ods
, use calamine engine forread_ods
(#15808) dt.truncate
supports broadcasting lhs (#15768)- Expressify
str.json_path_match
(#15764) - raise if
storage_options
is passed to read_csv butfsspec
isnt available (#15778) - Support decimal float parsing in CSV (#15774)
- Add context trace to
LazyFrame
conversion errors (#15761) - Improve error message when passing invalid input to
lit
(#15718) - Remove outdated join validation checks (#15701)
🐞 Bug fixes
- drop-nulls edge case; remove drop-nulls special case (#15815)
- ewm_mean_by was skipping initial nulls when it was already sorted by "by" column (#15812)
- Consult cgroups to determine free memory (#15798)
- raise if index count like 2i is used when performing rolling, group_by_dynamic, upsample, or other temporal operatios (#15751)
- Don't deduplicate sort that has slice pushdown (#15784)
- Allow passing files opened by fsspec in
read_parquet
(#15770) - Fix incorrect
is_between
pushdown toscan_pyarrow_dataset
(#15769) - Handle null index correctly for list take (#15737)
- Preserve lexical ordering on concat (#15753)
- Remove incorrect unsafe pointer cast for int -> enum (#15740)
- pass series name to apply for cut/qcut (#15715)
- count of null column shouldn't panic in agg context (#15710)
- manual cache (#15711)
- Ensure we don't hold onto Mutex when grabbing join tuples (#15704)
- allow null dtypes in UDFs if they match the schema (#15699)
- Respect join_null argument for semi/anti joins (#15696)
- Ensure we don't hold RwLock when spawning group parallelism in w… (#15697)
- Ensure empty with_columns is a no-op (#15694)
- Include predicate in cache state union (#15693)
- Add the missing feature flag for
ewm_mean_by
(#15687) - 8/16-bits int could also apply in place for log expr (#15680)
prepare_expression_for_context
shouldn't panic if exceptions raised from optimizer (#15681)
📖 Documentation
- Add docstring examples for datetimes (#13161) (#15804)
- Fix a typo in categorical section of the user guide (#15777)
- Fix a docstring mistake for DataType.is_float (#15773)
- Remove incorrect "1i (1 index count)" from some docs methods (#15750)
- Add example for
Config.set_tbl_width_chars
(#15566) - Align docstring phrasing in
Series/Expr.dt.truncate/round
(#15698) - Various deprecation docstring improvements (#15648)
🛠️ Other improvements
- Always expand horizontal_any/all (#15816)
- Rename decimal_float to decimal_comma (#15817)
- Split coverage calculation (#15780)
- Update readme (#15787)
- Start at using new Bound<> API from PyO3 (#15752)
- Make
json_path_match
expr non-anonymous (#15682)
Thank you to all our contributors for making this release possible!
@MarcoGorelli, @NedJWestern, @Robinsane, @TobiasDummschat, @alexander-beedie, @c-peters, @dependabot, @dependabot[bot], @gasmith, @henryharbeck, @itamarst, @kszlim, @mbuhidar, @nameexhaustion, @orlp, @reswqa, @ritchie46, @stinodego and @wsyxbcl
Rust Polars 0.39.2
🚀 Performance improvements
- use two binary searches for equality mask when data is sorted (#15702)
- improve filter parallelism (#15686)
✨ Enhancements
- Remove outdated join validation checks (#15701)
🐞 Bug fixes
- manual cache (#15711)
- Ensure we don't hold onto Mutex when grabbing join tuples (#15704)
- allow null dtypes in UDFs if they match the schema (#15699)
- Respect join_null argument for semi/anti joins (#15696)
- Ensure we don't hold RwLock when spawning group parallelism in w… (#15697)
- Ensure empty with_columns is a no-op (#15694)
- Include predicate in cache state union (#15693)
- Add the missing feature flag for
ewm_mean_by
(#15687) - 8/16-bits int could also apply in place for log expr (#15680)
prepare_expression_for_context
shouldn't panic if exceptions raised from optimizer (#15681)
🛠️ Other improvements
- Make
json_path_match
expr non-anonymous (#15682)
Thank you to all our contributors for making this release possible!
@henryharbeck, @kszlim, @orlp, @reswqa and @ritchie46
Python Polars 0.20.22-rc.1
🚀 Performance improvements
- improve filter parallelism (#15686)
🐞 Bug fixes
- Add the missing feature flag for
ewm_mean_by
(#15687) - 8/16-bits int could also apply in place for log expr (#15680)
prepare_expression_for_context
shouldn't panic if exceptions raised from optimizer (#15681)
📖 Documentation
- Various deprecation docstring improvements (#15648)
🛠️ Other improvements
- Make
json_path_match
expr non-anonymous (#15682)
Thank you to all our contributors for making this release possible!
@henryharbeck, @reswqa and @ritchie46
Rust Polars 0.39.1
🚀 Performance improvements
- Fix regression that led to using only a single thread (#15667)
✨ Enhancements
- add ewm_mean_by (#15638)
🐞 Bug fixes
- Ensure profile of simple-projection only take own runtime (#15671)
- Panic if invalid array in object (#15664)
- Ensure 'CachedSchema' doesn't get synced between plans (#15661)
group_by
multiple null columns produce phantom row (#15659)- rolling_* aggs were behaving as if they return scalars in group-by (#15657)
- Correct the unsoundness slice range of
arr.min/max
(#15654) list.mean
fast path shouldn't produce NaN (#15652)- Fix Display implementation of Duration (#15647)
📖 Documentation
🛠️ Other improvements
- Remove the remaining usage of deprecated
numpy
crate APIs (#15668) - make Duration.is_constant_duration less strict for non-timezone-aware case (#15639)
- Fix some typos in comments (#15665)
- remove unnecessary unsafe in list mean/sum (#15660)
Thank you to all our contributors for making this release possible!
@MarcoGorelli, @Priyansh4444, @StevenMia, @itamarst, @mcrumiller, @orlp, @reswqa, @ritchie46 and @stinodego
Python Polars 0.20.21
🚀 Performance improvements
- Fix regression that led to using only a single thread (#15667)
✨ Enhancements
- add ewm_mean_by (#15638)
🐞 Bug fixes
- Ensure profile of simple-projection only take own runtime (#15671)
- Panic if invalid array in object (#15664)
- Ensure 'CachedSchema' doesn't get synced between plans (#15661)
group_by
multiple null columns produce phantom row (#15659)- rolling_* aggs were behaving as if they return scalars in group-by (#15657)
- Correct the unsoundness slice range of
arr.min/max
(#15654) list.mean
fast path shouldn't produce NaN (#15652)
📖 Documentation
- Add missing deprecation warning to
DataFrame.replace
(#15612) - Fix typo in legacy install instructions (#15662)
🛠️ Other improvements
- make Duration.is_constant_duration less strict for non-timezone-aware case (#15639)
- Fix some typos in comments (#15665)
- remove unnecessary unsafe in list mean/sum (#15660)
- fixup failing test due to
offset
deprecation inupsample
(#15636)
Thank you to all our contributors for making this release possible!
@MarcoGorelli, @Priyansh4444, @StevenMia, @eitsupi, @itamarst, @mcrumiller, @orlp, @reswqa, @ritchie46 and @stinodego
Rust Polars 0.39.0
🏆 Highlights
- Full plan CSE (#15264)
💥 Breaking changes
- rename
memmap
->memory_map
as like Python (#15642) - pref(rust!, python): Unify
sort
withSortOptions
andSortMultipleOptions
(#15590) - Update the argument name from
dims
todimensions
inreshape
(#15561) - Allow specifying Hive schema in
read/scan_parquet
(#15434) - Raise error when
schema_overrides
contains nonexistent column name (#15290) - Rename
Chunk
toRecordBatch
(#15298) - Refactor AnyValue supertype logic (#15280)
- Rename
group_by_rolling
torolling
and improve related error messages (#14765) - Rename
ChunkedArray.try_apply
totry_apply_values
(#14947) - Rename parameter
by
togroup_by
inDataFrame.upsample/group_by_dynamic/rolling
(#14840)
🚀 Performance improvements
- Fix cross join batch size when one of the DataFrames is tiny (#14347)
- Fix binview growable complexity O(n*m) -> O(n) (#15628)
- Remove extra thread spawn from row group fetcher (#15626)
- Use vertical parallelism if input is chunked for
Filter
,Select
,WithColumns
(#15608) - read_ipc memory usage tests, and writing fix (#15599)
- Refactor CSV serialization to not go thorough
AnyValue
(#15576) - don't use dynamic dispatch in visitors (#15607)
- Improve Bitmap construction performance (#15570)
- join by row-encoding (#15559)
- Replace std::thread spawn with tokio block_in_place (#15517)
- speed up offset_by when a single offset is passed (#15493)
- Avoid allocation in the hot path for struct JSON serialization (#15449)
- avoid double-allocation in rolling_apply_agg_window (#15423)
- Make LogicalPlan immutable (#15416)
- Add non-order preserving variable row-encoding (#15414)
- Use row-encoding for multiple key group by (#15392)
- load bits one word at a time for BitmapIter (#15333)
- Ipc exec multiple paths (#15040)
- add SIMD support for if-then-else kernels (#15131)
✨ Enhancements
- add Expr.dt.add_business_days and Series.dt.add_business_days (#15595)
- Add
str.head
andstr.tail
(#14425) - Extended
BytecodeParser
to handle additional math functions, and imports from the global namespace (#15627) - Push down
is_between
expressions to Arrow (#15180) - add holidays argument to business_day_count (#15580)
- change default to write parquet statistics (#15597)
- Expressify
to_integer
(#15604) - Optimizer; remove double SORT and redundant projections (#15573)
- Add
null_on_oob
parameter toexpr.array.get
(#15426) - support weekend argument in business_day_count (#15544)
- Enable
is_first/last_distinct
for not nested non-numeric list (#15552) - Turn off cse if cache node found (#15554)
- Tag concat list as elementwise (#15545)
- Support list group-by of non numeric lists (#15540)
- add business_day_count function (#15512)
- Add SQL support for
MEDIAN
aggfunc (#15519) - Implement
string
,boolean
andbinary
dtype intop_k
(#15488) - Add SQL support for
TRUNCATE TABLE
command (#15513) - Add SQL support for
GREATEST
andLEAST
(#15511) - Allow specifying Hive schema in
read/scan_parquet
(#15434) - Implements
agg_list
forNullChunked
(#15439) - Supports
explode_by_offsets
for decimal (#15417) - Add
null_on_oob
parameter toexpr.list.get
(#15395) - CSV-writer escape carriage return (#15399)
- Remove 'FileCacher' optimization (#15357)
- check input type in entropy (#15351)
- Implements
arr.n_unique
(#15296) - CSE don't scan share if predicate pushdown predicates don't match (#15328)
- Remove cached nodes when finished (#15310)
- Full plan CSE (#15264)
- Add IR for expressions. (#15168)
- Warn if
map_elements
is called withoutreturn_dtype
specified (#15188) - Rename
group_by_rolling
torolling
and improve related error messages (#14765) - Rename
ChunkedArray.try_apply
totry_apply_values
(#14947) - Implement strict AnyValue construction for temporal types (#15146)
🐞 Bug fixes
- Return appropriate data type for time
mean
andmedian
(#14471) - Support index upsampling (#13621)
- Fix issue in
write_excel
that could lead to incorrect spanning range determination (#15631) - Output correct dtype for
mean_horizontal
on a single column (#15118) - Recompute RowIndex schema after projection pd (#15625)
- Mean of boolean in streaming group_by incorrectly always gave NULL (#15616)
- Include cloud creds in cache key (#15609)
- Fix elementwise-apply if any input is
AggregatedScalar
(#15606) - Explode list should take validity into account (#15572)
- use larger recursive stack in debug mode (#15593)
- SQL interface "off-by-one' indexing error with
GROUP BY
clauses that use position ordinals (#15584) - Enable missing features in polars-time (#15558)
- Handle quoted identifiers when registering CTEs in the SQL engine (#15564)
- Decompress moved out of schema initialization (#15550)
- Turn off cse if cache node found (#15554)
- Resolve function names and prune all aliases. (#15522)
list.get
should take validity into account (#15516)- block decimal in streaming (#15520)
group_by
partitioned with literalSeries
panic (#15487)- Initialize validity for
GroupsProxy::Slice
windows (#15509) - Fix struct name resolving (#15507)
pow
return type evaluation (#15506)- Allow selectors inside frame-level
.filter()
(#15445) - Don't prune alias in AnonymousFunction subtree (#15453)
- Fix deadlock in async parquet scan (#15440)
- datetime operations (e.g. .dt.year) were raising when null values were backed by out-of-range integers (#15420)
- Ensure Binary -> Binview cast doesn't overflow the buffer size (#15408)
- Don't prune alias in function subtree (#15406)
- Return 0 for
n_unique()
in group-by context when group is empty (#15289) - Unset UpdateGroups after group-sensitive expression (#15400)
to_any_value
should supports all LiteralValue type (#15387)- Hash failure combining hash of two numeric columns containing equal values (#15397)
- Add FixedSizeBinary to arrow field conversion (#15389)
- Conversion of expr_ir in partition fast path (#15388)
sort
for series with unsupported dtype should raise instead of panic (#15385)- Return correct dtype for
s.clear()
when dtype isObject
(#15315) - ensure first datapoint is always included in group_by_dynamic (#15312)
- Non-exhaustive patterns: arrow-schema::DataType in polars-arrow (#15250)
- use dynamic stacks for problematic recursive functions (#15355)
- Raise error when
schema_overrides
contains nonexistent column name (#15290) - Fix cache dot visualization (#15311)
- Properly propagate
strict
flag when constructing a Struct Series from any values (#15302) - ensure
eq
forBinaryViewArray
checks all elements (#15268) - Raise when join projects name with suffix that doesn't exist (#15256)
- fix kurtosis/skew (#15137)
- Ensure ooc_start is set (#15255)
- Fix bug where rolling operations were ignoring
check_sorted
in some cases (#15227) - Fix lazy schema for
rle
expression (#15248) - incorrect negative offset in multi-byte string slicing (#15140)
- do not clamp negative offsets to start of array prematurely (#15242)
- allow null index in list.get and array.get (#15239)
- properly support nulls_last + descending (#15212)
- Block rounding/truncating to negative durations (#15175)
- Make parse_url work on windows with object_store (#15191)
- divide by zero in download speed computation (#15182)
📖 Documentation
- Add legacy CPU install instructions in user guide (#13676)
- Various minor updates to User Guide's SQL intro section (#15557)
- Add
outer_coalesce
join strategy in the user guide (#15405) - Improve docs for
Series::new
withAnyValue
input (#15306) - Fix formatting in
Series::from_any_values_and_dtype
docs (#15244) - Correct the definition of an expression in the user guide (#14750)
📦 Build system
🛠️ Other improvements
- rename
memmap
->memory_map
as like Python (#15642) - fixup failing test due to
offset
deprecation inupsample
(#15636) - use bound api (#15630)
- Don't run streaming group-by in partitionable gb (#15611)
- pref(rust!, python): Unify
sort
withSortOptions
andSortMultipleOptions
(#15590) - remove try_binary_elementwise_values (#15592)
- remove raw pointers from visitors. (#15579)
- rename to IR (#15571)
- Update the argument name from
dims
todimensions
inreshape
(#15561) - Rename ALogicalPlan to FullAccessIR (#15553)
- Set up CodSpeed (#15537)
- make dsl immutable and cheap to clone (#15394)
- use recursive crate, add missing recursive tag (#15393)
- Update CODEOWNERS (polars-sql) (#15384)
- Update Rust toolchain (#15353)
- Update CODEOWNERS (#15352)
- remove try_apply_values (#15336)
- always use non-legacy float_sum for mean (#15343)
- remove legacy bitmap module (#15335)
- More clippy in Makefile (#15340)
- Rename
Cache[count]
toCache[cache_hits]
(#15300) - Cleanup file_caching optimization call (#15299)
- Rename
Chunk
toRecordBatch
(#15298) - Refactor AnyValue supertype logic (#15280)
- reuse message parsing in IPC (#15265)
- remove 'fast-projection' node (#15253)
- cleanup column names in optimizer (#15252)
- remove left_most_input_name from expr ir (#15251)
- add AlignedBitmapSlice (#15171)
- Refactor AnyValue construction for Categorical/Enum dtype (#15220)
- Move ConsecutiveCountState into support module (#15186)
- Run non-benchmark tests in benchmark workflow (#15207)
- Add
wrapping_abs
to arithmetic kernel (#15210) - remove raw buffers from BinViewArray (#15206)
- Enable
RUST_BACKTRACE=1
in the CI test suite (#15204) - Rename parameter
by
togroup_by
inDataFrame.upsample/group_by_dynamic/rolling
(#14840) - Set dual license for
polars-arrow
andpolars-parquet
(#15173)
...
Python Polars 0.20.20
🚀 Performance improvements
- Fix cross join batch size when one of the DataFrames is tiny (#14347)
- Fix binview growable complexity O(n*m) -> O(n) (#15628)
- Remove extra thread spawn from row group fetcher (#15626)
- Use vertical parallelism if input is chunked for
Filter
,Select
,WithColumns
(#15608) - Refactor CSV serialization to not go thorough
AnyValue
(#15576) - don't use dynamic dispatch in visitors (#15607)
- Improve Bitmap construction performance (#15570)
- join by row-encoding (#15559)
✨ Enhancements
- add Expr.dt.add_business_days and Series.dt.add_business_days (#15595)
- Add
str.head
andstr.tail
(#14425) - Add
union
/or
operator forpl.Enum
(#14965) - Extended
BytecodeParser
to handle additional math functions, and imports from the global namespace (#15627) - Push down
is_between
expressions to Arrow (#15180) - add holidays argument to business_day_count (#15580)
- change default to write parquet statistics (#15597)
- Expressify
to_integer
(#15604) - Optimizer; remove double SORT and redundant projections (#15573)
- Add
null_on_oob
parameter toexpr.array.get
(#15426) - support weekend argument in business_day_count (#15544)
- Enable
is_first/last_distinct
for not nested non-numeric list (#15552) - Turn off cse if cache node found (#15554)
- Tag concat list as elementwise (#15545)
🐞 Bug fixes
- Return appropriate data type for time
mean
andmedian
(#14471) - Fix issue in
write_excel
that could lead to incorrect spanning range determination (#15631) - Output correct dtype for
mean_horizontal
on a single column (#15118) - Recompute RowIndex schema after projection pd (#15625)
- Mean of boolean in streaming group_by incorrectly always gave NULL (#15616)
- Include cloud creds in cache key (#15609)
- Fix elementwise-apply if any input is
AggregatedScalar
(#15606) - Explode list should take validity into account (#15572)
- use larger recursive stack in debug mode (#15593)
- SQL interface "off-by-one' indexing error with
GROUP BY
clauses that use position ordinals (#15584) - Enable missing features in polars-time (#15558)
- Handle quoted identifiers when registering CTEs in the SQL engine (#15564)
- Decompress moved out of schema initialization (#15550)
- Turn off cse if cache node found (#15554)
📖 Documentation
- Add legacy CPU install instructions in user guide (#13676)
- Examples for errors (#13724)
- Add docstring examples for reading json (#14481)
- Add security warning in LazyFrame.deserialize() docstring (#15282)
- Various minor updates to User Guide's SQL intro section (#15557)
🛠️ Other improvements
- Replace most deprecated calls with bounded version (#15632)
- use bound api (#15630)
- Initial PyO3 0.21 support (#15622)
- Don't run streaming group-by in partitionable gb (#15611)
- pref(rust!, python): Unify
sort
withSortOptions
andSortMultipleOptions
(#15590) - Set up CodSpeed (#15537)
Thank you to all our contributors for making this release possible!
@CanglongCl, @ChayimFriedman2, @Fokko, @JamesCE2001, @MarcoGorelli, @NedJWestern, @TrevorWinstral, @alexander-beedie, @deanm0000, @douglas-raillard-arm, @eitsupi, @filabrazilska, @i-aki-y, @itamarst, @leoforney, @mcrumiller, @nameexhaustion, @orlp, @ozgrakkurt, @reswqa, @ritchie46 and @stinodego
Python Polars 0.20.19
🚀 Performance improvements
- Replace std::thread spawn with tokio block_in_place (#15517)
- speed up offset_by when a single offset is passed (#15493)
- Avoid allocation in the hot path for struct JSON serialization (#15449)
✨ Enhancements
- Support list group-by of non numeric lists (#15540)
- add business_day_count function (#15512)
- Add SQL support for
MEDIAN
aggfunc (#15519) - Implement
string
,boolean
andbinary
dtype intop_k
(#15488) - Add SQL support for
TRUNCATE TABLE
command (#15513) - Add SQL support for
GREATEST
andLEAST
(#15511) - Allow specifying Hive schema in
read/scan_parquet
(#15434) - Implements
agg_list
forNullChunked
(#15439)
🐞 Bug fixes
- dot product of two integer series is cast to float (#15502)
- Resolve function names and prune all aliases. (#15522)
- Pass
skip_rows_after_header
to pyarrow csv reader (#15533) - No longer error when
schema_overrides
contains nonexistent columns (#15528) list.get
should take validity into account (#15516)- block decimal in streaming (#15520)
group_by
partitioned with literalSeries
panic (#15487)- Initialize validity for
GroupsProxy::Slice
windows (#15509) - Fix struct name resolving (#15507)
pow
return type evaluation (#15506)- Address issue with
read_database
draining iter_batches early (#15504) - Allow selectors inside frame-level
.filter()
(#15445) - Don't prune alias in AnonymousFunction subtree (#15453)
- Raise if pass a negative
n
intoclear
(#15432) - Fix deadlock in async parquet scan (#15440)
📖 Documentation
- Update leftover references of
by
parameter togroup_by
inDataFrame/LazyFrame.upsample/group_by_dynamic/rolling
(#15527) - Add
make docs
command, DataType docs/layout tweak, minor README updates (#15386) - Add example for
Series.list.median
. (#15451)
🛠️ Other improvements
- Remove unused code paths in
read_parquet
(#15532) - Organize utils for I/O functionality (#15529)
- Remove private
DataFrame._read
classmethods (#15521) - Move dedicated inference code out of
io.database
executor module (#15526) - Add unstable warning to
hive_schema
functionality (#15508)
Thank you to all our contributors for making this release possible!
@CanglongCl, @ChayimFriedman2, @MarcoGorelli, @alexander-beedie, @cmdlineluser, @dependabot, @dependabot[bot], @henryharbeck, @mbuhidar, @nameexhaustion, @reswqa, @ritchie46, @rob-sil and @stinodego
Python Polars 0.20.18
🚀 Performance improvements
- CSV reading memory usage tests and fixes (#15422)
- avoid double-allocation in rolling_apply_agg_window (#15423)
- Make LogicalPlan immutable (#15416)
- Add non-order preserving variable row-encoding (#15414)
- Use row-encoding for multiple key group by (#15392)
✨ Enhancements
- Supports
explode_by_offsets
for decimal (#15417) - Add
read_clipboard
andDataFrame.write_clipboard
(#15272) - Add
null_on_oob
parameter toexpr.list.get
(#15395) - make Series.__bool__ error message Rusttier (#15407)
- CSV-writer escape carriage return (#15399)
🐞 Bug fixes
- datetime operations (e.g. .dt.year) were raising when null values were backed by out-of-range integers (#15420)
- Ensure Binary -> Binview cast doesn't overflow the buffer size (#15408)
- Don't prune alias in function subtree (#15406)
- Return 0 for
n_unique()
in group-by context when group is empty (#15289) - Unset UpdateGroups after group-sensitive expression (#15400)
to_any_value
should supports all LiteralValue type (#15387)- Hash failure combining hash of two numeric columns containing equal values (#15397)
- Add FixedSizeBinary to arrow field conversion (#15389)
- Conversion of expr_ir in partition fast path (#15388)
- fix panic when doing a scan_parquet with hive partioning (#15381)
sort
for series with unsupported dtype should raise instead of panic (#15385)
📖 Documentation
- Added example for
explode
mapping strategy inpl.Expr.over
(#15402) - Add
outer_coalesce
join strategy in the user guide (#15405) - Change the example to series for
series/array.py
(#15383) - Add "See Also" for
arg_sort
andarg_sort_by
(#15348)
🛠️ Other improvements
- make dsl immutable and cheap to clone (#15394)
- use recursive crate, add missing recursive tag (#15393)
- Update CODEOWNERS (polars-sql) (#15384)
Thank you to all our contributors for making this release possible!
@CanglongCl, @JamesCE2001, @MarcoGorelli, @Sol-Hee, @alexander-beedie, @dependabot, @dependabot[bot], @itamarst, @kszlim, @mcrumiller, @nameexhaustion, @orlp, @reswqa, @ritchie46, @rob-sil and @thomaslin2020