refactor: port the internals to use koerce
#10078
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kszucs
force-pushed
the
koerce
branch
4 times, most recently
from
b8afaef
to
f07026d
Compare
|
Benchmarking against Benchmarks: |
|
|
|
Makes sense. It seems like that might not be representative of an actual workload though. |
|
In any case a perf improvement + 10k fewer lines of code seems like an great improvement! |
|
Seems like we can delete some of these fully-commented out test modules. |
kszucs
commented
Sep 11, 2024
| return self.__argnames__ | ||
|
|
||
|
|
||
| ValidationError = SignatureValidationError = (MatchError, ValueError, TypeError) |
There was a problem hiding this comment.
Koerce should introduce a proper error type, that needs to be updated.
kszucs
commented
Sep 11, 2024
| @@ -463,15 +462,15 @@ def is_variadic(self) -> bool: | |||
|
|
|||
|
|
|||
| @public | |||
| class Unknown(DataType, Singleton): | |||
There was a problem hiding this comment.
koerce.Annotable now supports singleton, so this can be restored by using class Unknown(DataType, singleton=True)
kszucs
commented
Sep 11, 2024
| @@ -2483,7 +2482,7 @@ class NullValue(Value): | |||
|
|
|||
|
|
|||
| @public | |||
| class NullScalar(Scalar, NullValue, Singleton): | |||
| class NullScalar(Scalar, NullValue): | |||
There was a problem hiding this comment.
Expr is not a subclass of Annotable so singleton=True cannot be used here, although we could turn Expr into Annotable as well, should benchmark the impact of that.
kszucs
force-pushed
the
koerce
branch
2 times, most recently
from
869aa3b
to
665c5f4
Compare
kszucs
commented
Sep 11, 2024
| @@ -4300,7 +4302,8 @@ def pivot_wider( | |||
| rules = ( | |||
| # add in the where clause to filter the appropriate values | |||
| p.Reduction(where=None) >> _.copy(where=where) | |||
| | p.Reduction(where=x) >> _.copy(where=where & x) | |||
| # TODO(kszucs) | |||
| # | p.Reduction(where=+x) >> _.copy(where=where & x) | |||
There was a problem hiding this comment.
I am not sure that we actually need this.
github-actions
bot
added
tests
|
Thinking about this some more, I think we want to keep the pattern system inside Ibis for now. While the performance gains are nice, the development overhead of developing two libraries simultaneously is quite high, even if the library remained pure Python. Adding Cython to the mix complicates the picture. cc @jcrist for thoughts |
It can be used as a pure python package as well, at least the intention is not to hurt compatibility. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TBD