Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support concurrent processing like Map/Shuffle/Reduce pattern #10

Open
syucream opened this issue Apr 19, 2020 · 0 comments
Open

Support concurrent processing like Map/Shuffle/Reduce pattern #10

syucream opened this issue Apr 19, 2020 · 0 comments
Labels
enhancement New feature or request

Comments

@syucream
Copy link
Contributor

If we support multiple input files #9 , al least we can support input file based concurrent processing, just sharding Columnifier instance that will help to minimize processing time. And also we can support more flexible output files control if doing shuffle before reduce.

@t2y t2y added the enhancement New feature or request label Apr 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants