Merge cascading3 to develop #1586
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Conflicts: CHANGES.md README.md project/Build.scala scalding-core/src/main/scala/com/twitter/package.scala tutorial/execution-tutorial/ExecutionTutorial.scala version.sbt
Remove dependency on parquet-cascading
Update the build
…r internally. This will be updated as part of the eventual merge to develop.
…ojects [cascading3] Move parquet cascading schemes to subprojects
[cascading3] Migrate parquet schemes
[cascading3] Migrate maple
Hadoop's -libjars doesn't support wildcards, with large class paths its easy to exhaust the max arg length for linux/os x when running commands. This acts as a filter above our interaction with the generic options parser to expand wildcards
|
|
|
Just signed it. Glad to see this being merged! |
|
Just FYI others, you don't need to sign this if you were a twitter employee when you did the work (which I think is true). |
use https for twttr maven
Bump Cascading version to 3.2.1
| @@ -16,13 +16,13 @@ | |||
| * specific language governing permissions and limitations | |||
| * under the License. | |||
| */ | |||
| package com.twitter.scalding.parquet.scrooge; | |||
| package com.twitter.scalding.parquet.cascading.scrooge; | |||
There was a problem hiding this comment.
@piyushnarang looks like the package name does not match the file name. not scrooge/cascading, but we have cascading.scrooge.
There was a problem hiding this comment.
I can push a follow up fix for this tomorrow.
Update packages in scalding-parquet-scrooge cascading to match files
* Remove a quadradic function in our cascading3 support * fix a bug
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a single merge consisting of the piecemeal changes listed in #1465
For testing, I've published
0.16.1-cascading3-RC2to sonatype.@johnynek @ianoc would be great if this can be integrated/tested at Stripe before we merge to develop. Thanks!
Note that this does not include the HashJoin DistCacheTap change from #1585, which can be treated as an add-on change and merged subsequently. We do not have any pre-existing usage outside of Twitter afaik (the corresponding 2.5 fix was made in Twitter's Cascading fork).
cc @piyushnarang @isnotinvain