Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add writeStream #480

Merged
merged 3 commits into from
Jan 8, 2021
Merged

Add writeStream #480

merged 3 commits into from
Jan 8, 2021

Conversation

etspaceman
Copy link
Contributor

@etspaceman etspaceman commented Dec 23, 2020

Solves #232 partially by adding writeStream

@etspaceman
Copy link
Contributor Author

This still needs a test for withWatermark, I will look into adding that in the near future.

@codecov
Copy link

codecov bot commented Dec 23, 2020

Codecov Report

Merging #480 (01f389d) into master (854d5b6) will increase coverage by 0.27%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #480      +/-   ##
==========================================
+ Coverage   96.23%   96.51%   +0.27%     
==========================================
  Files          60       60              
  Lines        1036     1119      +83     
  Branches        7        8       +1     
==========================================
+ Hits          997     1080      +83     
  Misses         39       39              
Flag Coverage Δ
2.12.12 96.51% <100.00%> (+0.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...c/main/scala/frameless/TypedDatasetForwarded.scala 74.28% <100.00%> (+0.75%) ⬆️
dataset/src/main/scala/frameless/TypedColumn.scala 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 854d5b6...01f389d. Read the comment docs.

@imarios
Copy link
Contributor

imarios commented Dec 23, 2020

Thank you for the help @etspaceman! The build I think is failing because of low coverage. It should get green as soon as we add the missing unit tests. I will try to look into the PR today else it may have to wait until after Christmas.

@etspaceman
Copy link
Contributor Author

No rush! I'll try to get that test in tonight. I'm not quite sure how I'll test it yet but I'll come up with something :D

@etspaceman etspaceman changed the title Add writeStream, withWatermark Add writeStream Dec 28, 2020
@etspaceman
Copy link
Contributor Author

etspaceman commented Dec 28, 2020

@imarios I was trying to add tests for withWatermark but this is not really possible without the window functions. I think that can be in another PR. So I removed that from this PR

@imarios imarios closed this Jan 8, 2021
@imarios imarios reopened this Jan 8, 2021
@imarios
Copy link
Contributor

imarios commented Jan 8, 2021

@etspaceman looks great! I am running the tests one more time and merging.

@imarios imarios merged commit 45764e8 into typelevel:master Jan 8, 2021
@etspaceman etspaceman deleted the structuredStreaming branch July 14, 2023 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants