Skip to content

Commit

Permalink
release: 0.1.0 (#35)
Browse files Browse the repository at this point in the history
* release: 0.1.0

* tests: skip on cran

* docs: update readme

* chore: ignore cran-submission

* fix: license file

* fix: license

* fix: cran

* chore: global env
  • Loading branch information
be-marc authored Jun 21, 2024
1 parent 3dee48b commit 119ef21
Show file tree
Hide file tree
Showing 37 changed files with 104 additions and 35 deletions.
2 changes: 1 addition & 1 deletion .Rbuildignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
^README\.Rmd$
benchmark/
^LICENSE$
.ignore
.editorconfig
.gitignore
Expand All @@ -23,3 +22,4 @@ benchmark/
^CRAN-SUBMISSION$
^man/figures$
.vscode
^LICENSE\.md$
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -99,3 +99,4 @@ rsconnect/
attic/
cran-comments.md
README.html
CRAN-SUBMISSION
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
Package: rush
Title: Rapid Parallel and Distributed Computing
Version: 0.0.0.9000
Version: 0.1.0
Authors@R:
person("Marc", "Becker", , "[email protected]", role = c("cre", "aut", "cph"),
comment = c(ORCID = "0000-0002-8115-0400"))
Description: Parallel computing with a network of local and remote
workers. Fast exchange of results between the workers through a
'Redis' database. Key features include task queues, local caching, and
sophisticated error handling.
License: LGPL-3
URL: https://github.com/mlr-org/rush
BugReports: https://github.com/mlr-org/rush/issues
Depends:
Expand All @@ -33,3 +32,4 @@ Config/testthat/parallel: false
Encoding: UTF-8
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.3.1
License: MIT + file LICENSE
23 changes: 2 additions & 21 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,21 +1,2 @@
MIT License

Copyright (c) 2024 mlr-org

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
YEAR: 2024
COPYRIGHT HOLDER: rush authors
21 changes: 21 additions & 0 deletions LICENSE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# MIT License

Copyright (c) 2024 rush authors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
1 change: 0 additions & 1 deletion NAMESPACE
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Generated by roxygen2: do not edit by hand

export(AppenderRedis)
export(Rush)
export(RushWorker)
export(assert_rush)
export(assert_rush_worker)
Expand Down
2 changes: 1 addition & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# rush (development version)
# rush 0.1.0

* Initial CRAN submission.
2 changes: 2 additions & 0 deletions R/AppenderRedis.R
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,10 @@
#' @param filters (`list`)\cr
#' List of filters.
#'
#' @return Object of class [R6::R6Class] and `AppenderRedis` with methods for writing log events to Redis data bases.
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#'
Expand Down
5 changes: 3 additions & 2 deletions R/Rush.R
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,9 @@
#' @template param_seed
#' @template param_data_format
#'
#' @export
#' @return Object of class [R6::R6Class] and `Rush` with controller methods.
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#' rush = rsh(network_id = "test_network", config = config_local)
Expand Down Expand Up @@ -560,13 +561,13 @@ Rush = R6::R6Class("Rush",
r$DEL(private$.get_key("terminated_worker_ids"))
r$DEL(private$.get_key("killed_worker_ids"))
r$DEL(private$.get_key("lost_worker_ids"))
r$DEL(private$.get_key("pre_worker_ids"))
r$DEL(private$.get_key("start_args"))
r$DEL(private$.get_key("terminate_on_idle"))
r$DEL(private$.get_key("local_workers"))
r$DEL(private$.get_key("heartbeat_keys"))

# reset counters and caches
# private$.cached_results_dt = data.table()
private$.cached_tasks = list()
private$.n_seen_results = 0

Expand Down
2 changes: 2 additions & 0 deletions R/RushWorker.R
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,10 @@
#' @template param_lgr_buffer_size
#' @template param_seed
#'
#' @return Object of class [R6::R6Class] and `RushWorker` with worker methods.
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#' rush = rsh(network_id = "test_network", config = config_local)
Expand Down
1 change: 1 addition & 0 deletions R/assertions.R
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
#' @name rush_assertions
#' @keywords internal
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#' rush = rsh(network_id = "test_network", config = config_local)
Expand Down
1 change: 1 addition & 0 deletions R/heartbeat_loops.R
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
#' @keywords internal
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#'
Expand Down
2 changes: 2 additions & 0 deletions R/helper.R
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ get_hostname = function() {
#'
#' @description
#' This function sets the rng state before running a function.
#' Use with caution.
#' The global environment is changed.
#'
#' @param fun (`function`)\cr
#' Function to run.
Expand Down
5 changes: 5 additions & 0 deletions R/rush_plan.R
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
#' @return `list()` with the stored configuration.
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#' rush_plan(config = config_local, n_workers = 2)
Expand Down Expand Up @@ -59,6 +60,7 @@ rush_plan = function(
#'
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#' rush_plan(config = config_local, n_workers = 2)
Expand All @@ -78,8 +80,10 @@ rush_config = function() {
#' @description
#' Removes the rush plan that was set by [rush_plan()].
#'
#' @return Invisible `TRUE`. Function called for side effects.
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#' rush_plan(config = config_local, n_workers = 2)
Expand All @@ -99,6 +103,7 @@ remove_rush_plan = function() {
#'
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#' rush_plan(config = config_local, n_workers = 2)
Expand Down
6 changes: 5 additions & 1 deletion R/start_worker.R
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
#' Starts a worker.
#' The function loads the globals and packages, initializes the [RushWorker] instance and invokes the worker loop.
#' This function is called by `$start_local_workers()` or by the user after creating the worker script with `$create_worker_script()`.
#' Use with caution.
#' The global environment is changed.
#'
#' @note
#' The function initializes the connection to the Redis data base.
Expand All @@ -21,6 +23,7 @@
#' @return `NULL`
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \dontrun{
#' rush::start_worker(
#' network_id = 'test-rush',
Expand Down Expand Up @@ -66,8 +69,9 @@ start_worker = function(
}

# load packages and globals to worker environment
envir = .GlobalEnv
mlr3misc::walk(start_args$packages, function(package) library(package, character.only = TRUE))
mlr3misc::iwalk(start_args$globals, function(value, name) assign(name, value, .GlobalEnv))
mlr3misc::iwalk(start_args$globals, function(value, name) assign(name, value, envir))

# initialize rush worker
rush = invoke(rush::RushWorker$new,
Expand Down
1 change: 1 addition & 0 deletions R/sugar.R
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
#' @return [Rush] controller.
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#' rush = rsh(network_id = "test_network", config = config_local)
Expand Down
2 changes: 2 additions & 0 deletions R/worker_loops.R
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
#' @return `NULL`
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#' rush = rsh(network_id = "test_network", config = config_local)
Expand Down Expand Up @@ -63,6 +64,7 @@ worker_loop_default = function(fun, constants = NULL, rush) {
#' @return `NULL`
#' @export
#' @examples
#' # This example is not executed since Redis must be installed
#' \donttest{
#' config_local = redux::redis_config()
#' rush = rsh(network_id = "test_network", config = config_local)
Expand Down
15 changes: 10 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,12 @@ We start two workers.
rush$start_local_workers(fun = fun, n_workers = 2)
```

## INFO [16:06:19.324] [rush] Starting 2 worker(s)
## INFO [17:30:43.422] [rush] Starting 2 worker(s)
## $`d82afa87-9f2c-41a4-891c-7490ea5eba46`
## PROCESS 'Rscript', running, pid 24531.
##
## $`fad19573-bfe6-4809-8be2-bdde44191210`
## PROCESS 'Rscript', running, pid 24542.

Now we can push tasks to the workers.

Expand All @@ -100,10 +105,10 @@ And retrieve the results.
rush$fetch_finished_tasks()
```

## x1 x2 y pid worker_id keys
## <num> <num> <num> <int> <char> <char>
## 1: 4 6 10 448184 ebb0d9b4-3... 37ba4683-5...
## 2: 3 5 8 448173 f49aad09-a... 27b4b03b-d...
## x1 x2 y pid worker_id keys
## <num> <num> <num> <int> <char> <char>
## 1: 3 5 8 24542 fad19573-b... 7ac77ca4-6...
## 2: 4 6 10 24531 d82afa87-9... 0fb60722-6...

## Decentralized Rush Network

Expand Down
4 changes: 4 additions & 0 deletions man/AppenderRedis.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions man/Rush.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions man/RushWorker.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Binary file modified man/figures/README-flow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions man/heartbeat.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions man/remove_rush_plan.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/rsh.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions man/rush-package.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/rush_assertions.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/rush_available.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 119ef21

Please sign in to comment.