furrr/ 0000755 0001762 0000144 00000000000 14065413512 011413 5 ustar ligges users furrr/NAMESPACE 0000644 0001762 0000144 00000003675 14065377245 012661 0 ustar ligges users # Generated by roxygen2: do not edit by hand
S3method(future_modify,character)
S3method(future_modify,default)
S3method(future_modify,double)
S3method(future_modify,integer)
S3method(future_modify,logical)
S3method(future_modify,pairlist)
S3method(future_modify_at,character)
S3method(future_modify_at,default)
S3method(future_modify_at,double)
S3method(future_modify_at,integer)
S3method(future_modify_at,logical)
S3method(future_modify_if,character)
S3method(future_modify_if,default)
S3method(future_modify_if,double)
S3method(future_modify_if,integer)
S3method(future_modify_if,logical)
S3method(print,furrr_options)
export(furrr_options)
export(future_imap)
export(future_imap_chr)
export(future_imap_dbl)
export(future_imap_dfc)
export(future_imap_dfr)
export(future_imap_int)
export(future_imap_lgl)
export(future_imap_raw)
export(future_invoke_map)
export(future_invoke_map_chr)
export(future_invoke_map_dbl)
export(future_invoke_map_dfc)
export(future_invoke_map_dfr)
export(future_invoke_map_int)
export(future_invoke_map_lgl)
export(future_invoke_map_raw)
export(future_iwalk)
export(future_map)
export(future_map2)
export(future_map2_chr)
export(future_map2_dbl)
export(future_map2_dfc)
export(future_map2_dfr)
export(future_map2_int)
export(future_map2_lgl)
export(future_map2_raw)
export(future_map_at)
export(future_map_chr)
export(future_map_dbl)
export(future_map_dfc)
export(future_map_dfr)
export(future_map_if)
export(future_map_int)
export(future_map_lgl)
export(future_map_raw)
export(future_modify)
export(future_modify_at)
export(future_modify_if)
export(future_options)
export(future_pmap)
export(future_pmap_chr)
export(future_pmap_dbl)
export(future_pmap_dfc)
export(future_pmap_dfr)
export(future_pmap_int)
export(future_pmap_lgl)
export(future_pmap_raw)
export(future_pwalk)
export(future_walk)
export(future_walk2)
import(rlang)
importFrom(future,future)
importFrom(parallel,nextRNGStream)
importFrom(parallel,nextRNGSubStream)
importFrom(purrr,map)
furrr/LICENSE 0000644 0001762 0000144 00000000053 13710077201 012413 0 ustar ligges users YEAR: 2020
COPYRIGHT HOLDER: Davis Vaughan
furrr/README.md 0000644 0001762 0000144 00000011367 14065377103 012707 0 ustar ligges users
# furrr
[](https://cran.r-project.org/package=furrr)
[](https://github.com/DavisVaughan/furrr/actions)
[](https://codecov.io/gh/DavisVaughan/furrr?branch=master)
## Overview
The goal of furrr is to combine purrr’s family of mapping functions with
future’s parallel processing capabilities. The result is near drop in
replacements for purrr functions such as `map()` and `map2_dbl()`, which
can be replaced with their furrr equivalents of `future_map()` and
`future_map2_dbl()` to map in parallel.
The code draws heavily from the implementations of purrr and
future.apply and this package would not be possible without either of
them.
## What has been implemented?
Every variant of the following functions has been implemented:
- `map()`
- `map2()`
- `pmap()`
- `walk()`
- `imap()`
- `modify()`
This includes atomic variants like `map_dbl()` through
`future_map_dbl()` and predicate variants like `map_at()` through
`future_map_at()`.
## Installation
You can install the released version of furrr from
[CRAN](https://CRAN.R-project.org) with:
``` r
install.packages("furrr")
```
And the development version from [GitHub](https://github.com/) with:
``` r
# install.packages("remotes")
remotes::install_github("DavisVaughan/furrr")
```
## Learning
The easiest way to learn about furrr is to browse [the
website](https://furrr.futureverse.org/). In particular, the [function
reference](https://furrr.futureverse.org/reference/index.html) page can
be useful to get a general overview of the functions in the package, and
the following vignettes are deep dives into various parts of furrr:
- [Common
gotchas](https://furrr.futureverse.org/articles/articles/gotchas.html)
- [Learn how furrr “chunks” your
input](https://furrr.futureverse.org/articles/articles/chunking.html)
- [carrier - An alternative to automatic globals
detection](https://furrr.futureverse.org/articles/articles/carrier.html)
- [Progress notifications with
progressr](https://furrr.futureverse.org/articles/articles/progress.html)
- [Using furrr with
connections](https://furrr.futureverse.org/articles/articles/remote-connections.html)
## Example
furrr has been designed to function as identically to purrr as possible,
so that you can immediately have familiarity with it.
``` r
library(furrr)
library(purrr)
map(c("hello", "world"), ~.x)
#> [[1]]
#> [1] "hello"
#>
#> [[2]]
#> [1] "world"
future_map(c("hello", "world"), ~.x)
#> [[1]]
#> [1] "hello"
#>
#> [[2]]
#> [1] "world"
```
The default backend for future (and through it, furrr) is a sequential
one. This means that the above code will run out of the box, but it will
*not* be in parallel. The design of future makes it incredibly easy to
change this so that your code will run in parallel.
``` r
# Set a "plan" for how the code should run.
plan(multisession, workers = 2)
# This does run in parallel!
future_map(c("hello", "world"), ~.x)
#> [[1]]
#> [1] "hello"
#>
#> [[2]]
#> [1] "world"
```
If you are still skeptical, here is some proof that we are running in
parallel.
``` r
library(tictoc)
# This should take 6 seconds in total running sequentially
plan(sequential)
tic()
nothingness <- future_map(c(2, 2, 2), ~Sys.sleep(.x))
toc()
#> 6.08 sec elapsed
```
``` r
# This should take ~2 seconds running in parallel, with a little overhead
# in `future_map()` from sending data to the workers. There is generally also
# a one time cost from `plan(multisession)` setting up the workers.
plan(multisession, workers = 3)
tic()
nothingness <- future_map(c(2, 2, 2), ~Sys.sleep(.x))
toc()
#> 2.212 sec elapsed
```
## Data transfer
It’s important to remember that data has to be passed back and forth
between the workers. This means that whatever performance gain you might
have gotten from your parallelization can be crushed by moving large
amounts of data around. For example, if you are moving large data frames
to the workers, running models in parallel, and returning large model
objects back, the shuffling of data can take a large chunk of that time.
Rather than returning the entire model object, you might consider only
returning a performance metric, or smaller specific pieces of that model
that you are most interested in.
This performance drop can especially be prominent if using
`future_pmap()` to iterate over rows and return large objects at each
iteration.
furrr/man/ 0000755 0001762 0000144 00000000000 13744061622 012172 5 ustar ligges users furrr/man/future_map.Rd 0000644 0001762 0000144 00000012115 13744061622 014630 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/future-map.R, R/future-walk.R
\name{future_map}
\alias{future_map}
\alias{future_map_chr}
\alias{future_map_dbl}
\alias{future_map_int}
\alias{future_map_lgl}
\alias{future_map_raw}
\alias{future_map_dfr}
\alias{future_map_dfc}
\alias{future_walk}
\title{Apply a function to each element of a vector via futures}
\usage{
future_map(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_map_chr(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_map_dbl(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_map_int(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_map_lgl(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_map_raw(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_map_dfr(
.x,
.f,
...,
.id = NULL,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_map_dfc(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_walk(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
}
\arguments{
\item{.x}{A list or atomic vector.}
\item{.f}{A function, formula, or vector (not necessarily atomic).
If a \strong{function}, it is used as is.
If a \strong{formula}, e.g. \code{~ .x + 2}, it is converted to a function. There
are three ways to refer to the arguments:
\itemize{
\item For a single argument function, use \code{.}
\item For a two argument function, use \code{.x} and \code{.y}
\item For more arguments, use \code{..1}, \code{..2}, \code{..3} etc
}
This syntax allows you to create very compact anonymous functions.
If \strong{character vector}, \strong{numeric vector}, or \strong{list}, it is
converted to an extractor function. Character vectors index by
name and numeric vectors index by position; use a list to index
by position and name at different levels. If a component is not
present, the value of \code{.default} will be returned.}
\item{...}{Additional arguments passed on to the mapped function.}
\item{.options}{The \code{future} specific options to use with the workers. This
must be the result from a call to \code{\link[=furrr_options]{furrr_options()}}.}
\item{.env_globals}{The environment to look for globals required by \code{.x} and
\code{...}. Globals required by \code{.f} are looked up in the function environment
of \code{.f}.}
\item{.progress}{A single logical. Should a progress bar be displayed?
Only works with multisession, multicore, and multiprocess futures. Note
that if a multicore/multisession future falls back to sequential, then
a progress bar will not be displayed.
\strong{Warning:} The \code{.progress} argument will be deprecated and removed
in a future version of furrr in favor of using the more robust
\href{https://CRAN.R-project.org/package=progressr}{progressr}
package.}
\item{.id}{Either a string or \code{NULL}. If a string, the output will contain
a variable with that name, storing either the name (if \code{.x} is named) or
the index (if \code{.x} is unnamed) of the input. If \code{NULL}, the default, no
variable will be created.
Only applies to \verb{_dfr} variant.}
}
\value{
All functions return a vector the same length as \code{.x}.
\itemize{
\item \code{\link[=future_map]{future_map()}} returns a list
\item \code{\link[=future_map_lgl]{future_map_lgl()}} a logical vector
\item \code{\link[=future_map_int]{future_map_int()}} an integer vector
\item \code{\link[=future_map_dbl]{future_map_dbl()}} a double vector
\item \code{\link[=future_map_chr]{future_map_chr()}} a character vector
}
The output of \code{.f} will be automatically typed upwards, e.g. logical ->
integer -> double -> character.
}
\description{
These functions work exactly the same as \code{\link[purrr:map]{purrr::map()}} and its variants, but
allow you to map in parallel.
}
\examples{
library(magrittr)
\donttest{plan(multisession, workers = 2)}
1:10 \%>\%
future_map(rnorm, n = 10, .options = furrr_options(seed = 123)) \%>\%
future_map_dbl(mean)
# If each element of the output is a data frame, use
# `future_map_dfr()` to row-bind them together:
mtcars \%>\%
split(.$cyl) \%>\%
future_map(~ lm(mpg ~ wt, data = .x)) \%>\%
future_map_dfr(~ as.data.frame(t(as.matrix(coef(.)))))
# You can be explicit about what gets exported to the workers.
# To see this, use multisession (not multicore as the forked workers
# still have access to this environment)
\donttest{plan(multisession)}
x <- 1
y <- 2
# This will fail, y is not exported (no black magic occurs)
try(future_map(1, ~y, .options = furrr_options(globals = "x")))
# y is exported
future_map(1, ~y, .options = furrr_options(globals = "y"))
\dontshow{
# Close open connections for R CMD Check
if (!inherits(plan(), "sequential")) plan(sequential)
}
}
furrr/man/future_invoke_map.Rd 0000644 0001762 0000144 00000007351 14065376117 016216 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/future-invoke-map.R
\name{future_invoke_map}
\alias{future_invoke_map}
\alias{future_invoke_map_chr}
\alias{future_invoke_map_dbl}
\alias{future_invoke_map_int}
\alias{future_invoke_map_lgl}
\alias{future_invoke_map_raw}
\alias{future_invoke_map_dfr}
\alias{future_invoke_map_dfc}
\title{Invoke functions via futures}
\usage{
future_invoke_map(
.f,
.x = list(NULL),
...,
.env = NULL,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_invoke_map_chr(
.f,
.x = list(NULL),
...,
.env = NULL,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_invoke_map_dbl(
.f,
.x = list(NULL),
...,
.env = NULL,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_invoke_map_int(
.f,
.x = list(NULL),
...,
.env = NULL,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_invoke_map_lgl(
.f,
.x = list(NULL),
...,
.env = NULL,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_invoke_map_raw(
.f,
.x = list(NULL),
...,
.env = NULL,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_invoke_map_dfr(
.f,
.x = list(NULL),
...,
.env = NULL,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_invoke_map_dfc(
.f,
.x = list(NULL),
...,
.env = NULL,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
}
\arguments{
\item{.f}{A list of functions.}
\item{.x}{A list of argument-lists the same length as \code{.f} (or length 1). The
default argument, \code{list(NULL)}, will be recycled to the same length as
\code{.f}, and will call each function with no arguments (apart from any
supplied in \code{...}).}
\item{...}{Additional arguments passed to each function.}
\item{.env}{Environment in which \code{\link[=do.call]{do.call()}} should
evaluate a constructed expression. This only matters if you pass
as \code{.f} the name of a function rather than its value, or as
\code{.x} symbols of objects rather than their values.}
\item{.options}{The \code{future} specific options to use with the workers. This
must be the result from a call to \code{\link[=furrr_options]{furrr_options()}}.}
\item{.env_globals}{The environment to look for globals required by \code{.x} and
\code{...}. Globals required by \code{.f} are looked up in the function environment
of \code{.f}.}
\item{.progress}{A single logical. Should a progress bar be displayed?
Only works with multisession, multicore, and multiprocess futures. Note
that if a multicore/multisession future falls back to sequential, then
a progress bar will not be displayed.
\strong{Warning:} The \code{.progress} argument will be deprecated and removed
in a future version of furrr in favor of using the more robust
\href{https://CRAN.R-project.org/package=progressr}{progressr}
package.}
}
\description{
\ifelse{html}{\href{https://lifecycle.r-lib.org/articles/stages.html#deprecated}{\figure{lifecycle-deprecated.svg}{options: alt='[Deprecated]'}}}{\strong{[Deprecated]}}
These functions work exactly the same as \code{\link[purrr:invoke]{purrr::invoke_map()}} functions, but
allow you to invoke in parallel.
}
\examples{
\donttest{plan(multisession, workers = 2)}
df <- dplyr::tibble(
f = c("runif", "rpois", "rnorm"),
params = list(
list(n = 10),
list(n = 5, lambda = 10),
list(n = 10, mean = -3, sd = 10)
)
)
future_invoke_map(df$f, df$params, .options = furrr_options(seed = 123))
\dontshow{
# Close open connections for R CMD Check
if (!inherits(plan(), "sequential")) plan(sequential)
}
}
furrr/man/future_imap.Rd 0000644 0001762 0000144 00000007637 13741124570 015015 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/future-imap.R, R/future-walk.R
\name{future_imap}
\alias{future_imap}
\alias{future_imap_chr}
\alias{future_imap_dbl}
\alias{future_imap_int}
\alias{future_imap_lgl}
\alias{future_imap_raw}
\alias{future_imap_dfr}
\alias{future_imap_dfc}
\alias{future_iwalk}
\title{Apply a function to each element of a vector, and its index via futures}
\usage{
future_imap(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_imap_chr(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_imap_dbl(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_imap_int(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_imap_lgl(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_imap_raw(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_imap_dfr(
.x,
.f,
...,
.id = NULL,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_imap_dfc(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
future_iwalk(
.x,
.f,
...,
.options = furrr_options(),
.env_globals = parent.frame(),
.progress = FALSE
)
}
\arguments{
\item{.x}{A list or atomic vector.}
\item{.f}{A function, formula, or vector (not necessarily atomic).
If a \strong{function}, it is used as is.
If a \strong{formula}, e.g. \code{~ .x + 2}, it is converted to a function. There
are three ways to refer to the arguments:
\itemize{
\item For a single argument function, use \code{.}
\item For a two argument function, use \code{.x} and \code{.y}
\item For more arguments, use \code{..1}, \code{..2}, \code{..3} etc
}
This syntax allows you to create very compact anonymous functions.
If \strong{character vector}, \strong{numeric vector}, or \strong{list}, it is
converted to an extractor function. Character vectors index by
name and numeric vectors index by position; use a list to index
by position and name at different levels. If a component is not
present, the value of \code{.default} will be returned.}
\item{...}{Additional arguments passed on to the mapped function.}
\item{.options}{The \code{future} specific options to use with the workers. This
must be the result from a call to \code{\link[=furrr_options]{furrr_options()}}.}
\item{.env_globals}{The environment to look for globals required by \code{.x} and
\code{...}. Globals required by \code{.f} are looked up in the function environment
of \code{.f}.}
\item{.progress}{A single logical. Should a progress bar be displayed?
Only works with multisession, multicore, and multiprocess futures. Note
that if a multicore/multisession future falls back to sequential, then
a progress bar will not be displayed.
\strong{Warning:} The \code{.progress} argument will be deprecated and removed
in a future version of furrr in favor of using the more robust
\href{https://CRAN.R-project.org/package=progressr}{progressr}
package.}
\item{.id}{Either a string or \code{NULL}. If a string, the output will contain
a variable with that name, storing either the name (if \code{.x} is named) or
the index (if \code{.x} is unnamed) of the input. If \code{NULL}, the default, no
variable will be created.
Only applies to \verb{_dfr} variant.}
}
\value{
A vector the same length as .x.
}
\description{
These functions work exactly the same as \code{\link[purrr:imap]{purrr::imap()}} functions,
but allow you to map in parallel.
}
\examples{
\donttest{plan(multisession, workers = 2)}
future_imap_chr(sample(10), ~ paste0(.y, ": ", .x))
\dontshow{
# Close open connections for R CMD Check
if (!inherits(plan(), "sequential")) plan(sequential)
}
}
furrr/man/figures/ 0000755 0001762 0000144 00000000000 13713033725 013635 5 ustar ligges users furrr/man/figures/lifecycle-defunct.svg 0000644 0001762 0000144 00000001704 14065375131 017746 0 ustar ligges users