R is great at automating a data analysis over many groups in a dataset, but errors, warnings and other side effects can stop it in its tracks. There’s nothing worse than returning after an hour (or a day!) and discovering that 90% of your data wasn’t analysed because one group threw an error.
collateral, you can capture side effects like errors to prevent them from stopping execution. Once your analysis is done, you can see a tidy view of the groups that failed, the ones that returned a results, and the ones that threw warnings, messages or other output as they ran.
You can even filter or summarise a data frame based on these side effects to automatically continue analysis without failed groups (or perhaps to show a report of the failed ones).
For example, this screenshot shows a data frame that’s been nested using groups of the
cyl column. The
qlog column shows the results of an operation that’s returned results for all three groups but also printed a warning for one of them:
If you’re not familiar with
purrr or haven’t used a list-column workflow in R before, the
collateral vignette shows you how it works, the benefits for your analysis and how
collateral simplifies the process of handling complex mapped operations.
If you’re already familiar with
purrr, the tl;dr is that
collateral::map_quietly() (and their
pmap variants) will automatically wrap your supplied function in
quietly() (or both:
peacefully()) and will provide enhanced printed output and tibble displays. You can then use the
tally_*() functions to filter or summarise the returned tibbles or lists.
You can install the released version of collateral from CRAN with:
And the development version from GitHub with:
# install.packages("devtools") devtools::install_github("rensa/collateral")
This example uses the famous
mtcars dataset—but first, we’re going to sabotage a few of the rows by making them negative. The
log function produces
NaN with a warning when you give it a negative number.
It’d be easy to miss this in a non-interactive script if you didn’t explicitly test for the presence of
NaN! Thankfully, with collateral, you can see which operations threw errors, which threw warnings, and which produced output:
library(tibble) library(dplyr) library(tidyr) library(collateral) test <- # tidy up and trim down for the example mtcars %>% rownames_to_column(var = "car") %>% as_tibble() %>% select(car, cyl, disp, wt) %>% # spike some rows in cyl == 4 to make them fail mutate(wt = dplyr::case_when( wt < 2 ~ -wt, TRUE ~ wt)) %>% # nest and do some operations peacefully nest(data = -cyl) %>% mutate(qlog = map_peacefully(data, ~ log(.$wt))) # look at the results test #> # A tibble: 3 x 3 #> cyl data qlog #> <dbl> <list> <collat> #> 1 6 <tibble [7 × 3]> R _ _ _ _ #> 2 4 <tibble [11 × 3]> R _ _ W _ #> 3 8 <tibble [14 × 3]> R _ _ _ _
Here, we can see that all operations produced output (because
NaN is still output)—but a few of them also produced warnings! You can then see those warnings…
test %>% mutate(qlog_warning = map_chr(qlog, "warnings", .null = NA)) #> # A tibble: 3 x 4 #> cyl data qlog qlog_warning #> <dbl> <list> <collat> <chr> #> 1 6 <tibble [7 × 3]> R _ _ _ _ <NA> #> 2 4 <tibble [11 × 3]> R _ _ W _ NaNs produced #> 3 8 <tibble [14 × 3]> R _ _ _ _ <NA>
… filter on them…
test %>% filter(!has_warnings(qlog)) #> # A tibble: 2 x 3 #> cyl data qlog #> <dbl> <list> <collat> #> 1 6 <tibble [7 × 3]> R _ _ _ _ #> 2 8 <tibble [14 × 3]> R _ _ _ _
… or get a summary of them, for either interactive or non-interactive purposes:
summary(test$qlog) #> 3 elements in total. #> 3 elements returned results, #> 3 elements delivered output, #> 0 elements delivered messages, #> 1 element delivered warnings, and #> 0 elements threw an error.
The collateral package is now fully integrated with the
furrr package, so you can safely and quietly iterate operations across CPUs cores or remote nodes. All collateral mappers have
future_*-prefixed variants for this purpose.
If you have a problem with
collateral, please don’t hesitate to file an issue or contact me!