I need your help!

If you find any typos, errors, or places where the text may be improved, please let me know. The best ways to provide feedback are by GitHub or hypothes.is annotations.

Opening an issue or submitting a pull request on GitHub

Hypothesis Adding an annotation using hypothes.is. To add an annotation, select some text and then click the on the pop-up menu. To see the annotations of others, click the in the upper right-hand corner of the page.

21 Iteration

21.1 Introduction

The microbenchmark package is used for timing code.

The map() function appears in both the purrr and maps packages. See the “Prerequisites” section of the Introduction. If you see errors like the following, you are using the wrong map() function.

> map(c(TRUE, FALSE, TRUE), ~ !.)
Error: $ operator is invalid for atomic vectors
> map(-2:2, rnorm, n = 5)
Error in map(-2:2, rnorm, n = 5) : 
argument 3 matches multiple formal arguments

You can check the package in which a function is defined using the environment() function:

The result should include namespace:purrr if map() is coming from the purrr package. To explicitly reference the package to get a function from, use the colon operator ::. For example,

21.2 For loops

Exercise 21.2.1

Write for-loops to:

  1. Compute the mean of every column in mtcars.
  2. Determine the type of each column in nycflights13::flights.
  3. Compute the number of unique values in each column of iris.
  4. Generate 10 random normals for each of \(\mu\) = -10, 0, 10, and 100.

The answers for each part are below.

  1. To compute the mean of every column in mtcars.

  2. Determine the type of each column in nycflights13::flights.

    I used a list, not a character vector, since the class of an object can have multiple values. For example, the class of the time_hour column is POSIXct, POSIXt.

  3. To compute the number of unique values in each column of the iris dataset.

  4. To generate 10 random normals for each of \(\mu\) = -10, 0, 10, and 100.

    However, we don’t need a for loop for this since rnorm() recycle the mean argument.

Exercise 21.2.2

Eliminate the for loop in each of the following examples by taking advantage of an existing function that works with vectors:

Since str_c() already works with vectors, use str_c() with the collapse argument to return a single string.

For this I’m going to rename the variable sd to something different because sd is the name of the function we want to use.

We could simply use the sd function.

Or if there was a need to use the equation (e.g. for pedagogical reasons), then the functions mean() and sum() already work with vectors:

The code above is calculating a cumulative sum. Use the function cumsum()

Exercise 21.2.3

Combine your function writing and for loop skills:

  1. Write a for loop that prints() the lyrics to the children’s song “Alice the camel”.

  2. Convert the nursery rhyme “ten in the bed” to a function. Generalize it to any number of people in any sleeping structure.

  3. Convert the song “99 bottles of beer on the wall” to a function. Generalize to any number of any vessel containing any liquid on surface.

The answers to each part follow.

  1. The lyrics for Alice the Camel are:

    Alice the camel has five humps.
    Alice the camel has five humps.
    Alice the camel has five humps.
    So go, Alice, go.

    This verse is repeated, each time with one fewer hump, until there are no humps. The last verse, with no humps, is:

    Alice the camel has no humps.
    Alice the camel has no humps.
    Alice the camel has no humps.
    Now Alice is a horse.

    We’ll iterate from five to no humps, and print out a different last line if there are no humps.

  2. The lyrics for Ten in the Bed are:

    Here we go!
    There were ten in the bed
    and the little one said,
    “Roll over, roll over.”
    So they all rolled over and one fell out.

    This verse is repeated, each time with one fewer in the bed, until there is one left. That last verse is:

    One! There was one in the bed
    and the little one said,
    “I’m lonely…”

    numbers <- c(
  "ten", "nine", "eight", "seven", "six", "five",
  "four", "three", "two", "one"
)
for (i in numbers) {
  cat(str_c("There were ", i, " in the bed\n"))
  cat("and the little one said\n")
  if (i == "one") {
    cat("I'm lonely...")
  } else {
    cat("Roll over, roll over\n")
    cat("So they all rolled over and one fell out.\n")
  }
  cat("\n")
}
#> There were ten in the bed
#> and the little one said
#> Roll over, roll over
#> So they all rolled over and one fell out.
#> 
#> There were nine in the bed
#> and the little one said
#> Roll over, roll over
#> So they all rolled over and one fell out.
#> 
#> There were eight in the bed
#> and the little one said
#> Roll over, roll over
#> So they all rolled over and one fell out.
#> 
#> There were seven in the bed
#> and the little one said
#> Roll over, roll over
#> So they all rolled over and one fell out.
#> 
#> There were six in the bed
#> and the little one said
#> Roll over, roll over
#> So they all rolled over and one fell out.
#> 
#> There were five in the bed
#> and the little one said
#> Roll over, roll over
#> So they all rolled over and one fell out.
#> 
#> There were four in the bed
#> and the little one said
#> Roll over, roll over
#> So they all rolled over and one fell out.
#> 
#> There were three in the bed
#> and the little one said
#> Roll over, roll over
#> So they all rolled over and one fell out.
#> 
#> There were two in the bed
#> and the little one said
#> Roll over, roll over
#> So they all rolled over and one fell out.
#> 
#> There were one in the bed
#> and the little one said
#> I'm lonely...

  3. The lyrics of Ninety-Nine Bottles of Beer on the Wall are

    99 bottles of beer on the wall, 99 bottles of beer.
    Take one down, pass it around, 98 bottles of beer on the wall

    This verse is repeated, each time with one few bottle, until there are no more bottles of beer. The last verse is

    No more bottles of beer on the wall, no more bottles of beer.
    We’ve taken them down and passed them around; now we’re drunk and passed out!

    For the bottles of beer, I define a helper function to correctly print the number of bottles.

Exercise 21.2.4

It’s common to see for loops that don’t preallocate the output and instead increase the length of a vector at each step:

How does this affect performance? Design and execute an experiment.

In order to compare these two approaches, I’ll define two functions: add_to_vector will append to a vector, like the example in the question, and add_to_vector_2 which pre-allocates a vector.

I’ll use the package microbenchmark to run these functions several times and compare the time it takes. The package microbenchmark contains utilities for benchmarking R expressions. In particular, the microbenchmark() function will run an R expression a number of times and time it.

In this example, appending to a vector takes 325 times longer than pre-allocating the vector. You may get different answers, but the longer the vector and the larger the objects, the more that pre-allocation will outperform appending.

21.3 For loop variations

Exercise 21.3.1

Imagine you have a directory full of CSV files that you want to read in. You have their paths in a vector, files <- dir("data/", pattern = "\\.csv$", full.names = TRUE), and now want to read each one with read_csv(). Write the for loop that will load them into a single data frame.

Since, the number of files is known, pre-allocate a list with a length equal to the number of files.

Then, read each file into a data frame, and assign it to an element in that list. The result is a list of data frames.

Finally, use use bind_rows() to combine the list of data frames into a single data frame.

Alternatively, I could have pre-allocated a list with the names of the files.

Exercise 21.3.2

What happens if you use for (nm in names(x)) and x has no names? What if only some of the elements are named? What if the names are not unique?

Let’s try it out and see what happens. When there are no names for the vector, it does not run the code in the loop. In other words, it runs zero iterations of the loop.

Note that the length of NULL is zero:

If there only some names, then we get an error for trying to access an element without a name.

Finally, if the vector contains duplicate names, then x[[nm]] returns the first element with that name.

Exercise 21.3.3

Write a function that prints the mean of each numeric column in a data frame, along with its name. For example, show_mean(iris) would print:

Extra challenge: what function did I use to make sure that the numbers lined up nicely, even though the variable names had different lengths?

There may be other functions to do this, but I’ll use str_pad(), and str_length() to ensure that the space given to the variable names is the same. I messed around with the options to format() until I got two digits.

Exercise 21.3.4

What does this code do? How does it work?

This code mutates the disp and am columns:

  • disp is multiplied by 0.0163871
  • am is replaced by a factor variable.

The code works by looping over a named list of functions. It calls the named function in the list on the column of mtcars with the same name, and replaces the values of that column.

This is a function.

This applies the function to the column of mtcars with the same name

21.4 For loops vs. functionals

Exercise 21.4.1

Read the documentation for apply(). In the 2nd case, what two for-loops does it generalize.

For an object with two-dimensions, such as a matrix or data frame, apply() replaces looping over the rows or columns of a matrix or data-frame. The apply() function is used like apply(X, MARGIN, FUN, ...), where X is a matrix or array, FUN is a function to apply, and ... are additional arguments passed to FUN.

When MARGIN = 1, then the function is applied to each row. For example, the following example calculates the row means of a matrix.

That is equivalent to this for-loop.

When MARGIN = 2, apply() is equivalent to a for-loop looping over columns.

Exercise 21.4.2

Adapt col_summary() so that it only applies to numeric columns. You might want to start with an is_numeric() function that returns a logical vector that has a TRUE corresponding to each numeric column.

The original col_summary() function is

The adapted version adds extra logic to only apply the function to numeric columns.

Let’s test that col_summary2() works by creating a small data frame with some numeric and non-numeric columns.

As expected, it only calculates the mean of the numeric columns, X1 and X3. Let’s test that it works with another function.

21.5 The map functions

Exercise 21.5.1

Write code that uses one of the map functions to:

  1. Compute the mean of every column in mtcars.
  2. Determine the type of each column in nycflights13::flights.
  3. Compute the number of unique values in each column of iris.
  4. Generate 10 random normals for each of \(\mu = -10\), \(0\), \(10\), and \(100\).

  1. To calculate the mean of every column in mtcars, apply the function mean() to each column, and use map_dbl, since the results are numeric.

  2. To calculate the type of every column in nycflights13::flights apply the function typeof(), discussed in the section on Vector basics, and use map_chr(), since the results are character.

  3. The function n_distinct() calculates the number of unique values in a vector.

    The map_int() function is used since length() returns an integer. However, the map_dbl() function will also work.

    An alternative to the n_distinct() function is the expression, length(unique(...)). The n_distinct() function is more concise and faster, but length(unique(...)) provides an example of using anonymous functions with map functions. An anonymous function can be written using the standard R syntax for a function:

    Additionally, map functions accept one-sided formulas as a more concise alternative to specify an anonymous function:

    In this case, the anonymous function accepts one argument, which is referenced by .x in the expression length(unique(.x)).

  4. To generate 10 random normals for each of \(\mu = -10\), \(0\), \(10\), and \(100\): The result is a list of numeric vectors.

    Since a single call of rnorm() returns a numeric vector with a length greater than one we cannot use map_dbl, which requires the function to return a numeric vector that is only length one (see Exercise 21.5.4). The map functions pass any additional arguments to the function being called.

Exercise 21.5.2

How can you create a single vector that for each column in a data frame indicates whether or not it’s a factor?

The function is.factor() indicates whether a vector is a factor.

Checking all columns in a data frame is a job for a map_*() function. Since the result of is.factor() is logical, we will use map_lgl() to apply is.factor() to the columns of the data frame.

Exercise 21.5.3

What happens when you use the map functions on vectors that aren’t lists? What does map(1:5, runif) do? Why?

Map functions work with any vectors, not just lists. As with lists, the map functions will apply the function to each element of the vector. In the following examples, the inputs to map() are atomic vectors (logical, character, integer, double).

It is important to be aware that while the input of map() can be any vector, the output is always a list.

This expression is equivalent to running the following.

The map() function loops through the numbers 1 to 5. For each value, it calls the runif() with that number as the first argument, which is the number of sample to draw. The result is a length five list with numeric vectors of sizes one through five, each with random samples from a uniform distribution. Note that although input to map() was an integer vector, the return value was a list.

Exercise 21.5.4

What does map(-2:2, rnorm, n = 5) do? Why?

What does map_dbl(-2:2, rnorm, n = 5) do? Why?

Consider the first expression.

This expression takes samples of size five from five normal distributions, with means of (-2, -1, 0, 1, and 2), but the same standard deviation (1). It returns a list with each element a numeric vectors of length 5.

However, if instead, we use map_dbl(), the expression raises an error.

This is because the map_dbl() function requires the function it applies to each element to return a numeric vector of length one. If the function returns either a non-numeric vector or a numeric vector with a length greater than one, map_dbl() will raise an error. The reason for this strictness is that map_dbl() guarantees that it will return a numeric vector of the same length as its input vector.

This concept applies to the other map_*() functions. The function map_chr() requires that the function always return a character vector of length one; map_int() requires that the function always return an integer vector of length one; map_lgl() requires that the function always return an logical vector of length one. Use the map() function if the function will return values of varying types or lengths.

To return a numeric vector, use flatten_dbl() to coerce the list returned by map() to a numeric vector.

Exercise 21.5.5

Rewrite map(x, function(df) lm(mpg ~ wt, data = df)) to eliminate the anonymous function.

This code in this question does not run, so I will use the following code.

We can eliminate the use of an anonymous function using the ~ shortcut.

Though not the intent of this question, the other way to eliminate anonymous function is to create a named one.

21.6 Dealing with failure

No exercises

21.7 Mapping over multiple arguments

No exercises

21.8 Walk

No exercises

21.9 Other patterns of for loops

Exercise 21.9.1

Implement your own version of every() using a for loop. Compare it with purrr::every(). What does purrr’s version do that your version doesn’t?

The function purrr::every() does fancy things with the predicate function argument .p, like taking a logical vector instead of a function, or being able to test part of a string if the elements of .x are lists.

Exercise 21.9.2

Create an enhanced col_summary() that applies a summary function to every numeric column in a data frame.

I will use map to apply the function to all the columns, and keep to only select numeric columns.

Exercise 21.9.3

A possible base R equivalent of col_summary() is:

But it has a number of bugs as illustrated with the following inputs:

What causes these bugs?

The cause of these bugs is the behavior of sapply(). The sapply() function does not guarantee the type of vector it returns, and will returns different types of vectors depending on its inputs. If no columns are selected, instead of returning an empty numeric vector, it returns an empty list. This causes an error since we can’t use a list with [.

The sapply() function tries to be helpful by simplifying the results, but this behavior can be counterproductive. It is okay to use the sapply() function interactively, but avoid programming with it.