Puzzler 15

Count Me Now, Count Me Later

Scala uses the underscore character (_) as a wildcard symbol quite extensively. The following program focuses on two uses of this symbol. What does it print?

  var x = 0
  def counter() = { x += 1; x }
  def add(a: Int)(b: Int) = a + b
  val adder1 = add(counter)(_)
  val adder2 = add(counter) _
  
  println("x = " + x)
  println(adder1(10))
  println("x = " + x)
  println(adder2(10))
  println("x = " + x)

Possibilities

1. Prints:

     x = 1
     12
     x = 2
     11
     x = 2
   

2. Prints:

     x = 1
     11
     x = 1
     12
     x = 2
   

3. Prints:

     x = 0
     11
     x = 1
     12
     x = 2
   

4. Prints:

     x = 2
     11
     x = 2
     12
     x = 2
   

Explanation

The lines that define counter, adder1, and adder2 seem to be crucial in the code snippet, so let's focus on fully understanding them. The counter method definition contains an example of a side effect. When evaluated, the current value of x is incremented and returned, because in Scala, the last expression of a method is its return value.

The expressions defining adder1 and adder2 seem more interesting. They look deceptively similar and produce the same function value type:

  scala> val adder1 = add(counter)(_)
  adder1: Int => Int = <function1>
  
  scala> val adder2 = add(counter) _
  adder2: Int => Int = <function1>

Yet, their semantics are completely different. In the case of adder1, it might appear as if a function value is being created by partially applying method add. Actually, the underscore in this case is an example of placeholder syntax for anonymous functions.[1] For instance, consider the expression:

  _ + 1

This is equivalent to an anonymous function:

  x => x + 1

The expression defining adder1 is similar:

  add(counter)(_)

It expands to:

  a => add(counter)(a)

This is a function value, so the evaluation of argument counter is deferred until adder1 is evaluated.

The kind of expression that defines adder2 is different:

  val adder2 = add(counter) _

This expression is an example of a partially applied function, since the parameter b of method add is not supplied. Thus eta expansion[2] takes place to convert the method into a function of the remaining parameters. This entails creating fresh values for every supplied method argument, which, as a result, causes the arguments to be evaluated eagerly. In the case of adder2, the compiler generates something along these lines behind the scenes:

  val adder2 = {
    val fresh = counter()
    a => add(fresh)(a)
  }

Hence, the crucial difference between adder1 and adder2 has to do with the evaluation of argument counter. In adder1, counter will be evaluated each time adder1 is used. In adder2, counter is evaluated once and will not be evaluated again when the function is called.

  def foo[A, B](a: A): B

Now let's walk through the program. In the first three lines, the value of x is 0:

  var x = 0
  def counter() = { x += 1; x }
  def add(a: Int)(b: Int) = a + b

In the next line, the adder1 function value is defined but counter is not evaluated, so x is still 0:

  val adder1 = add(counter)(_)

Method counter is executed for the first time in the next line, when adder2 is initialized:

  val adder2 = add(counter) _

This, therefore, increments x to 1. At this point, parameter a of method add for adder1 is bound to value 1. After that, the current value of x is printed, which is still 1 at this point:

  println("x = " + x)

The subsequent line applies adder1 to 10, executing counter along the way and incrementing x to 2:

  println(adder1(10))

Consequently, this prints 12, and the next statement reveals that x equals 2. The following line then applies adder2 to 10:

  println(adder2(10))

At the time function adder2 was created, counter evaluated to 1. This call to adder2 adds 10 to 1 to produce 11. And finally, the last line prints 2, because the value of x has not since it was last printed.

Therefore, the correct answer is number 1:

  scala> println("x = " + x)
  x = 1
  scala> println(adder1(10))
  12
  scala> println("x = " + x)
  x = 2
  scala> println(adder2(10))
  11
  scala> println("x = " + x)
  x = 2

Every subsequent invocation of adder2 will return the same result, unlike invoking adder1, which will keep incrementing x:

  scala> println(adder1(10))
  13
  scala> println(adder2(10))
  11

Discussion

It is worth noting a couple of points about this program. First, the use of vars does not reflect idiomatic Scala style. Even though Scala supports mutable state, it encourages a functional style of programming, i.e., preferring vals and immutability.

Second, the surprising behavior is manifested only in side-effecting code. The side effect inside counter, x += 1, is what makes this program difficult to reason about. This is one of the drawbacks of imperative programming: when functions are passed around and manipulated, it is hard to know when and how many times they will be called. The outcome is more predictable when calling a function that has no effect on the outside world.

Lastly, a common error is to try to construct adder with the intent of partially applying method add, but without explicitly writing the underscore symbol, like this:

  val adder3 = add(counter)

That, unfortunately, results in a compiler error:

  scala> val adder3 = add(counter)
  <console>:10: error: missing arguments for method add;
    follow this method with `_' if you want to treat it
    as a partially applied function
         val adder3 = add(counter)
                         ^

The reason for this error is that the compiler is not explicitly expecting adder3 to be a function value (i.e., of type Function1[Int, Int]). If you specify the type of adder3, the compiler performs eta expansion without complaining:

  scala> val adder3: Int => Int = add(counter)
  adder3:Int => Int => <function1>

This comes in handy when a method is expecting a function as an argument. For example, consider the fold method on a List[A]:

  def fold[A1 >: A](z: A1)(op: (A1, A1) => A1): A1

Imagine you want to fold using a sum operation defined as:

  def sum(a: Int, b: Int) = a + b

You can explicitly pass sum with the underscore:

  List(1, 2, 3).fold(0)(sum _)

But you can also omit it:

  List(1, 2, 3).fold(0)(sum)

In short, Scala allows you to leave off the underscore after a method name when it knows the expected type is a function, and the type of the function is consistent with the signature of the method.

Familiarize yourself with anonymous and partially applied functions as well as the other uses of the underscore symbol. In general, try to avoid writing code that readers can easily misinterpret.

Footnotes for Chapter 15:

[1] Odersky, The Scala Language Specification, Section 6.23. [Ode14]

[2] Odersky, The Scala Language Specification, Section 6.26.5. [Ode14]

[3] Odersky, The Scala Language Specification, Section 6.26.5. [Ode14]

[4] Gleichmann, "Functional Scala: Turning Methods into Functions." [Gle11]

[5] Zaugg, "In Scala, why can't I partially apply a function without explicitly specifying its argument types?" [Zau10a]