Chapter 21. Implicits
Topics in This Chapter 
21.2 Using Implicits for Enriching Existing Classes
21.4 Rules for Implicit Conversions
21.6 Implicit Conversions with Implicit Parameters
21.10 The @implicitNotFound Annotation
21.11 CanBuildFrom Demystified
Implicit conversions and implicit parameters are Scala’s power tools that do useful work behind the scenes. In this chapter, you will learn how implicit conversions can be used to enrich existing classes, and how implicit objects are summoned automatically to carry out conversions or other tasks. With implicits, you can provide elegant libraries that hide tedious details from library users.
The key points of this chapter are:
• Implicit conversions are used to convert between types.
• You must import implicit conversions so that they are in scope.
• An implicit parameter list requests objects of a given type. They can be obtained from implicit objects that are in scope, or from the companion object of the desired type.
• If an implicit parameter is a single-argument function, it is also used as an implicit conversion.
• A context bound of a type parameter requires the existence of an implicit object of the given type.
• If it is possible to locate an implicit object, this can serve as evidence that a type conversion is valid.
21.1 Implicit Conversions
An implicit conversion function is a function with a single parameter that is declared with the implicit keyword. As the name suggests, such a function is automatically applied to convert values from one type to another.
Consider the Fraction class from Section 11.2, “Infix Operators,” on page 143 with a method * for multiplying a fraction with another. We want to convert integers n to fractions n / 1.
implicit def int2Fraction(n: Int) = Fraction(n, 1)
Now we can evaluate
val result = 3 * Fraction(4, 5) // Calls int2Fraction(3)
The implicit conversion function turns the integer 3 into a Fraction object. That object is then multiplied by Fraction(4, 5).
You can give any name to the conversion function. Since you don’t call it explicitly, you may be tempted to use something short such as i2f. But, as you will see in Section 21.3, “Importing Implicits,” on page 325, sometimes it is useful to import a conversion function. I suggest that you stick with the source2target convention.
Scala is not the first language that allows the programmer to provide automatic conversions. However, Scala gives programmers a great deal of control over when to apply these conversions. In the following sections, we will discuss exactly when the conversions happen, and how you can fine-tune the process.
Note
Even though Scala gives you tools to fine-tune implicit conversions, the language designers realize that implicit conversions are potentially problematic. To avoid a warning when using implicit functions, add the statement import scala.language.implicitConversions or the compiler option -language:implicitConversions.
Note
In C++, you specify implicit conversions as one-argument constructors or member functions with the name operator Type(). However, in C++, you cannot selectively allow or disallow these functions, and it is common to run into unwanted conversions.
21.2 Using Implicits for Enriching Existing Classes
Did you ever wish that a class had a method its creator failed to provide? For example, wouldn’t it be nice if the java.io.File class had a read method for reading a file:
val contents = new File("README").read
As a Java programmer, your only recourse is to petition Oracle Corporation to add that method. Good luck!
In Scala, you can define an enriched class that provides what you want:
class RichFile(val from: File) {
def read = Source.fromFile(from.getPath).mkString
}
Then, provide an implicit conversion to that type:
implicit def file2RichFile(from: File) = new RichFile(from)
Now it is possible to call read on a File object. It is implicitly converted to a RichFile.
Instead of providing a conversion function, you can declare RichFile as an implicit class:
implicit class RichFile(val from: File) { ... }
An implicit class must have a primary constructor with exactly one argument. That constructor becomes the implicit conversion function.
It is a good idea to declare the enriched class as a value class:
implicit class RichFile(val from: File) extends AnyVal { ... }
In that case, no RichFile objects are created. A call file.read is directly compiled into a static method call RichFile$.read$extension(file).
Caution
An implicit class cannot be a top-level class. You can place it inside the class that uses the type conversion, or inside another object or class that you import, as explained in the next section.
21.3 Importing Implicits
Scala will consider the following implicit conversion functions:
1. Implicit functions or classes in the companion object of the source or target type
2. Implicit functions or classes that are in scope
For example, consider the int2Fraction function. We can place it into the Fraction companion object, and it will be available for converting fractions.
Alternatively, let’s suppose we put it inside a FractionConversions object, which we define in the com.horstmann.impatient package. If you want to use the conversion, import the FractionConversions object, like this:
import com.horstmann.impatient.FractionConversions._
For an implicit conversion to be in scope, it must be imported without a prefix. For example, if you import com.horstmann.impatient.FractionConversions or com.horstmann, then the int2Fraction method is in scope as FractionConversions.int2Fraction or impatient.FractionConversions.int2Fraction, to anyone who wants to call it explicitly. But if the function is not available as int2Fraction, without a prefix, the compiler won’t use it implicitly.
Tip
In the REPL, type :implicits to see all implicits that have been imported from a source other than Predef, or :implicits -v to see all implicits.
You can localize the import to minimize unintended conversions. For example,
object Main extends App {
import com.horstmann.impatient.FractionConversions._
val result = 3 * Fraction(4, 5) // Uses imported conversion
println(result)
}
You can even select the specific conversions that you want. Suppose you have a second conversion
object FractionConversions {
...
implicit def fraction2Double(f: Fraction) = f.num * 1.0 / f.den
}
If you prefer this conversion over int2Fraction, you can import it:
import com.horstmann.impatient.FractionConversions.fraction2Double
val result = 3 * Fraction(4, 5) // result is 2.4
You can also exclude a specific conversion if it causes you trouble:
import com.horstmann.impatient.FractionConversions.{fraction2Double => _, _}
// Imports everything but fraction2Double
Tip
If you want to find out why the compiler doesn’t use an implicit conversion that you think it should use, try adding it explicitly, for example by calling fraction2Double(3) * Fraction(4, 5). You may get an error message that shows the problem.
21.4 Rules for Implicit Conversions
In this section, you will see when implicit conversions are attempted. To illustrate the rules, we again use the Fraction class and assume that the implicit conversions int2Fraction and fraction2Double are both available.
Implicit conversions are considered in three distinct situations:
• If the type of an expression differs from the expected type:
3 * Fraction(4, 5) // Calls fraction2Double
The Int class doesn’t have a method *(Fraction), but it has a method *(Double).
• If an object accesses a nonexistent member:
3.den // Calls int2Fraction
The Int class doesn’t have a den member but the Fraction class does.
• If an object invokes a method whose parameters don’t match the given arguments:
Fraction(4, 5) * 3
// Calls int2Fraction
The * method of Fraction doesn’t accept an Int but it accepts a Fraction.
On the other hand, there are three situations when an implicit conversion is not attempted:
• No implicit conversion is used if the code compiles without it. For example, if a * b compiles, the compiler won’t try a * convert(b) or convert(a) * b.
• The compiler will never attempt multiple conversions, such as convert1( convert2(a)) * b.
• Ambiguous conversions are an error. For example, if both convert1(a) * b and convert2(a) * b are valid, the compiler will report an error.
Caution
It is not an ambiguity that
3 * Fraction(4, 5)
could be either
3 * fraction2Double(Fraction(4, 5))
or
int2Fraction(3) * Fraction(4, 5)
The first conversion wins over the second, since it does not require modification of the object to which the * method is applied.
Tip
If you want to find out which implicit conversion the compiler uses, compile your program as
scalac -Xprint:typer MyProg.scala
You will see the source after implicit conversions have been added.
21.5 Implicit Parameters
A function or method can have a parameter list that is marked implicit. In that case, the compiler will look for default values to supply with the function call. Here is a simple example:
case class Delimiters(left: String, right: String)
def quote(what: String)(implicit delims: Delimiters) =
delims.left + what + delims.right
You can call the quote method with an explicit Delimiters object, like this:
quote("Bonjour le monde")(Delimiters("«", "»")) // Returns «Bonjour le monde»
Note that there are two argument lists. This function is “curried”—see Chapter 12.
You can also omit the implicit parameter list:
quote("Bonjour le monde")
In that case, the compiler will look for an implicit value of type Delimiters. This must be a value that is declared as implicit. The compiler looks for such an object in two places:
• Among all val and def of the desired type that are in scope without a prefix.
• In the companion object of a type that is associated with the desired type. Associated types include the desired type itself and, if it is a parameterized type, its type parameters.
In our example, it is useful to make an object, such as
object FrenchPunctuation {
implicit val quoteDelimiters = Delimiters("«", "»")
...
}
Then one imports all values from the object:
import FrenchPunctuation._
or just the specific value:
import FrenchPunctuation.quoteDelimiters
Now the French delimiters are supplied implicitly to the quote function.
Note
There can only be one implicit value for a given data type. Thus, it is not a good idea to use implicit parameters of common types. For example,
def quote(what: String)(implicit left: String, right: String) // No!
would not work—one could not supply two different strings.
21.6 Implicit Conversions with Implicit Parameters
An implicit function parameter is also usable as an implicit conversion. To understand the significance, consider first this simple generic function:
def smaller[T](a: T, b: T) = if (a < b) a else b // Not quite
That doesn’t actually work. The compiler won’t accept the function because it doesn’t know that a and b belong to a type with a < operator.
We can supply a conversion function for that purpose:
def smaller[T](a: T, b: T)(implicit order: T => Ordered[T])
= if (order(a) < b) a else b
Since the Ordered[T] trait has a < operator that consumes a T, this version is correct.
As it happens, this is such a common situation that the Predef object defines implicit values of type T => Ordered[T] for a large number of types, including all types that already implement Ordered[T] or Comparable[T]. Therefore, you can call
smaller(40, 2)
and
smaller("Hello", "World")
If you want to call
smaller(Fraction(1, 7), Fraction(2, 9))
then you need to define a function Fraction => Ordered[Fraction] and either supply it in the call or make it available as an implicit val. I leave this as an exercise because it moves us too far from the point that I want to make in this section.
Here, finally, is the point. Look again at
def smaller[T](a: T, b: T)(implicit order: T => Ordered[T])
Note that order is a function that is tagged implicit and is in scope. Therefore, it is an implicit conversion, in addition to being an implicit parameter. So, we can omit the call to order in the body of the function:
def smaller[T](a: T, b: T)(implicit order: T => Ordered[T])
= if (a < b) a else b // Calls order(a) < b if a doesn't have a < operator
21.7 Context Bounds
A type parameter can have a context bound of the form T : M, where M is another generic type. It requires that there is an implicit value of type M[T] in scope.
For example,
class Pair[T : Ordering]
requires that there is an implicit value of type Ordering[T]. That implicit value can then be used in the methods of the class. Consider this example:
class Pair[T : Ordering](val first: T, val second: T) {
def smaller(implicit ord: Ordering[T]) =
if (ord.compare(first, second) < 0) first else second
}
If we form a new Pair(40, 2), then the compiler infers that we want a Pair[Int]. Since there is an implicit value of type Ordering[Int] in the Ordering companion object, Int fulfills the context bound. That ordering becomes a field of the class, and it is passed to the methods that need it.
If you prefer, you can retrieve the ordering with the implicitly method in the Predef class:
class Pair[T : Ordering](val first: T, val second: T) {
def smaller =
if (implicitly[Ordering[T]].compare(first, second) < 0) first else second
}
The implicitly function is defined as follows in Predef.scala:
def implicitly[T](implicit e: T) = e
// For summoning implicit values from the nether world
Note
The comment is apt—the implicit objects live in the “nether world” and are invisibly added to methods.
Alternatively, you can take advantage of the fact that the Ordered trait defines an implicit conversion from Ordering to Ordered. If you import that conversion, you can use relational operators:
class Pair[T : Ordering](val first: T, val second: T) {
def smaller = {
import Ordered._;
if (first < second) first else second
}
}
These are just minor variations; the important point is that you can instantiate Pair[T] whenever there is an implicit value of type Ordering[T]. For example, if you want a Pair[Point], arrange for an implicit Ordering[Point] value:
implicit object PointOrdering extends Ordering[Point] {
def compare(a: Point, b: Point) = ...
}
21.8 Type Classes
Have another look at the Ordering trait in the preceding section. We had an algorithm that required the parameters to have an ordering. Normally, in object-oriented programming, we would require the parameter types to extend a trait. But that’s not what happened here. In order to make a class usable with the algorithm, it is not necessary to modify the class at all. Instead, one provides an implicit conversion. This is much more flexible than the object-oriented approach.
A trait such as Ordering is called a type class. A type class defines some behavior, and a type can join the class by providing that behavior. (The term comes from Haskell, and “class” is not used in the same way as in object-oriented programming. Think of “class” as in a “class action”—types banding together for a common purpose.)
To see how a type joins a type class, let us look at a simple example. We want to compute averages, (x1 + . . . + xn) / n. To do that, we need to be able to add two values and divide a value by an integer. There is a type class Numeric in the Scala library, which requires that values can be added, multiplied, and compared. But it doesn’t have any requirement that values can be divided by an integer. Therefore, let’s define our own:
trait NumberLike[T] {
def plus(x: T, y: T): T
def divideBy(x: T, n: Int): T
}
Next, to make sure that the type class is useful out of the gate, let’s add some common types as members. That’s easy to do by providing implicit objects in the companion object:
object NumberLike {
implicit object NumberLikeDouble extends NumberLike[Double] {
def plus(x: Double, y: Double) = x + y
def divideBy(x: Double, n: Int) = x / n
}
implicit object NumberLikeBigDecimal extends NumberLike[BigDecimal] {
def plus(x: BigDecimal, y: BigDecimal) = x + y
def divideBy(x: BigDecimal, n: Int) = x / n
}
}
Now we are ready to put the type class to use. In the average method, we need an instance of the type class so that we can call plus and divideBy. (Note that these are methods of the type class, not the member types.)
Here, we’ll just compute the average of two values. The general case is left as an exercise. There are two ways in which we can supply the type class instance: as an implicit parameter, or with a context bound. Here is the first approach:
def average[T](x: T, y: T)(implicit ev: NumberLike[T]) =
ev.divideBy(ev.plus(x, y), 2)
The parameter name ev is shorthand for “evidence”—see the next section.
When using a context bound, we retrieve the implicit object from the “nether world.”
def average[T : NumberLike](x: T, y: T) = {
val ev = implicitly[NumberLike[T]]
ev.divideBy(ev.plus(x, y), 2)
}
That’s all there is to it. Finally, let’s see what a type needs to do to join the NumberLike type class. It must provide an implicit object, just like the NumberLikeDouble and NumberLikeBigDecimal objects that we provided out of the gate. Here is how Point can join:
class Point(val x: Double, val y: Double) {
...
}
object Point {
def apply(x: Double, y: Double) = new Point(x, y)
implicit object NumberLikePoint extends NumberLike[Point] {
def plus(p: Point, q: Point) = Point(p.x + q.x, p.y + q.y)
def divideBy(p: Point, n: Int) = Point(p.x * 1.0 / n, p.y * 1.0 / n)
}
}
Here we added the implicit object to the companion object of Point. If you can’t modify the Point class, you can put the implicit object elsewhere and import it as needed.
The standard Scala library provides useful type classes, such as Equiv, Ordering, Numeric, Fractional, Hashing, IsTraversableOnce, IsTraversableLike. As you have seen, it is easy to provide your own.
The important point about type classes is that they provide an “ad-hoc” way of providing polymorphism that is less rigid than inheritance.
21.9 Evidence
In Chapter 18, you saw the type constraints
T =:= U
T <:< U
T => U
The constraints test whether T equals U, is a subtype of U, or is convertible to U. To use such a type constraint, you supply an implicit parameter, such as
def firstLast[A, C](it: C)(implicit ev: C <:< Iterable[A]) =
(it.head, it.last)
The =:= and <:< are classes with implicit values, defined in the Predef object. For example, <:< is essentially
abstract class <:<[-From, +To] extends Function1[From, To]
object <:< {
implicit def conforms[A] = new (A <:< A) { def apply(x: A) = x }
}
Suppose the compiler processes a constraint implicit ev: String <:< AnyRef. It looks in the companion object for an implicit object of type String <:< AnyRef. Note that <:< is contravariant in From and covariant in To. Therefore the object
<:<.conforms[String]
is usable as a String <:< AnyRef instance. (The <:<.conforms[AnyRef] object is also usable, but it is less specific and therefore not considered.)
We call ev an “evidence object”—its existence is evidence of the fact that, in this case, String is a subtype of AnyRef.
Here, the evidence object is the identity function. To see why the identity function is required, have a closer look at
def firstLast[A, C](it: C)(implicit ev: C <:< Iterable[A]) =
(it.head, it.last)
The compiler doesn’t actually know that C is an Iterable[A]—recall that <:< is not a feature of the language, but just a class. So, the calls it.head and it.last are not valid. But ev is a function with one parameter, and therefore an implicit conversion from C to Iterable[A]. The compiler applies it, computing ev(it).head and ev(it).last.
Tip
To test whether a generic implicit object exists, you can call the implicitly function in the REPL. For example, type implicitly[String <:< AnyRef] in the REPL, and you get a result (which happens to be a function). But implicitly[AnyRef <:< String] fails with an error message.
21.10 The @implicitNotFound Annotation
The @implicitNotFound annotation raises an error message when the compiler cannot construct an implicit parameter of the annotated type. The intent is to give a useful error message to the programmer. For example, the <:< class is annotated as
@implicitNotFound(msg = "Cannot prove that ${From} <:< ${To}.")
abstract class <:<[-From, +To] extends Function1[From, To]
For example, if you call
firstLast[String, List[Int]](List(1, 2, 3))
then the error message is
Cannot prove that List[Int] <:< Iterable[String]
That is more likely to give the programmer a hint than the default
Could not find implicit value for parameter ev: <:<[List[Int],Iterable[String]]
Note that ${From} and ${To} in the error message are replaced with the type parameters From and To of the annotated class.
21.11 CanBuildFrom Demystified
In Chapter 1, I wrote that you should ignore the implicit CanBuildFrom parameter. Now you are finally ready to understand how it works.
Consider the map method. Simplifying slightly, map is a method of Iterable[A, Repr] with the following implementation:
def map[B, That](f : (A) => B)(implicit bf: CanBuildFrom[Repr, B, That]): That = {
val builder = bf()
val iter = iterator()
while (iter.hasNext) builder += f(iter.next())
builder.result
}
Here, Repr is the “representation type.” That parameter will enable us to select appropriate builder factories for unusual collections such as Range or String.
Note
In the Scala library, map is actually defined in the TraversableLike[A, Repr] trait. That way, the more commonly used Iterable trait doesn’t need to carry with it the Repr type parameter.
The CanBuildFrom[From, E, To] trait provides evidence that it is possible to create a collection of type To, holding values of type E, that is compatible with type From. Before discussing how these evidence objects are generated, let’s see what they do.
The CanBuildFrom trait has an apply method that yields an object of type Builder[E, To]. A Builder has methods += for adding elements into an internal buffer, and result for producing the desired collection.
trait Builder[-E, +To] {
def +=(e: E): Unit
def result(): To
}
trait CanBuildFrom[-From, -E, +To] {
def apply(): Builder[E, To]
}
Therefore, the map method simply constructs a builder for the target type, fills the builder with the values of the function f, and yields the resulting collection.
Each collection provides an implicit CanBuildFrom object in its companion object. Consider a simplified version of the standard ArrayBuffer class:
class Buffer[E : ClassTag] extends Iterable[E, Buffer[E]]
with Builder[E, Buffer[E]] {
private var elems = new Array[E](10)
...
def iterator() = ...
private var i = 0
def hasNext = i < length
def next() = { i += 1; elems(i - 1) }
}
def +=(e: E) { ... }
def result() = this
}
object Buffer {
implicit def canBuildFrom[E : ClassTag] =
new CanBuildFrom[Buffer[_], E, Buffer[E]] {
def apply() = new Buffer[E]
}
}
Consider a call buffer.map(f), where f is a function of type A => B. The implicit bf parameter is obtained by calling the canBuildFrom[B] method in the Buffer companion object. Its apply method returns the builder, in this case a Buffer[B].
As it happens, the Buffer class already has a += method, and its result method is defined to return itself. Therefore, a Buffer is its own builder.
However, a builder for the Range class doesn’t return a Range, and clearly it can’t. For example, (1 to 10).map(x => x * x) has a value that isn’t a Range. In the actual Scala library, Range extends IndexedSeq[Int], and the IndexedSeq companion object defines a builder that constructs a Vector.
Here is a simplified Range class that provides a Buffer as builder:
class Range(val low: Int, val high: Int) extends Iterable[Int, Range] {
def iterator() = ...
}
object Range {
implicit def canBuildFrom[E : ClassTag] = new CanBuildFrom[Range, E, Buffer[E]] {
def apply() = new Buffer[E]
}
}
Now consider a call Range(1, 10).map(f). That method needs an implicit bf: CanBuildFrom[Repr, B, That]. Since Repr is Range, the associated types are CanBuildFrom, Range, B, and the unknown That. The Range object yields a match by calling its canBuildFrom[B] method, which returns a CanBuildFrom[Range, B, Buffer[B]]. That object is bf, and its apply method yields a Buffer[B] for building the result.
As you just saw, the implicit CanBuildFrom[Repr, B, That] parameter locates a factory object that can produce a builder for the target collection. The builder factory is defined as implicit in the companion object of Repr.
Exercises
1. How does -> work? That is, how can "Hello" -> 42 and 42 -> "Hello" be pairs ("Hello", 42) and (42, "Hello")? Hint: Predef.ArrowAssoc.
2. Define an operator +% that adds a given percentage to a value. For example, 120 +% 10 should be 132. Use an implicit class.
3. Define a ! operator that computes the factorial of an integer. For example, 5.! is 120. Use an implicit class.
4. Some people are fond of “fluent APIs” that read vaguely like English sentences. Create such an API for reading integers, floating-point numbers, and strings from the console. For example: Read in aString askingFor "Your name" and anInt askingFor "Your age" and aDouble askingFor "Your weight".
5. Provide the machinery that is needed to compute
smaller(Fraction(1, 7), Fraction(2, 9))
with the Fraction class of Chapter 11. Supply an implicit class RichFraction that extends Ordered[Fraction].
6. Compare objects of the class java.awt.Point by lexicographic comparison.
7. Continue the previous exercise, comparing two points according to their distance to the origin. How can you switch between the two orderings?
8. Use the implicitly command in the REPL to summon the implicit objects described in Section 21.5, “Implicit Parameters,” on page 328 and Section 21.6, “Implicit Conversions with Implicit Parameters,” on page 329. What objects do you get?
9. Explain why Ordering is a type class and why Ordered is not.
10. Generalize the average method in Section 21.8, “Type Classes,” on page 331 to a Seq[T].
11. Make String a member of the NumberLike type class in Section 21.8, “Type Classes,” on page 331. The divBy method should retain every nth letter, so that average("Hello", "World") becomes "Hlool".
12. Look up the =:= object in Predef.scala. Explain how it works.
13. The result of "abc".map(_.toUpper) is a String, but the result of "abc".map(_.toInt) is a Vector. Find out why.