Puzzler 22

Cast Away

Although all values in Scala are objects, its basic value types (Byte, Short, Int, etc.) are compiled where possible into their primitive counterparts in Java. This allows you to think of instances of those types as regular objects, simplifying the programming model. Treating Java primitives as Scala value types also makes working with Java libraries easier.

By contrast, no similar translation exists between Java and Scala collection types—you have to convert between them. Scala provides two objects, JavaConversions and JavaConverters, to help you deal with such conversions. JavaConverters is usually preferred, because it makes the conversions more obvious in the code.

The following program showcases the use of a Java collection in Scala. What does it do?

  import collection.JavaConverters._
  
  def javaMap: java.util.Map[String, java.lang.Integer] = {
    val map =
      new java.util.HashMap[String, java.lang.Integer]()
    map.put("key", null)
    map
  }
  
  val scalaMap = javaMap.asScala
  val scalaTypesMap =
    scalaMap.asInstanceOf[scala.collection.Map[String, Int]]
  
  println(scalaTypesMap("key") == null)
  println(scalaTypesMap("key") == 0)

Possibilities

1. Prints:

     true
     true
   

2. Both println statements throw a NullPointerException.
3. Prints:

     true
     false
   

4. Prints:

     false
     true
   

Explanation

As a first step, let's walk through the program. As its name implies, the javaMap method mimics calling into a Java library. It returns a Java map with Java key and value types.

The asScala method converts the Java map to a Scala map:

  scala> javaMap
  res0: java.util.Map[String,Integer] = {key=null}
  
  scala> val scalaMap = javaMap.asScala
  scalaMap: scala.collection.mutable.Map[String,Integer] = 
    Map(key -> null)

At this point, you have a Scala map, but the value type java.lang.Integer looks a bit unusual to a Scala programmer's eye. In order to switch to Scala types, you can cast the java.lang.Integer to scala.Int:

  scala> val scalaTypesMap = scalaMap.asInstanceOf[
           scala.collection.Map[String, Int]]
  scalaTypesMap: scala.collection.Map[String,Int] =
    Map(key -> null)

Having finally arrived at a native Scala map, you can start using it—in this case, in the two subsequent println statements.

You may have noticed that the result of the last code snippet revealed that the actual value of scalaTypesMap was Map(key -> null). You might therefore assume that the first println statement must produce true and the second false (i.e., candidate answer number 3).

The REPL tells a different story:

  scala> println(scalaTypesMap("key") == null)
  true
  
  scala> println(scalaTypesMap("key") == 0)
  true

So the correct answer is number 1. How is it possible for a value to be equal to null and 0 at the same time? In essence, the problem lies in the casting of scalaMap and the fact that java.lang.Integer and scala.Int are unfortunately not quite the same.

You may have noticed that scalaTypesMap contains the value null even though the type of its values is Int. Values of type Int should by definition never be null, since Int extends AnyVal.[1] Here, however, null is inside a collection. This is important, because Scala collections, like Java collections (except for arrays), cannot store Java primitive types directly. Because collections are generic, and generic classes are subject to type erasure, the type of collection elements is erased to AnyRef (java.lang.Object). All Scala's value types, including Int, are AnyVals and thus need to be boxed into AnyRef wrapper types when stored in a collection.[2] So every Scala Map[String, Int] actually contains Integer values under the covers. Reading Integer values out of a Map[String, Int] is thus what the compiler does all the time.

The real action of the puzzler lies in the two println statements. The first println statement compares the map value to null:

  println(scalaTypesMap("key") == null)

Because the value extracted from the map is an AnyRef wrapper type, it is compared to null (also an AnyRef) directly. The unsurprising result of null == null is true.

The second println statement, on the other hand, compares the map value with zero:

  println(scalaTypesMap("key") == 0)

For performance reasons, the compiler will always try to carry out operations on primitive types where possible, instead of on wrapper types. This means that the compiler unboxes the wrapper type extracted from the map, which results in code similar to:

  println(unbox(scalaTypesMap("key")) == 0)

If you decompile the code example, you can see the exact calls the compiler uses to perform the unboxing:

  Predef$.MODULE$.println(BoxesRunTime.boxToBoolean(
    scalaTypesMap.apply("key") == null));
  Predef$.MODULE$.println(BoxesRunTime.boxToBoolean(
    BoxesRunTime.unboxToInt(scalaTypesMap.apply("key")) == 0));

Method BoxesRunTime.unboxToInt is implemented as follows:[3]

  public static int unboxToInt(Object i) {
    return i == null ? 0 : ((java.lang.Integer)i).intValue();
  }

This is why when the result of scalaTypesMap("key") is unboxed for the second println statement, zero is returned. As a result, this equality comparison also evaluates to true.

You can observe the behavior just described by examining the output of certain compiler phases. The most relevant one here is the erasure phase. Passing the -Xprint:erasure option to the compiler prints the following (simplified for clarity):

  def javaMap(): java.util.Map = {
    val map: java.util.HashMap = new java.util.HashMap();
    map.put("key", null);
    map
  };
  // types erased
  val scalaMap: collection.mutable.Map =
    mapAsScalaMapConverter(javaMap()).asScala()
    .asInstanceOf[collection.mutable.Map]();
  val scalaTypesMap: collection.Map = 
    scalaMap.asInstanceOf[collection.Map]();
  println(scala.Boolean.box(
    scalaTypesMap.apply("key").==(null)));
  // wrapper type unboxed to primitive
  println(scala.Boolean.box(
    unbox(scalaTypesMap.apply("key")).==(0)))

Discussion

You might be wondering what would happen if you tried the same set of println statements against scalaMap, which retains the original Java key and value types:

  println(scalaMap("key") == null)
  println(scalaMap("key") == 0)

Notice the difference after the erasure phase:

  println(scala.Boolean.box(scalaMap.apply("key").==(null)));
  println(scala.Boolean.box(
    scalaMap.apply("key").==(scala.Int.box(0))))

The crucial difference is in the last statement. Instead of unboxing the map value, which has type java.lang.Integer, the compiler boxes the literal zero and compares the resulting AnyRefs. This time the result is more in line with expectations:

  scala> println(scalaMap("key") == null)
  true
  
  scala> println(scalaMap("key") == 0)
  false

In short, a wrapped value type stored in a collection will be unboxed when compared against another value type. If the declared type of the collection is a wrapper type, by contrast, no unboxing will occur and the the value type will be boxed instead.

Casts are inherently unsafe and should be generally avoided, because they bypass type checking entirely. In particular, casting Scala value to Java wrapper types, and vice versa, can produce unexpected results. Let the compiler handle the conversions implicitly instead of explicitly casting yourself.

---

Footnotes for Chapter 22:

[1] See Puzzler 28 for a more detailed discussion of value types.

[2] Odersky, Spoon, Venners, Programming in Scala, [Ode10]

[3] The BoxesRunTime.unboxToInt logic differs from that of Predef.Integer2int and scala.Int.unbox, which when passed null throw a NullPointerException.

[4] Odersky, The Scala Language Specification, Section 6.3. [Ode14]