So far, we have learnt how to provide the user with a dummy JSON array of repositories in response to a request to /api/repos/:username
. In this section, we will replace the dummy data with the user's actual repositories, dowloaded from GitHub.
In Chapter 7, Web APIs, we learned how to query the GitHub API using Scala's Source.fromURL
method and scalaj-http
. It should come as no surprise that the Play framework implements its own library for interacting with external web services.
Let's edit the Api
controller to fetch information about a user's repositories from GitHub, rather than using dummy data. When called with a username as argument, the controller will:
List[Repo]
.List[Repo]
to a JSON array, forming the response.We start by giving the full code listing before explaining the thornier parts in detail:
// app/controllers/Api.scala package controllers import play.api._ import play.api.mvc._ import play.api.libs.ws.WS // query external APIs import play.api.Play.current import play.api.libs.json._ // parsing JSON import play.api.libs.functional.syntax._ import play.api.libs.concurrent.Execution.Implicits.defaultContext import models.Repo class Api extends Controller { // type class for Repo -> Json conversion implicit val writesRepo = new Writes[Repo] { def writes(repo:Repo) = Json.obj( "name" -> repo.name, "language" -> repo.language, "is_fork" -> repo.isFork, "size" -> repo.size ) } // type class for Github Json -> Repo conversion implicit val readsRepoFromGithub:Reads[Repo] = ( (JsPath "name").read[String] and (JsPath "language").read[String] and (JsPath "fork").read[Boolean] and (JsPath "size").read[Long] )(Repo.apply _) // controller def repos(username:String) = Action.async { // GitHub URL val url = s"https://api.github.com/users/$username/repos" val response = WS.url(url).get() // compose get request // "response" is a Future response.map { r => // executed when the request completes if (r.status == 200) { // extract a list of repos from the response body val reposOpt = Json.parse(r.body).validate[List[Repo]] reposOpt match { // if the extraction was successful: case JsSuccess(repos, _) => Ok(Json.toJson(repos)) // If there was an error during the extraction case _ => InternalServerError } } else { // GitHub returned something other than 200 NotFound } } } }
If you have written all this, point your browser to, for instance, 127.0.0.1:9000/api/repos/odersky
to see the list of repositories owned by Martin Odersky:
[{"name":"dotty","language":"Scala","is_fork":true,"size":14653},{"name":"frontend","language":"JavaScript","is_fork":true,"size":392},...
This code sample is a lot to take in, so let's break it down.
The first step in querying external APIs is to import the WS
object, which defines factory methods for creating HTTP requests. These factory methods rely on a reference to an implicit Play application in the namespace. The easiest way to ensure this is the case is to import play.api.Play.current
, a reference to the current application.
Let's ignore the readsRepoFromGithub
type class for now and jump straight to the controller body. The URL that we want to hit with a GET request is "https://api.github.com/users/$username/repos"
, with the appropriate value for $username
. We create a GET request with WS.url(url).get()
. We can also add headers to an existing request. For instance, to specify the content type, we could have written:
WS.url(url).withHeaders("Content-Type" -> "application/json").get()
We can use headers to pass a GitHub OAuth token using:
val token = "2502761d..." WS.url(url).withHeaders("Authorization" -> s"token $token").get()
To formulate a POST request, rather than a GET request, replace the final .get()
with .post(data)
. Here, data
can be JSON, XML or a string.
Adding .get
or .post
fires the request, returning a Future[WSResponse]
. You should, by now, be familiar with futures. By writing response.map { r => ... }
, we specify a transformation to be executed on the future result, when it returns. The transformation verifies the response's status, returning NotFound
if the status code of the response is anything but 200.
If the status code is 200, the callback parses the response body to JSON and converts the parsed JSON to a List[Repo]
instance. We already know how to convert from a Repo
object to JSON using the Writes[Repo]
type class. The converse, going from JSON to a Repo
object, is a little more challenging, because we have to account for incorrectly formatted JSON. To this effect, the Play framework provides the .validate[T]
method on JSON objects. This method tries to convert the JSON to an instance of type T
, returning JsSuccess
if the JSON is well-formatted, or JsError
otherwise (similar to Scala's Try
object). The .validate
method relies on the existence of a type class Reads[Repo]
. Let's experiment with a Scala console:
$ activator console scala> import play.api.libs.json._ import play.api.libs.json._ scala> val s = """ { "name": "dotty", "size": 150, "language": "Scala", "fork": true } """ s: String = " { "name": "dotty", "size": 150, "language": "Scala", "fork": true } " scala> val parsedJson = Json.parse(s) parsedJson: play.api.libs.json.JsValue = {"name":"dotty","size":150,"language":"Scala","fork":true}
Using Json.parse
converts a string to an instance of JsValue
, the super-type for JSON instances. We can access specific fields in parsedJson
using XPath-like syntax (if you are not familiar with XPath-like syntax, you might want to read Chapter 6, Slick – A Functional Interface for SQL):
scala> parsedJson "name" play.api.libs.json.JsLookupResult = JsDefined("dotty")
XPath-like lookups return an instance with type JsLookupResult
. This takes two values: either JsDefined
, if the path is valid, or JsUndefined
if it is not:
scala> parsedJson "age" play.api.libs.json.JsLookupResult = JsUndefined('age' is undefined on object: {"name":"dotty","size":150,"language":"Scala","fork":true})
To go from a JsLookupResult
instance to a String in a type-safe way, we can use the .validate[String]
method:
scala> (parsedJson "name").validate[String] play.api.libs.json.JsResult[String] = JsSuccess(dotty,)
The .validate[T]
method returns either JsSuccess
if the JsDefined
instance could be successfully cast to T
, or JsError
otherwise. To illustrate the latter, let's try validating this as an Int
:
scala> (parsedJson "name").validate[Int] dplay.api.libs.json.JsResult[Int] = JsError(List((,List(ValidationError(List(error.expected.jsnumber),WrappedArray())))))
Calling .validate
on an instance of type JsUndefined
also returns in a JsError
:
scala> (parsedJson "age").validate[Int] play.api.libs.json.JsResult[Int] = JsError(List((,List(ValidationError(List('age' is undefined on object: {"name":"dotty","size":150,"language":"Scala","fork":true}),WrappedArray())))))
To convert from an instance of JsResult[T]
to an instance of type T
, we can use pattern matching:
scala> val name = (parsedJson "name").validate[String] match { case JsSuccess(n, _) => n case JsError(e) => throw new IllegalStateException( s"Error extracting name: $e") } name: String = dotty
We can now use .validate
to cast JSON to simple types in a type-safe manner. But, in the code example, we used .validate[Repo]
. This works provided a Reads[Repo]
type class is implicitly available in the namespace.
The most common way of defining Reads[T]
type classes is through a DSL provided in import play.api.libs.functional.syntax._
. The DSL works by chaining operations returning either JsSuccess
or JsError
together. Discussing exactly how this DSL works is outside the scope of this chapter (see, for instance, the Play framework documentation page on JSON combinators: https://www.playframework.com/documentation/2.4.x/ScalaJsonCombinators). We will stick to discussing the syntax.
scala> import play.api.libs.functional.syntax._ import play.api.libs.functional.syntax._ scala> import models.Repo import models.Repo scala> implicit val readsRepoFromGithub:Reads[Repo] = ( (JsPath "name").read[String] and (JsPath "language").read[String] and (JsPath "fork").read[Boolean] and (JsPath "size").read[Long] )(Repo.apply _) readsRepoFromGithub: play.api.libs.json.Reads[models.Repo] = play.api.libs.json.Reads$$anon$8@a198ddb
The Reads
type class is defined in two stages. The first chains together read[T]
methods with and
, combining successes and errors. The second uses the apply method of the companion object of a case class (or Tuple
instance) to construct the object, provided the first stage completed successfully. Now that we have defined the type class, we can call validate[Repo]
on a JsValue
object:
scala> val repoOpt = parsedJson.validate[Repo] play.api.libs.json.JsResult[models.Repo] = JsSuccess(Repo(dotty,Scala,true,150),)
We can then use pattern matching to extract the Repo
object from the JsSuccess
instance:
scala> val JsSuccess(repo, _) = repoOpt repo: models.Repo = Repo(dotty,Scala,true,150)
We have, so far, only talked about validating single repos. The Play framework defines type classes for collection types, so, provided Reads[Repo]
is defined, Reads[List[Repo]]
will also be defined.
Now that we understand how to extract Scala objects from JSON, let's get back to the code. If we manage to successfully convert the repositories to a List[Repo]
, we emit it again as JSON. Of course, converting from GitHub's JSON representation of a repository to a Scala object, and from that Scala object directly to our JSON representation of the object, might seem convoluted. However, if this were a real application, we would have additional logic. We could, for instance, store repos in a cache, and try and fetch from that cache instead of querying the GitHub API. Converting from JSON to Scala objects as early as possible decouples the code that we write from the way GitHub returns repositories.
The last bit of the code sample that is new is the call to Action.async
, rather than just Action
. Recall that an Action
instance is a thin wrapper around a Request => Result
method. Our code, however, returns a Future[Result]
, rather than a Result
. When that is the case, use the Action.async
to construct the action, rather than Action
directly. Using Action.async
tells the Play framework that the code creating the Action
is asynchronous.