So far, we've seen basic Cascalog predicates and queries. We saw queries that pull data from one source generator and maybe include one predicate test. In this recipe, we'll see several more complex queries.
For this recipe, we'll need the same project.clj
file and dependencies from the Initializing Cascalog and Hadoop for distributed processing recipe. We'll also use the Doctor Who companion data that we defined in that recipe. The source code for this data is available in the code for the book, and you can also download just the code from http://www.ericrochester.com/clj-data-analysis/data/companions.clj to create this dataset.
We'll start with simple queries and build up from there:
user=> (?<- (stdout) [?name ?dr]
(full-name ?c ?name) (doctor ?c ?dr))
…
RESULTS
-----------------------
Ace 7
Adam Mitchell 9
Adelaide Brooke 10
Adric 4
Adric 5
Amy Pond 11
Astrid Peth 10
…
This pulls each companion's full name from one table and the numbers of Doctors they accompanied from another table. The binding for the companion's key, ?c
, stays the same across all the generators and predicates for each row of the output. Also, notice that the second predicate (doctor ?c ?dr)
can generate more than one row of output for each value of ?c
, as it did for Adric.
user=> (?<- (stdout)
[?name ?dr ?actor ?tenure]
(full-name ?c ?name) (doctor ?c ?dr)
(actor ?dr ?actor ?tenure))
…
RESULTS
-----------------------
Ace 7 Sylvester McCoy 1987–89, 1996
Adam Mitchell 9 Christopher Eccleston 2005
Adelaide Brooke 10 David Tennant 2005–10
Adric 4 Tom Baker 1974–81
Adric 5 Peter Davison 1981–84
Amy Pond 11 Matt Smith 2010–present
Astrid Peth 10 David Tennant 2005–10
…
This time, we've reused the same query as the preceding one, but we've also included the table of actors who've played Doctor Who.
user=> (?<- (stdout) [?name]
(full-name ?c ?name) (doctor ?c ?dr)
(>= ?dr 9))
…
RESULTS
-----------------------
Adelaide Brooke
Amy Pond
Astrid Peth
Christina de Souza
Craig Owens
Donna Noble
…
modern
flag to each row in order to indicate whether they are from the modern era or not. We do this by assigning the test predicate's output to a variable binding using the :>
operator:user=> (?<- (stdout)
[?name ?modern]
(full-name ?c ?name) (doctor ?c ?dr)
(>= ?dr 9 :> ?modern))
…
RESULTS
-----------------------
Ace false
Adam Mitchell true
Adelaide Brooke true
Adric false
Adric false
Amy Pond true
Astrid Peth true
…
Out of the simple building blocks of generator and predicates, Cascalog makes it easy to build complex queries in an intuitive way. We'll see even more complex queries in the upcoming recipes, but composing generators and predicates and sharing bindings are the basis of all of them.