Chapter 19. Databases

WHAT'S IN THIS CHAPTER?

  • Implementing basic CRUD operations

  • Understanding how ORMs and F# interact

  • Creating your own mapping layer

  • Implementing DDL functionality

There are few, if any, interesting applications that do not use data in one way, shape, or form. If we are to write interesting programs in F#, at some point, it is likely a database will be involved. This chapter explains how databases can be accessed using F# and walks through an example framework for dealing with data in F#.

OVERVIEW

One of the core strengths of F# is processing large sets of data and doing interesting things with it. Most examples where F# was used early on were around things like computation of XBox 360 TrueSkill (http://blogs.technet.com/apg/archive/2008/06/16/trueskill-in-f.aspx). This case, which involves taking a great deal of information gathered from players of Xbox 360 consoles and using said information to produce rankings that allow for smarter matching of players when engaging in multiplayer games, is a canonical use case where F# shines. These cases — where lots of incoming raw information can be processed by complex algorithms to generate business value — are well suited to the F# language.

One thing you might start to notice are the similarities between some of the declarative properties of F# and the declarative nature of the primary languages used for database interaction, Structured Query Language. For example, the select construct in SQL is quite similar to the Seq.map construct in F#. The same can be said for (no pun intended) mapping the concept of where to Seq.filter, and various other constructs based on Seq. F#, when examined closely, is ideally situated as a functional language that has all the power of what you would do with SQL, with the added bonus that it is easily leveraged on the .NET framework. It allows separation of code that works with data (using F#), from code that retrieves data (using SQL).

In this chapter, various ways to work with F# will be demonstrated. It covers core fundamentals using core ADO.net libraries, what happens when you introduce an ORM into the mix, and then explores some possibilities around more F#-friendly ways to work with data.

RETRIEVING DATA USING ADO.NET

There is little you can do without at least fundamentally getting some data in the first place. Thankfully, there are a multitude of means by which you can retrieve data on the .NET framework. The most fundamental way, however, is to simply use ADO.NET primitives to accomplish this task. Assume the following exist:

  • A local instance of Microsoft SQL Server 2008

  • Integrated security, with the current user having administrative access to the local server

  • A local database called DemoData

Creating a Database Connection

Under those assumptions, a database connection can be created with the following code:

module RawADOExample
open System.Data
open System.Data.SqlClient

//raw ADO example
let readPeople =
  use connection =
    new SqlConnection(
      "Server=localhost;Integrated security=SSPI;database=DemoData")
                                                          
Creating a Database Connection

Most people familiar with other .NET languages will not find the preceding line of code that different than a similar line of code in C# that accomplishes the same thing. Of course, it is important to note that like many other objects that use external resources and implement IDisposable to have a means to release said resources, it is important to use the use binding when instantiating them. This is analogous to putting code in a using block in C#.

Creating connections to other data sources is a matter of finding other concrete classes that support the IDbConnection interface, of which there are a robust number of options if you prefer to work with something other than MS SQL Server.

Reading Data

Of course, a connection alone is not going to do much all by itself. Performing a query requires a command object that holds the query. The query itself is SQL code, which for this example, can be used in string form:

module RawADOExample
open System.Data
open System.Data.SqlClient

//raw ADO example
let readPeople =
  use connection = new SqlConnection(
    "Server=localhost;Integrated security=SSPI;database=DemoData")
  use command = new SqlCommand("select firstname, lastname from person",connection)
                                                          
Reading Data

Again, this is not that different than code you would write in any other .NET language for reaching a database. Nothing really interesting has yet been done, however. If the desire is to retrieve some actual data, then the command is going to need to be executed. If an assumption is made that the firstname and lastname columns of the person table can be read as strings (that is, varchar, char, or something like that), the following record type can be used as a container:

type Person = {Id:int;FirstName:string;LastName:string}

The next step is to actually run the query, take the results, and populate a list of Person records:

module RawADOExample
open System.Data
open System.Data.SqlClient

type Person = {Id:int;FirstName:string;LastName:string}
//raw ADO example
let readPeople =
  use connection =
   new SqlConnection("Server=localhost;Integrated security=SSPI;database=DemoData")
  connection.Open()
  use command = new SqlCommand("select firstname, lastname from person",connection)
  use rawResult = command.ExecuteReader()
  let people = seq {
        while rawResult.Read() do
          yield {
            Id = rawResult.["FirstName"] :?> int;
            FirstName = rawResult.["FirstName"] :?> string;
            LastName = rawResult.["LastName"] :?> string
          }
      }
  people |> Seq.toList
                                                            
Reading Data

For those familiar with how ADO.NET works, this is a familiar pattern. A call to command.ExecuteReader() is made, which provides a SqlDataReader through which data can be accessed. In the example, there is an F# spin on the old while-Read()-next loop. In this case, the result is compiled into a sequence expression that specifies how the loop can be iterated through and reads the results into Person records. It is important to note that, like all sequence definitions, nothing has actually happened yet — this code merely specifies how the sequence is to be read. It does not actually read it until the next line, people |> Seq.toList, which iterates through the people sequence and materializes an actual list of Person records.

Filtering Data

As simple as it may seem to simply do a Seq.filter operation on the result of selecting for all the records in a given table at once, such operations do not scale for anything but very small data sets. As a result, when working with larger databases, the need to let the database do some of the work of filtering data right at the source will present itself. This is, of course, what databases do best, and it would be pretty silly not to let the database help in the task of querying data.

For this sample, it is going to be assumed that the goal is to implement a query for people based on the FirstName column in the database. Such code needs to add a parameter object into the query that specifies which first name is to be queried for:

let retrievePeopleNamed firstName =
  use connection =
   new SqlConnection("Server=localhost;Integrated security=SSPI;database=DemoData")
  connection.Open()
  use command =
    new SqlCommand("select firstname, lastname from person where firstname =
@firstname",connection)
  let parameter = new SqlParameter("firstName",firstName)
  do command.Parameters.Add parameter |> ignore
  let rawResult = command.ExecuteReader()
  let people = seq {
        while rawResult.Read() do
          yield {
            Id = rawResult.["FirstName"] :?> int;
            FirstName = rawResult.["FirstName"] :?> string;
            LastName = rawResult.["LastName"] :?> string
          }
      }
  people |> Seq.toList
                                                             
Filtering Data

The first key difference is the inclusion of the @firstname literal within the string used to build the command:

use command =
  new SqlCommand("select firstname, lastname from person where
firstname = @firstname",connection)

The @firstname literal is a placeholder for where the parameter will go. This approach of using parameters is important because without them, SQL injection bugs are almost always introduced:

use command = new SqlCommand("select firstname, lastname from person where
firstname = " + firstName, connection)

If a user decides to be clever and submit a firstName of '; drop database, it is not an understatement to imagine that there could be some serious issues!

When there is a query and placeholder in place, parameters can be added:

let parameter = new SqlParameter("firstName",firstName)
do command.Parameters.Add parameter |> ignore

Insert, Update, and Delete

Using the same concept of commands and parameters, the next step is to implement the other three main database operations. The following code demonstrates how, on the vast majority of databases, we might do those operations:

let doCreateUpdateDelete() =
  use connection =
   new SqlConnection("Server=localhost;Integrated security=SSPI;database=DemoData")
  connection.Open()

  //create
  use createCommand =
    new SqlCommand("insert into person (firstname,lastname) values
(@firstname,@lastname)",connection)
  let firstNameParameterCreate = new SqlParameter("firstName","Aaron")
  let lastNameParameterCreate = new SqlParameter("lastName","Erickson")
  do createCommand.Parameters.Add firstNameParameterCreate |> ignore
  do createCommand.Parameters.Add lastNameParameterCreate |> ignore
  do createCommand.ExecuteNonQuery() |> ignore

  //update
  use updateCommand =
    new SqlCommand("update person set firstname=@firstname, lastname=@lastname
where id=@id", connection)
  let firstNameParameterUpdate = new SqlParameter("firstName","Not")
  let lastNameParameterUpdate = new SqlParameter("lastName","Sure")
  let idParameterUpdate = new SqlParameter("id",42)
  do updateCommand.Parameters.Add firstNameParameterUpdate |> ignore
  do updateCommand.Parameters.Add lastNameParameterUpdate |> ignore
  do updateCommand.Parameters.Add idParameterUpdate |> ignore
  do updateCommand.ExecuteNonQuery() |> ignore

  //delete
  use deleteCommand = new SqlCommand("delete person where id=@id", connection)
  let idParameterDelete = new SqlParameter("id","42")
  do deleteCommand.Parameters.Add idParameterDelete |> ignore
  do deleteCommand.ExecuteNonQuery() |> ignore
                                                             
Insert, Update, and Delete

The only difference between the read examples and these are the use of .ExecuteNonQuery(), which is the standard way to perform operations that do not return records from a database using ADO.NET. Outside of that, the exact same techniques for inserting parameters into standard SQL queries apply.

One striking thing, however, about using raw ADO.net in any language, F# included, is the raw volume of code required to do, what are in essence, really simple tasks. When one considers that the entire code base of the Microsoft TrueSkill system is around 100 lines of F# code, having 15 lines of code to do three simple operations seems excessive. Thankfully, there are frameworks built on top of ADO.net to help deal with this complexity.

F# AND OBJECT RELATIONAL MAPPING

The first place someone who is frustrated with the verbosity of raw ADO.NET might look to get some relief is to the many Object/Relational Mapping layers out there, such as NHibernate or the Microsoft Entity Framework. Despite the general conclusion in Ted Neward's Famous Essay, "The Vietnam of Computer Science" (http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx), for many cases, ORMs are a great solution for many types of problems.

You need to remember what the 'O' in ORM stands for — namely that it expects objects. Objects, of course, tend to have a state that is typically assumed to be mutable. Further, most ORMs require that properties be virtual so that their implementation can be overridden to do things like lazy loading.

The following is a typical implementation of the Person record from the previous examples in this chapter if you were going to use it with NHibernate:

type NHPerson() = class
  let mutable _id : int = 0
  let mutable _firstName : string = ""
  let mutable _lastName : string = ""

  abstract Id : int with get, set
  default x.Id with get() = _id and set(v) = _id <- v

  abstract FirstName : string with get, set
  default x.FirstName with get() = _firstName and set(v) = _firstName <- v

  abstract LastName : string with get, set
  default x.LastName with get() = _lastName and set(v) = _lastName <- v
end
                                                         
F# AND OBJECT RELATIONAL MAPPING

The situation in Entity Framework v4 (the first version to support POCO objects) is quite similar, with the only important difference being the lack of a need for doing the abstract declarations for properties that are not going to be lazy loaded. Nearly all ORMs have a "special thing" of one type or another that need to be done to objects to work with the ORM.

In either case there are problems. Most ORM systems depend on mutability. The internal mechanism of ORM systems for object construction tends to be something like:

  1. Create object, usually through a parameterless constructor.

  2. Set properties based on a configuration file mapping — or perhaps a fluent convention based mapping à la FluentNHibernate.

  3. Use a dynamic proxy mechanism that will intercept access to lazily loaded properties, submitting additional queries to load said properties.

As has been covered before, F# programs generally avoid mutability. This is especially true if the reason that mutability is allowed is solely to support an ORM. If one of the reasons we consider F# is to move away from mutability, having to introduce it to make an ORM work is actually a step backward. Generally speaking, unless there is an externally defined requirement to use an ORM, we recommend that software developers avoid using ORM technologies with F#.

INTRODUCING F# ACTIVE RECORD (FAR)

F# Active Record (FAR) is an Open Source Project I founded that makes it possible to get ORM-like features, but in a way that uses convention over configuration, while embracing an approach more idiomatic to F#.

The idea with FAR is to make mapping from rows in a database to records in F# drop-dead easy, while retaining a functional style of programming. It is not designed to cover everything that we might ever want to do with a database. In fact, it is biased toward simplicity, leaving out edge case features in favor of making the included functionality very robust and predictable.

Reading Data

For example, consider the following Person record definition:

type Person = {Id:int;FirstName:string;LastName:string;Age:int}

Code for populating a sequence of Person objects should look something like this:

use context = new ForSqlContext("SomeConnectionString")
let people = context.SequenceFrom<Person>()

In this case, people is now a seq<Person> that further F# code can do work with. It is assumed that there is a table name in the targeted database that matches the type name of Person, and that there are columns in said table that correspond to the record properties of Id, FirstName, LastName, and Age. This approach works so long as a convention that type names match table names, and property names match column names, holds true. Although this obviously can't work in all situations, there are a great number of cases where it does. It is those situations where the convention can be maintained where this kind of approach can result in a drastic drop in the amount of mapping code in the system.

For more on convention over configuration, please see http://en.wikipedia.org/wiki/Convention_over_configuration.

In a case where the Person table has a smaller number of rows, this approach works particularly well. Filtering on a small set is a matter of applying Seq.filter with an appropriate predicate to the following result.

A predicate is a function that returns true or false. A filtering predicate in this context needs to start with one parameter of the type we are filtering and resolve to a Boolean value:

let me = people |> Seq.filter( fun p -> p.FirstName = "Aaron" )

This, however, is not an approach that will scale. If people is something more like the list of people in the United States Social Security database, rather than the list of employees in a small company, bringing that sequence out of the database and into memory to analyze it is not the most scalable approach.

Querying Data

To not have to bring all the records into memory, a need will exist for a way to easily tell the system that the predicate — the part where it is specified p -> p.FirstName = "Aaron" — should be analyzed in the database. Using FAR, the technique for doing so looks like this:

use context = new ForSqlContext("SomeConnectionString")
let me = context.SequenceFrom<Person>( <@ fun p -> p.FirstName = "Aaron" @>  )

Notice two key differences in this version compared to the previous one. The first one is that the predicate is being passed to the SequenceFrom method of the context object. This method's purpose is to analyze the predicate passed in, convert that predicate to the appropriate raw SQL code that will be used for the query on the database management system, and retrieve the result in a manner similar to the version of SequenceFrom that has no parameters and simply returns everything.

The other key difference is that the predicate was quoted (using <@ @> syntax). Quoting the predicate is necessary in this case, as the SequenceFrom method needs to analyze the predicate to evaluate it and generate a database query that will ultimately run against the database. Remember, one of the key reasons for quotation is to execute code "by other means." In a case like this, the desire is to execute the predicate against the data. For that to happen, the predicate has to be converted to a form that is useful where the query will occur, which happens to be in the database, not in F#.

Adding Data

Creating new records in the database is nearly as simple as reading them:

type Person = {Id:int;FirstName:string;LastName:string;Age:int}
type Cat = {PetName:string;Color:string;IsCute:bool;IsMean:bool}

[<Test>]
static member TestReadPeopleAndPets =

  use context = new ForDataContext("SomeConnectionString")
  do context.Create {Id=1;FirstName="Aaron";LastName="Erickson";Age=37} |> ignore
  do context.Create {Id=2;FirstName="Erin";LastName="Erickson";Age=34} |> ignore
  do context.Create {Id=3;FirstName="Adriana";LastName="Erickson";Age=13} |> ignore
  do context.Create {Id=4;FirstName="Matthew";LastName="Erickson";Age=8} |> ignore
let people =
    context.SequenceFrom<Person>( <@ fun p -> p.LastName = "Erickson" @> )
      |> Seq.toArray

  Assert.AreEqual(people.Length,4)

  do context.Create {PetName="Puppy Cat";Color="Ginger";IsCute=true;IsMean=false}
    |> ignore
  do context.Create {PetName="Dmitry";Color="Blue-Gray";IsCute=true;IsMean=true}
    |> ignore

  let theCats =
    context.SequenceFrom<Cat>() |> Seq.toArray

  Assert.AreEqual(theCats.Length,2)
                                                           
Adding Data

In this example, the Person and Cat types are declared using simple F# record types. Using the ForDataContext that is created for when you want to read data, the Create method can be used to add rows to the database.

The Create method is generic, in that it can detect the type of record passed to it and use that record's type to determine which table it should insert the record to. The Create method returns a number of rows affected, which in most operations where you are creating a row, should tend to resolve to 1. In the preceding example, because the result is not needed for further processing, it is simply passed to ignore, and the next record is added.

Updating Data

Update, like create, uses a convention-based minimalist approach:

do context.Update {Id=3;FirstName="Adriana";LastName="Erickson";Age=13} |> ignore

Update uses a convention that the first property that ends in Id is expected to be, for database purposes, the identity upon which the update will be based. It would roughly translate to the following SQL code:

update person set firstname='Adriana', lastname='Erickson', age=13 where id=3

If the Id column in the database were personid, rather than simply id, so long as personid is the first field that ends in id in the record, the following would also work:

do context.Update {PersonId=3;FirstName="Adriana";LastName="Erickson";Age=13}
  |> ignore

Deleting Data

As you might expect, deletion does not stray from our simplistic formula:

do context.Delete {Id=4;FirstName="Matthew";LastName="Erickson";Age=8} |> ignore

The preceding code specifies a query that, in this case would roughly translate to:

delete from person where id=4

Technically, for delete, only the Id field is needed. However, because the record type defines the table, and the record does not allow null values in the other fields, the rest of the record is specified in the example.

What Isn't Supported

One of the reasons the library can remain as simple as it does is that it is explicitly stateless. There is no change tracking, no concept of attach/detach, and no pretense that the library is something other than a means to act as either a producer or consumer of F# record types. Other things that make more traditional ORM systems complex, such as configuration files, mappings, and base classes to inherit from, are also avoided in favor of simplicity and predictability in terms of how it works.

Coming Soon

At the time of this writing, there are some features that have yet to be implemented but should be complete by the time of publication. Support for stored procedures, inner joins, and a broader set of predicate types are in the works. Progress on this project can be followed at http://github.com/ericksoa/FAR.

HOW FAR WORKS

One of the principles of being a good software developer is to have at least some level of understanding about how the libraries work. As such, this section explores the internals of the FAR library.

Dependencies

To do our decomposition of a query, .NET reflection will be used, as well as various elements of Microsoft.FSharp.Quotations. As well, System.Data.SqlClient will be used so that the appropriate connections and commands can be created. Although future versions may refactor this by extracting out database specific concerns from parsing concerns, for now, it will be assumed that the target database is Microsoft SQL Server:

module FSharpActiveRecord
open System
open System.Reflection
open System.Data.SqlClient
open Microsoft.FSharp.Quotations
open Microsoft.FSharp.Quotations.DerivedPatterns
                                                          
Dependencies

Utility Routines

The next two routines provide a wrapper behind creation of typical ADO.NET primitives that will be used across various specific methods in the library:

let MakeConnection connectionString =
  new SqlConnection(connectionString)

let MakeCommand commandText connection =
  new SqlCommand(commandText, connection)

These functions provide a means for various routines within the system that need connections without having to; at least in the method itself, depend on System.Data.SqlClient specifics.

The next routine is an important utility that allows you to take a sequence of strings and return a comma-separated list:

//ConvertToCommaSeparatedString courtesy of Mark Needham, ThoughtWorks
let ConvertToCommaSeparatedString (value:seq<string>) =
    let rec convert (innerVal:List<string>) acc =
        match innerVal with
            | [] -> acc
            | hd::[] -> convert [] (acc + hd)
            | hd::tl -> convert tl (acc + hd + ",")
    convert (Seq.toList value) ""
                                                                
Utility Routines

When generating SQL, a frequent need is to generate comma-separated lists of things such as column names and parameters. Because this list should not have a comma at the end of the list, something is needed that distinguishes between the last item and everything else when generating the list. This routine fits the bill rather nicely.

The next thing that is needed, if you generate a lot of SQL, is something that converts primitive F# types to strings that reflect their SQL counterparts:

let FSharpTypeToSqlType fSharpType size =
  match fSharpType, size with
  | t, Some(s) when t = typeof<string> -> sprintf "varchar(%i)" s
  | t, _ when t = typeof<string> -> "varchar(255)"
  | t, _ when t = typeof<int> -> "int"
  | t, _ when t = typeof<bool> -> "bit"
  | t, _ when t = typeof<DateTime> -> "datetime"
  | _, _ -> raise( new NotSupportedException() )
                                                               
Utility Routines

This routine takes a type (as a .NET Type object) and a size as an int option. Based on combinations of type and size, it generates a SQL type signature. This type of routine is especially useful in cases where we need to drive SQL types from .NET types, as is done in the creation of tables in a database.

Table Creation

This is especially useful for cases when the desire is to create a table based on a record type:

let private CreateTable tableName columns connectionString =
  let columnList =
    columns
    |> Seq.map (fun (name, colType, size) ->
      sprintf "%s %s" name (FSharpTypeToSqlType colType size))
    |> ConvertToCommaSeparatedString
  let query = sprintf "create table %s (%s)" tableName columnList
  use connection = connectionString |> MakeConnection
  use createCommand = MakeCommand query connection
  do connection.Open() |> ignore
  do createCommand.ExecuteNonQuery() |> ignore

let CreateTableFor<'a> connectionString =
  let tableName = typeof<'a>.Name
  let columnSpecSelector (p:PropertyInfo) = (p.Name,p.PropertyType,None)
  let columnSpecs = typeof<'a>.GetProperties() |> Seq.map columnSpecSelector
  do CreateTable tableName columnSpecs connectionString
                                                              
Table Creation

CreateTableFor is a generic function that takes a record type that will be used as a template for creating a table and a connection string used to reach an actual database. Using .NET reflection, it uses Seq.map to create a set of (type * name * size) tuples (columnSpecs). It passes the tableName (from the generic type), the columnSpecs, and the connection string to a private CreateTable method that does the work to convert things into a SQL create table script that can be executed against a database.

The job of generating SQL is made easy by some of the utility functions that are already in place. The column list part of the create script, usually taking the form of something like somename sometype somesize, anothername anothertype anothersize is handled by mapping the tuples through FSharpToSqlType and formatting them using sprintf.

From there, the next step is to format the rest of the SQL, putting together the table and columnList and then do typical ADO.NET work of creating a connection and command needed to run the actual query.

The FAR library contains many routines such as CreateTable that simplify database creation, destruction, and other similar tasks that are especially important in setup and teardown of integration tests. Please see the book's sample code or the FAR library website for more details on these routines.

Query Processing

The real useful part of this library is the conversion of idiomatic F# code to SQL. However, for this to work, the following is needed:

let makeSimpleSelect table fields =
  let commaSeperatedFields = fields |> ConvertToCommaSeparatedString
  sprintf "select %s from %s" commaSeperatedFields table

This first routine takes a table name and a seq<string> that represents database fields. It then generates a typical SQL query that we would use against a single table. For example, passing Person and [FirstName,LastName] would generate:

select firstname,lastname from person

This is a good start, but far more needs to be done if you want the ability for more complex predicates to be passed along with a record type. More complex queries require that an expression be parsed and tree structure be built that represents it so it can be converted to SQL. Start by defining the following tree structure in the form of a discriminated union:

type ParseNode =
  | EqualNode of Type list * Expr list
  | AndNode of ParseNode option * ParseNode option
  | OrNode of ParseNode option * ParseNode option
//other nodes as needed

The preceding example is a simplistic version of what can represent a parse tree. It will start with equality tests as terminal nodes in the tree, which can be combined with various combinations of the terms and or or. For example p.FirstName = "Aaron" and (p.LastName = "Erickson" or p.LastName="Burr") might look like:

---------AndNode-------
  |                     |
EqualNode               OrNode--------------------
p.FirstName = "Aaron"   |                        |
                        p.LastName = "Erickson"  p.LastName = "Burr"

Two routines will use this tree. One has to take an F# quoted predicate and generate the tree; the other will recursively walk the tree to generate equivalent SQL code.

Generating the Parse Tree

Start by declaring a context from which we will run queries that contain the connection object:

type ForDataContext =
  val connection : SqlConnection
  new(connectionString) = { connection = connectionString |> MakeConnection }
  with
    interface IDisposable with
      member disposable.Dispose() =
        disposable.connection.Close()

Within this type, members are created that depend on this common context. The first is the routine that generates the parse tree:

//takes a predicate and generates a tuple composed of
//  parameterized sql * parameters
static member private ParseCriteria<'a> (criteria:Expr<'a -> bool>) =

  let rec predicateParser expr =
    match expr with
      | SpecificCall <@ (=) @> (optionExpr, types, exprs)
        -> Some(EqualNode(types,exprs))
      | SpecificCall <@ (&&) @> (optionExpr, types, exprs)
        -> Some(AndNode(predicateParser(exprs.[0]),predicateParser(exprs.[1])))
      | SpecificCall <@ (||) @> (optionExpr, types, exprs)
        -> Some(OrNode(predicateParser(exprs.[0]),predicateParser(exprs.[1])))
      | Patterns.IfThenElse (left, middle, right)
        -> match middle with
              | Patterns.Call(optionExpr, types, exprs)
                -> Some(AndNode(predicateParser(left), predicateParser(middle)))
              | Patterns.Value(value, valueType)
                -> Some(OrNode(predicateParser(left), predicateParser(right)))
              | _ -> None
        | _ -> None
                                                                
Generating the Parse Tree

Our overall goal with ParseCriteria is to take a quoted predicate and generate a SQL string and a series of ordered parameters that will be applied to that string to perform a query. This parsing is made much simpler by inclusion of the Microsoft.FSharp.Quotations and Microsoft.FSharp.Quotations.DerivedPatterns namespaces. Most of the work is done by the recursive predicateParser routine, which recursively walks the provided predicate, mapping the expression tree to something we can more easily consume.

SpecificCall is an active pattern that matches a given quoted expression (such as <@ (=) @>) against what the tree contains. In F#, this works very well for the EqualNode type, which when matched, emits an EqualNode that contains a pair of expressions and types that can be used for generating an equals SQL expression.

The other instances of SpecificCall work in certain circumstances where one would expect them to, and if they are present, will emit the appropriate AndNode or OrNode. AndNode and OrNode both take two arguments, for their left and right side, which will be provided by recursive calls to predicateParser on the respective left and right sides of the provided expression tree.

Of course, there is a special case, which is that sometimes optimizations get applied to the tree. If the quoted and or quoted or get converted for some reason into IfThenElse calls in the predicate, those need to be handled as well. IfThenElse is another active pattern that can be used to match against for that case. And and Or both get converted to IfThenElse in certain ways that are recognizable. If the middle node in IfThenElse is a Call, it can be deduced that it was converted from (&&), and be handled accordingly. If it is a Value, it can be deduced it was converted from (||).

The result of all this, provided that the expression matches the rules, is a tree that we can much more cleanly generate SQL from, as the following demonstrates:

let paramEnumerator =
  let paramNames = 1
    |> Seq.unfold (fun i -> Some(i+1,i))
    |> Seq.map( fun i -> sprintf "param%i" i)
  paramNames.GetEnumerator()

let nextParam() =
  paramEnumerator.MoveNext() |> ignore
  paramEnumerator.Current

let paramList = new System.Collections.Generic.List<string * obj>()

let rec queryString (treeNode:ParseNode option) =
  match treeNode with
  | Some(node) ->
    match node with
    | EqualNode (left,right) ->
      match (right.[0],right.[1]) with
      | (Patterns.PropertyGet (option,property,someList),
          Patterns.Value(value, valueType)) ->
        let p = nextParam()
        do (p,value) |> paramList.Add
        sprintf "%s = @%s" property.Name p
      | _ -> raise(new InvalidOperationException())
    | AndNode (left,right)
         -> sprintf "(%s and %s)" (queryString left) (queryString right)
    | OrNode (left,right)
        -> sprintf "(%s or %s)" (queryString left) (queryString right)
  | None -> ""
                                                         
Generating the Parse Tree

There are a couple of things that need to be done here. The first step is to generate SQL that will be transformed into the where clause at the end of the overall SQL query. The next step is to generate a set of parameter objects that will be used with the ADO.NET command object to thread the query parameters together.

To do the parameter list, it will be necessary to use a mutable list structure that can be added to and that holds the parameter names and values. Also needed will be something that generates unique parameter names. These parameters need to be generated on an on-demand basis while parsing is taking place. The paramEnumerator is a utility that can be used to act as something of a dispenser for parameters that are guaranteed to be unique to the query. The nextParam() function executes this generator, as is used by queryString as it is doing its work. paramList is then used to accumulate these parameters as queryString does its work.

The real work of queryString is to handle the various node cases, and either just generate SQL (in the case of an EqualNode), or — in the case of AndNode or OrNode — take the result of recursive queryString calls and compound the results together into the appropriate syntax.

This is all brought together in the full implementation of ParseCriteria:

//takes a predicate and generates a tuple composed of
//  parameterized sql * parameters
static member private ParseCriteria<'a> (criteria:Expr<'a -> bool>) =

  let rec predicateParser expr =
    match expr with
      | SpecificCall <@ (=) @> (optionExpr, types, exprs)
        -> Some(EqualNode(types,exprs))
      | SpecificCall <@ (&&) @> (optionExpr, types, exprs)
        -> Some(AndNode(predicateParser(exprs.[0]),predicateParser(exprs.[1])))
      | SpecificCall <@ (||) @> (optionExpr, types, exprs)
        -> Some(OrNode(predicateParser(exprs.[0]),predicateParser(exprs.[1])))
      | Patterns.IfThenElse (left, middle, right)
        -> match middle with
              | Patterns.Call(optionExpr, types, exprs)
                -> Some(AndNode(predicateParser(left), predicateParser(middle)))
              | Patterns.Value(value, valueType)
                -> Some(OrNode(predicateParser(left), predicateParser(right)))
              | _ -> None
        | _ -> None

  let parsedResult =
    match criteria with
      | Patterns.Lambda(var,lambda) -> predicateParser lambda
      | _ -> None

  let paramEnumerator =
    let paramNames = 1
      |> Seq.unfold (fun i -> Some(i+1,i)) |> Seq.map( fun i
               -> sprintf "param%i" i)
    paramNames.GetEnumerator()

  let nextParam() =
    paramEnumerator.MoveNext() |> ignore
    paramEnumerator.Current

  let paramList = new System.Collections.Generic.List<string * obj>()

  let rec queryString (treeNode:ParseNode option) =
    match treeNode with
    | Some(node) ->
      match node with
      | EqualNode (left,right) ->
        match (right.[0],right.[1]) with
        | (Patterns.PropertyGet (option,property,someList),
             Patterns.Value(value, valueType)) ->
          let p = nextParam()
          do (p,value) |> paramList.Add
          sprintf "%s = @%s" property.Name p
        | _ -> raise(new InvalidOperationException())
      | AndNode (left,right)
          -> sprintf "(%s and %s)" (queryString left) (queryString right)
      | OrNode (left,right)
-> sprintf "(%s or %s)" (queryString left) (queryString right)
      | None -> ""

    queryString parsedResult, paramList.ToArray()
                                                             
Generating the Parse Tree

Of course, a couple more utility functions are needed to make the context work. A connection needs to be opened if it isn't open yet. (Remember, we can run multiple queries from the same context.) Also needed is a way to easily take the SQL and command objects and execute them:

member private context.OpenConnectionIfNeeded() =
  if (context.connection.State <> Data.ConnectionState.Open)
    then context.connection.Open()

//core routine that takes sql + parameters and yields readers that
//  eventually get composed into records we want to work with
member private context.DoQuery query (rawParameters:array<string * #obj>) =
  use command = MakeCommand query context.connection
  do rawParameters
    |> Array.map( fun r -> new SqlParameter(fst r,snd r) )
    |> Array.iter( fun p -> command.Parameters.Add p |> ignore )
  do context.OpenConnectionIfNeeded()
  let reader = command.ExecuteReader()
  seq { while reader.Read() do yield reader }

member private context.DoCommand command (rawParameters:array<string * #obj>) =
  use command = MakeCommand command context.connection
  do rawParameters
    |> Array.map( fun r -> new SqlParameter(fst r,snd r) )
    |> Array.iter( fun p -> command.Parameters.Add p |> ignore )
  do context.OpenConnectionIfNeeded()
  command.ExecuteNonQuery()
                                                            
Generating the Parse Tree

These methods use similar techniques that were used when doing raw ADO for creating and using connections and commands. The only difference is they do so in a much more generic fashion. With these utility methods in place, the next step is to go forth and implement the query interface:

//core routine that composes and executes the query
member private context.SequenceFrom<'a>
  ((whereClause:string),(parameters:array<string * #obj>)) =
  let tableName = typeof<'a>.Name
  let memberNameSelector (m:#MemberInfo) = m.Name
  let propertyNames = typeof<'a>.GetProperties() |> Seq.map memberNameSelector
  let query = (makeSimpleSelect tableName propertyNames) + " " + whereClause
  let creator = Reflection.FSharpValue.PreComputeRecordConstructor(
     typeof<'a>,BindingFlags.Public)
  let data = context.DoQuery query parameters
  let readObjectsFromReaderByField (reader:SqlDataReader) (keys:seq<string>) =
    keys |> Seq.map( fun(k) -> reader.[k] ) |> Seq.toArray
  data |> Seq.map(
fun r -> creator(readObjectsFromReaderByField r propertyNames) :?> 'a )

//Performs queries where you have (for now) simple object.Property = someValue
member context.SequenceFrom<'a> (criteria:Expr<'a -> bool>) =
  let queryAndParams = criteria |> ForDataContext.ParseCriteria
  context.SequenceFrom<'a> (
    (sprintf "where %s" (fst queryAndParams)),(snd queryAndParams))
                                                           
Generating the Parse Tree

The public version takes the quoted predicate criteria, generates a SQL query and parameter set, and calls the private SequenceFrom, which expects a where clause in string form, as well as an array of parameter name and object tuples that will be used for parameters.

In the private overload of SequenceFrom, the rest of the query will be built. F# reflection is used to build a mechanism (creator, in the preceding example) for creating F# records based on results we get returned from the SqlDataReader.

When a creator is set up, the next step is to call context.DoQuery that yields a DataReader that, when iterated, advances through the records. What is returned is a sequence iterator that converts each state of the SqlDataReader into the appropriate record type. What is notable about this is that it returns a seq<'a> for a reason. A user of the library could choose to take only the top five elements, and in doing so, will use the built in forward-only cursor functionality of MS SQL Server to avoid taking the entire set of rows over the wire for only the top five elements.

Wrapping up, for the convenience of people who want to read all the records, this overload is added:

//simple case where we are getting all the rows from a table
member context.SequenceFrom<'a>() =
  context.SequenceFrom<'a>(String.Empty,Array.empty)

Implementation of Other FOR Operations

Other operations are, thankfully, much simpler to implement. Take create for example:

member context.Create someObject =
  let tableName = someObject.GetType().Name
  let columnsAndValues =
    someObject.GetType().GetProperties() |> Seq.map(
      fun p -> (p.Name,p.GetValue(someObject,Array.empty)))
  let columnNames columnsValuePair =
    columnsValuePair
    |> Seq.map( fun pair -> fst pair)
    |> ConvertToCommaSeparatedString
  let convertToValueParameterBucket columnsValuePair =
    columnsValuePair
    |> Seq.map( fun pair -> sprintf "@%s" (fst pair))
    |> ConvertToCommaSeparatedString
  let query =
    sprintf
      "insert into %s (%s) values (%s)"
      tableName
(columnsAndValues |> columnNames)
          (columnsAndValues |> convertToValueParameterBucket)
      let parameters = columnsAndValues |> Seq.toArray
      context.DoCommand query parameters
                                                                
Implementation of Other FOR Operations

The common pattern for nonquery operations where a parse tree need not be analyzed is simply to do some reflection over the generic type, read the values from the provided record in some manner that matches a convention, construct a SQL command, and execute it. The implementations of delete and update differ only by the SQL that is generated; nearly everything else is the same.

The Use Case for F# Active Record

FAR, as a library, is meant to be a simple, minimalist, and illustrative implementation that covers a large, but not exhaustive, set of cases for how F# applications might access data. It does so in a manner that allows the library to be independent of the domain, avoiding inheritance or any other special "things" that need to be done to make a record work with FAR. If you have a record, and you have a table and columns that match up, FAR can work.

SUMMARY

In this chapter, we covered ways for F# programs to interact with data, particular in ways that will feel idiomatic to F# developers. You may choose to use raw ADO.NET, use F# Active Record, or use one of the more object-oriented frameworks. What is important, across all of these, is to minimize the amount of plumbing code you write, so you can focus on doing something with data, rather than simply gathering it.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset