Implementing basic CRUD operations
Understanding how ORMs and F# interact
Creating your own mapping layer
Implementing DDL functionality
There are few, if any, interesting applications that do not use data in one way, shape, or form. If we are to write interesting programs in F#, at some point, it is likely a database will be involved. This chapter explains how databases can be accessed using F# and walks through an example framework for dealing with data in F#.
One of the core strengths of F# is processing large sets of data and doing interesting things with it. Most examples where F# was used early on were around things like computation of XBox 360 TrueSkill (http://blogs.technet.com/apg/archive/2008/06/16/trueskill-in-f.aspx
). This case, which involves taking a great deal of information gathered from players of Xbox 360 consoles and using said information to produce rankings that allow for smarter matching of players when engaging in multiplayer games, is a canonical use case where F# shines. These cases — where lots of incoming raw information can be processed by complex algorithms to generate business value — are well suited to the F# language.
One thing you might start to notice are the similarities between some of the declarative properties of F# and the declarative nature of the primary languages used for database interaction, Structured Query Language. For example, the select
construct in SQL is quite similar to the Seq.map
construct in F#. The same can be said for (no pun intended) mapping the concept of where to Seq.filter
, and various other constructs based on Seq
. F#, when examined closely, is ideally situated as a functional language that has all the power of what you would do with SQL, with the added bonus that it is easily leveraged on the .NET framework. It allows separation of code that works with data (using F#), from code that retrieves data (using SQL).
In this chapter, various ways to work with F# will be demonstrated. It covers core fundamentals using core ADO.net libraries, what happens when you introduce an ORM into the mix, and then explores some possibilities around more F#-friendly ways to work with data.
There is little you can do without at least fundamentally getting some data in the first place. Thankfully, there are a multitude of means by which you can retrieve data on the .NET framework. The most fundamental way, however, is to simply use ADO.NET primitives to accomplish this task. Assume the following exist:
A local instance of Microsoft SQL Server 2008
Integrated security, with the current user having administrative access to the local server
A local database called DemoData
Under those assumptions, a database connection can be created with the following code:
module RawADOExample open System.Data open System.Data.SqlClient //raw ADO example let readPeople = use connection = new SqlConnection( "Server=localhost;Integrated security=SSPI;database=DemoData")
Most people familiar with other .NET languages will not find the preceding line of code that different than a similar line of code in C# that accomplishes the same thing. Of course, it is important to note that like many other objects that use external resources and implement IDisposable
to have a means to release said resources, it is important to use the use
binding when instantiating them. This is analogous to putting code in a using block in C#.
Creating connections to other data sources is a matter of finding other concrete classes that support the IDbConnection
interface, of which there are a robust number of options if you prefer to work with something other than MS SQL Server.
Of course, a connection alone is not going to do much all by itself. Performing a query requires a command object that holds the query. The query itself is SQL code, which for this example, can be used in string form:
module RawADOExample open System.Data open System.Data.SqlClient //raw ADO example let readPeople = use connection = new SqlConnection( "Server=localhost;Integrated security=SSPI;database=DemoData") use command = new SqlCommand("select firstname, lastname from person",connection)
Again, this is not that different than code you would write in any other .NET language for reaching a database. Nothing really interesting has yet been done, however. If the desire is to retrieve some actual data, then the command is going to need to be executed. If an assumption is made that the firstname
and lastname
columns of the person table can be read as strings (that is, varchar, char
, or something like that), the following record type can be used as a container:
type Person = {Id:int;FirstName:string;LastName:string}
The next step is to actually run the query, take the results, and populate a list of Person
records:
module RawADOExample open System.Data open System.Data.SqlClient type Person = {Id:int;FirstName:string;LastName:string} //raw ADO example let readPeople = use connection = new SqlConnection("Server=localhost;Integrated security=SSPI;database=DemoData") connection.Open() use command = new SqlCommand("select firstname, lastname from person",connection) use rawResult = command.ExecuteReader() let people = seq { while rawResult.Read() do yield { Id = rawResult.["FirstName"] :?> int; FirstName = rawResult.["FirstName"] :?> string; LastName = rawResult.["LastName"] :?> string } } people |> Seq.toList
For those familiar with how ADO.NET works, this is a familiar pattern. A call to command.ExecuteReader()
is made, which provides a SqlDataReader
through which data can be accessed. In the example, there is an F# spin on the old while-Read()-next
loop. In this case, the result is compiled into a sequence expression that specifies how the loop can be iterated through and reads the results into Person
records. It is important to note that, like all sequence definitions, nothing has actually happened yet — this code merely specifies how the sequence is to be read. It does not actually read it until the next line, people |> Seq.toList
, which iterates through the people
sequence and materializes an actual list of Person
records.
As simple as it may seem to simply do a Seq.filter
operation on the result of selecting for all the records in a given table at once, such operations do not scale for anything but very small data sets. As a result, when working with larger databases, the need to let the database do some of the work of filtering data right at the source will present itself. This is, of course, what databases do best, and it would be pretty silly not to let the database help in the task of querying data.
For this sample, it is going to be assumed that the goal is to implement a query for people based on the FirstName
column in the database. Such code needs to add a parameter object into the query that specifies which first name is to be queried for:
let retrievePeopleNamed firstName = use connection = new SqlConnection("Server=localhost;Integrated security=SSPI;database=DemoData") connection.Open() use command = new SqlCommand("select firstname, lastname from person where firstname = @firstname",connection) let parameter = new SqlParameter("firstName",firstName) do command.Parameters.Add parameter |> ignore let rawResult = command.ExecuteReader() let people = seq { while rawResult.Read() do yield { Id = rawResult.["FirstName"] :?> int; FirstName = rawResult.["FirstName"] :?> string; LastName = rawResult.["LastName"] :?> string } } people |> Seq.toList
The first key difference is the inclusion of the @firstname
literal within the string used to build the command:
use command = new SqlCommand("select firstname, lastname from person where firstname = @firstname",connection)
The @firstname
literal is a placeholder for where the parameter will go. This approach of using parameters is important because without them, SQL injection bugs are almost always introduced:
use command = new SqlCommand("select firstname, lastname from person where firstname = " + firstName, connection)
If a user decides to be clever and submit a firstName
of '; drop database
, it is not an understatement to imagine that there could be some serious issues!
When there is a query and placeholder in place, parameters can be added:
let parameter = new SqlParameter("firstName",firstName) do command.Parameters.Add parameter |> ignore
Using the same concept of commands and parameters, the next step is to implement the other three main database operations. The following code demonstrates how, on the vast majority of databases, we might do those operations:
let doCreateUpdateDelete() = use connection = new SqlConnection("Server=localhost;Integrated security=SSPI;database=DemoData") connection.Open() //create use createCommand = new SqlCommand("insert into person (firstname,lastname) values (@firstname,@lastname)",connection) let firstNameParameterCreate = new SqlParameter("firstName","Aaron") let lastNameParameterCreate = new SqlParameter("lastName","Erickson") do createCommand.Parameters.Add firstNameParameterCreate |> ignore do createCommand.Parameters.Add lastNameParameterCreate |> ignore do createCommand.ExecuteNonQuery() |> ignore //update use updateCommand = new SqlCommand("update person set firstname=@firstname, lastname=@lastname where id=@id", connection) let firstNameParameterUpdate = new SqlParameter("firstName","Not") let lastNameParameterUpdate = new SqlParameter("lastName","Sure") let idParameterUpdate = new SqlParameter("id",42) do updateCommand.Parameters.Add firstNameParameterUpdate |> ignore do updateCommand.Parameters.Add lastNameParameterUpdate |> ignore do updateCommand.Parameters.Add idParameterUpdate |> ignore do updateCommand.ExecuteNonQuery() |> ignore //delete use deleteCommand = new SqlCommand("delete person where id=@id", connection) let idParameterDelete = new SqlParameter("id","42") do deleteCommand.Parameters.Add idParameterDelete |> ignore do deleteCommand.ExecuteNonQuery() |> ignore
The only difference between the read examples and these are the use of .ExecuteNonQuery()
, which is the standard way to perform operations that do not return records from a database using ADO.NET. Outside of that, the exact same techniques for inserting parameters into standard SQL queries apply.
One striking thing, however, about using raw ADO.net in any language, F# included, is the raw volume of code required to do, what are in essence, really simple tasks. When one considers that the entire code base of the Microsoft TrueSkill system is around 100 lines of F# code, having 15 lines of code to do three simple operations seems excessive. Thankfully, there are frameworks built on top of ADO.net to help deal with this complexity.
The first place someone who is frustrated with the verbosity of raw ADO.NET might look to get some relief is to the many Object/Relational Mapping layers out there, such as NHibernate
or the Microsoft Entity Framework. Despite the general conclusion in Ted Neward's Famous Essay, "The Vietnam of Computer Science" (http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx
), for many cases, ORMs are a great solution for many types of problems.
You need to remember what the 'O' in ORM stands for — namely that it expects objects. Objects, of course, tend to have a state that is typically assumed to be mutable. Further, most ORMs require that properties be virtual so that their implementation can be overridden to do things like lazy loading.
The following is a typical implementation of the Person
record from the previous examples in this chapter if you were going to use it with NHibernate
:
type NHPerson() = class let mutable _id : int = 0 let mutable _firstName : string = "" let mutable _lastName : string = "" abstract Id : int with get, set default x.Id with get() = _id and set(v) = _id <- v abstract FirstName : string with get, set default x.FirstName with get() = _firstName and set(v) = _firstName <- v abstract LastName : string with get, set default x.LastName with get() = _lastName and set(v) = _lastName <- v end
The situation in Entity Framework v4 (the first version to support POCO
objects) is quite similar, with the only important difference being the lack of a need for doing the abstract declarations for properties that are not going to be lazy loaded. Nearly all ORMs have a "special thing" of one type or another that need to be done to objects to work with the ORM.
In either case there are problems. Most ORM systems depend on mutability. The internal mechanism of ORM systems for object construction tends to be something like:
Create object, usually through a parameterless constructor.
Set properties based on a configuration file mapping — or perhaps a fluent
convention based mapping à la FluentNHibernate
.
Use a dynamic proxy mechanism that will intercept access to lazily loaded properties, submitting additional queries to load said properties.
As has been covered before, F# programs generally avoid mutability. This is especially true if the reason that mutability is allowed is solely to support an ORM. If one of the reasons we consider F# is to move away from mutability, having to introduce it to make an ORM work is actually a step backward. Generally speaking, unless there is an externally defined requirement to use an ORM, we recommend that software developers avoid using ORM technologies with F#.
F# Active Record (FAR) is an Open Source Project I founded that makes it possible to get ORM-like features, but in a way that uses convention over configuration, while embracing an approach more idiomatic to F#.
The idea with FAR is to make mapping from rows in a database to records in F# drop-dead easy, while retaining a functional style of programming. It is not designed to cover everything that we might ever want to do with a database. In fact, it is biased toward simplicity, leaving out edge case features in favor of making the included functionality very robust and predictable.
For example, consider the following Person
record definition:
type Person = {Id:int;FirstName:string;LastName:string;Age:int}
Code for populating a sequence of Person
objects should look something like this:
use context = new ForSqlContext("SomeConnectionString") let people = context.SequenceFrom<Person>()
In this case, people
is now a seq<Person>
that further F# code can do work with. It is assumed that there is a table name in the targeted database that matches the type name of Person
, and that there are columns in said table that correspond to the record properties of Id, FirstName, LastName
, and Age
. This approach works so long as a convention that type names match table names, and property names match column names, holds true. Although this obviously can't work in all situations, there are a great number of cases where it does. It is those situations where the convention can be maintained where this kind of approach can result in a drastic drop in the amount of mapping code in the system.
For more on convention over configuration, please see http://en.wikipedia.org/wiki/Convention_over_configuration
.
In a case where the Person
table has a smaller number of rows, this approach works particularly well. Filtering on a small set is a matter of applying Seq.filter
with an appropriate predicate to the following result.
A predicate is a function that returns true
or false
. A filtering predicate in this context needs to start with one parameter of the type we are filtering and resolve to a Boolean value:
let me = people |> Seq.filter( fun p -> p.FirstName = "Aaron" )
This, however, is not an approach that will scale. If people
is something more like the list of people in the United States Social Security database, rather than the list of employees in a small company, bringing that sequence out of the database and into memory to analyze it is not the most scalable approach.
To not have to bring all the records into memory, a need will exist for a way to easily tell the system that the predicate — the part where it is specified p -> p.FirstName = "Aaron" —
should be analyzed in the database. Using FAR, the technique for doing so looks like this:
use context = new ForSqlContext("SomeConnectionString") let me = context.SequenceFrom<Person>( <@ fun p -> p.FirstName = "Aaron" @> )
Notice two key differences in this version compared to the previous one. The first one is that the predicate is being passed to the SequenceFrom
method of the context object. This method's purpose is to analyze the predicate passed in, convert that predicate to the appropriate raw SQL code that will be used for the query on the database management system, and retrieve the result in a manner similar to the version of SequenceFrom
that has no parameters and simply returns everything.
The other key difference is that the predicate was quoted (using <@ @>
syntax). Quoting the predicate is necessary in this case, as the SequenceFrom
method needs to analyze the predicate to evaluate it and generate a database query that will ultimately run against the database. Remember, one of the key reasons for quotation is to execute code "by other means." In a case like this, the desire is to execute the predicate against the data. For that to happen, the predicate has to be converted to a form that is useful where the query will occur, which happens to be in the database, not in F#.
Creating new records in the database is nearly as simple as reading them:
type Person = {Id:int;FirstName:string;LastName:string;Age:int} type Cat = {PetName:string;Color:string;IsCute:bool;IsMean:bool} [<Test>] static member TestReadPeopleAndPets = use context = new ForDataContext("SomeConnectionString") do context.Create {Id=1;FirstName="Aaron";LastName="Erickson";Age=37} |> ignore do context.Create {Id=2;FirstName="Erin";LastName="Erickson";Age=34} |> ignore do context.Create {Id=3;FirstName="Adriana";LastName="Erickson";Age=13} |> ignore do context.Create {Id=4;FirstName="Matthew";LastName="Erickson";Age=8} |> ignore
let people = context.SequenceFrom<Person>( <@ fun p -> p.LastName = "Erickson" @> ) |> Seq.toArray Assert.AreEqual(people.Length,4) do context.Create {PetName="Puppy Cat";Color="Ginger";IsCute=true;IsMean=false} |> ignore do context.Create {PetName="Dmitry";Color="Blue-Gray";IsCute=true;IsMean=true} |> ignore let theCats = context.SequenceFrom<Cat>() |> Seq.toArray Assert.AreEqual(theCats.Length,2)
In this example, the Person
and Cat
types are declared using simple F# record types. Using the ForDataContext
that is created for when you want to read data, the Create
method can be used to add rows to the database.
The Create
method is generic, in that it can detect the type of record passed to it and use that record's type to determine which table it should insert the record to. The Create
method returns a number of rows affected, which in most operations where you are creating a row, should tend to resolve to 1. In the preceding example, because the result is not needed for further processing, it is simply passed to ignore, and the next record is added.
Update, like create, uses a convention-based minimalist approach:
do context.Update {Id=3;FirstName="Adriana";LastName="Erickson";Age=13} |> ignore
Update uses a convention that the first property that ends in Id
is expected to be, for database purposes, the identity upon which the update will be based. It would roughly translate to the following SQL code:
update person set firstname='Adriana', lastname='Erickson', age=13 where id=3
If the Id
column in the database were personid
, rather than simply id
, so long as personid
is the first field that ends in id
in the record, the following would also work:
do context.Update {PersonId=3;FirstName="Adriana";LastName="Erickson";Age=13} |> ignore
As you might expect, deletion does not stray from our simplistic formula:
do context.Delete {Id=4;FirstName="Matthew";LastName="Erickson";Age=8} |> ignore
The preceding code specifies a query that, in this case would roughly translate to:
delete from person where id=4
Technically, for delete
, only the Id
field is needed. However, because the record type defines the table, and the record does not allow null
values in the other fields, the rest of the record is specified in the example.
One of the reasons the library can remain as simple as it does is that it is explicitly stateless. There is no change tracking, no concept of attach/detach, and no pretense that the library is something other than a means to act as either a producer or consumer of F# record types. Other things that make more traditional ORM systems complex, such as configuration files, mappings, and base classes to inherit from, are also avoided in favor of simplicity and predictability in terms of how it works.
At the time of this writing, there are some features that have yet to be implemented but should be complete by the time of publication. Support for stored procedures, inner joins, and a broader set of predicate types are in the works. Progress on this project can be followed at http://github.com/ericksoa/FAR
.
One of the principles of being a good software developer is to have at least some level of understanding about how the libraries work. As such, this section explores the internals of the FAR library.
To do our decomposition of a query, .NET reflection will be used, as well as various elements of Microsoft.FSharp.Quotations
. As well, System.Data.SqlClient
will be used so that the appropriate connections and commands can be created. Although future versions may refactor this by extracting out database specific concerns from parsing concerns, for now, it will be assumed that the target database is Microsoft SQL Server:
module FSharpActiveRecord open System open System.Reflection open System.Data.SqlClient open Microsoft.FSharp.Quotations open Microsoft.FSharp.Quotations.DerivedPatterns
The next two routines provide a wrapper behind creation of typical ADO.NET primitives that will be used across various specific methods in the library:
let MakeConnection connectionString = new SqlConnection(connectionString) let MakeCommand commandText connection = new SqlCommand(commandText, connection)
These functions provide a means for various routines within the system that need connections without having to; at least in the method itself, depend on System.Data.SqlClient
specifics.
The next routine is an important utility that allows you to take a sequence of strings and return a comma-separated list:
//ConvertToCommaSeparatedString courtesy of Mark Needham, ThoughtWorks let ConvertToCommaSeparatedString (value:seq<string>) = let rec convert (innerVal:List<string>) acc = match innerVal with | [] -> acc | hd::[] -> convert [] (acc + hd) | hd::tl -> convert tl (acc + hd + ",") convert (Seq.toList value) ""
When generating SQL, a frequent need is to generate comma-separated lists of things such as column names and parameters. Because this list should not have a comma at the end of the list, something is needed that distinguishes between the last item and everything else when generating the list. This routine fits the bill rather nicely.
The next thing that is needed, if you generate a lot of SQL, is something that converts primitive F# types to strings that reflect their SQL counterparts:
let FSharpTypeToSqlType fSharpType size = match fSharpType, size with | t, Some(s) when t = typeof<string> -> sprintf "varchar(%i)" s | t, _ when t = typeof<string> -> "varchar(255)" | t, _ when t = typeof<int> -> "int" | t, _ when t = typeof<bool> -> "bit" | t, _ when t = typeof<DateTime> -> "datetime" | _, _ -> raise( new NotSupportedException() )
This routine takes a type
(as a .NET Type object) and a size
as an int option
. Based on combinations of type and size, it generates a SQL type signature. This type of routine is especially useful in cases where we need to drive SQL types from .NET types, as is done in the creation of tables in a database.
This is especially useful for cases when the desire is to create a table based on a record type:
let private CreateTable tableName columns connectionString = let columnList = columns |> Seq.map (fun (name, colType, size) -> sprintf "%s %s" name (FSharpTypeToSqlType colType size)) |> ConvertToCommaSeparatedString let query = sprintf "create table %s (%s)" tableName columnList use connection = connectionString |> MakeConnection use createCommand = MakeCommand query connection do connection.Open() |> ignore do createCommand.ExecuteNonQuery() |> ignore let CreateTableFor<'a> connectionString = let tableName = typeof<'a>.Name let columnSpecSelector (p:PropertyInfo) = (p.Name,p.PropertyType,None) let columnSpecs = typeof<'a>.GetProperties() |> Seq.map columnSpecSelector do CreateTable tableName columnSpecs connectionString
CreateTableFor
is a generic function that takes a record type that will be used as a template for creating a table and a connection string used to reach an actual database. Using .NET reflection, it uses Seq.map
to create a set of (type * name * size)
tuples (columnSpecs
). It passes the tableName
(from the generic type), the columnSpecs
, and the connection string to a private CreateTable
method that does the work to convert things into a SQL create table script that can be executed against a database.
The job of generating SQL is made easy by some of the utility functions that are already in place. The column list part of the create script, usually taking the form of something like somename sometype somesize, anothername anothertype anothersize
is handled by mapping the tuples through FSharpToSqlType
and formatting them using sprintf
.
From there, the next step is to format the rest of the SQL, putting together the table and columnList
and then do typical ADO.NET work of creating a connection and command needed to run the actual query.
The FAR library contains many routines such as CreateTable
that simplify database creation, destruction, and other similar tasks that are especially important in setup and teardown of integration tests. Please see the book's sample code or the FAR library website for more details on these routines.
The real useful part of this library is the conversion of idiomatic F# code to SQL. However, for this to work, the following is needed:
let makeSimpleSelect table fields = let commaSeperatedFields = fields |> ConvertToCommaSeparatedString sprintf "select %s from %s" commaSeperatedFields table
This first routine takes a table name and a seq<string>
that represents database fields. It then generates a typical SQL query that we would use against a single table. For example, passing Person
and [FirstName,LastName]
would generate:
select firstname,lastname from person
This is a good start, but far more needs to be done if you want the ability for more complex predicates to be passed along with a record type. More complex queries require that an expression be parsed and tree structure be built that represents it so it can be converted to SQL. Start by defining the following tree structure in the form of a discriminated union:
type ParseNode = | EqualNode of Type list * Expr list | AndNode of ParseNode option * ParseNode option | OrNode of ParseNode option * ParseNode option //other nodes as needed
The preceding example is a simplistic version of what can represent a parse tree. It will start with equality tests as terminal nodes in the tree, which can be combined with various combinations of the terms and
or or
. For example p.FirstName = "Aaron" and (p.LastName = "Erickson" or p.LastName="Burr")
might look like:
---------AndNode------- | | EqualNode OrNode-------------------- p.FirstName = "Aaron" | | p.LastName = "Erickson" p.LastName = "Burr"
Two routines will use this tree. One has to take an F# quoted predicate and generate the tree; the other will recursively walk the tree to generate equivalent SQL code.
Start by declaring a context from which we will run queries that contain the connection object:
type ForDataContext = val connection : SqlConnection new(connectionString) = { connection = connectionString |> MakeConnection } with interface IDisposable with member disposable.Dispose() = disposable.connection.Close()
Within this type, members are created that depend on this common context. The first is the routine that generates the parse tree:
//takes a predicate and generates a tuple composed of // parameterized sql * parameters static member private ParseCriteria<'a> (criteria:Expr<'a -> bool>) = let rec predicateParser expr = match expr with | SpecificCall <@ (=) @> (optionExpr, types, exprs) -> Some(EqualNode(types,exprs)) | SpecificCall <@ (&&) @> (optionExpr, types, exprs) -> Some(AndNode(predicateParser(exprs.[0]),predicateParser(exprs.[1]))) | SpecificCall <@ (||) @> (optionExpr, types, exprs) -> Some(OrNode(predicateParser(exprs.[0]),predicateParser(exprs.[1]))) | Patterns.IfThenElse (left, middle, right) -> match middle with | Patterns.Call(optionExpr, types, exprs) -> Some(AndNode(predicateParser(left), predicateParser(middle))) | Patterns.Value(value, valueType) -> Some(OrNode(predicateParser(left), predicateParser(right))) | _ -> None | _ -> None
Our overall goal with ParseCriteria
is to take a quoted predicate and generate a SQL string and a series of ordered parameters that will be applied to that string to perform a query. This parsing is made much simpler by inclusion of the Microsoft.FSharp.Quotations
and Microsoft.FSharp.Quotations.DerivedPatterns
namespaces. Most of the work is done by the recursive predicateParser
routine, which recursively walks the provided predicate, mapping the expression tree to something we can more easily consume.
SpecificCall
is an active pattern that matches a given quoted expression (such as <@ (=) @>
) against what the tree contains. In F#, this works very well for the EqualNode
type, which when matched, emits an EqualNode
that contains a pair of expressions and types that can be used for generating an equals SQL expression.
The other instances of SpecificCall
work in certain circumstances where one would expect them to, and if they are present, will emit the appropriate AndNode
or OrNode
. AndNode
and OrNode
both take two arguments, for their left and right side, which will be provided by recursive calls to predicateParser
on the respective left and right sides of the provided expression tree.
Of course, there is a special case, which is that sometimes optimizations get applied to the tree. If the quoted and
or quoted or
get converted for some reason into IfThenElse
calls in the predicate, those need to be handled as well. IfThenElse
is another active pattern that can be used to match against for that case. And
and Or
both get converted to IfThenElse
in certain ways that are recognizable. If the middle node in IfThenElse
is a Call
, it can be deduced that it was converted from (&&)
, and be handled accordingly. If it is a Value
, it can be deduced it was converted from (||)
.
The result of all this, provided that the expression matches the rules, is a tree that we can much more cleanly generate SQL from, as the following demonstrates:
let paramEnumerator = let paramNames = 1 |> Seq.unfold (fun i -> Some(i+1,i)) |> Seq.map( fun i -> sprintf "param%i" i) paramNames.GetEnumerator() let nextParam() = paramEnumerator.MoveNext() |> ignore paramEnumerator.Current let paramList = new System.Collections.Generic.List<string * obj>() let rec queryString (treeNode:ParseNode option) = match treeNode with | Some(node) -> match node with | EqualNode (left,right) -> match (right.[0],right.[1]) with | (Patterns.PropertyGet (option,property,someList), Patterns.Value(value, valueType)) -> let p = nextParam() do (p,value) |> paramList.Add sprintf "%s = @%s" property.Name p | _ -> raise(new InvalidOperationException()) | AndNode (left,right) -> sprintf "(%s and %s)" (queryString left) (queryString right) | OrNode (left,right) -> sprintf "(%s or %s)" (queryString left) (queryString right) | None -> ""
There are a couple of things that need to be done here. The first step is to generate SQL that will be transformed into the where
clause at the end of the overall SQL query. The next step is to generate a set of parameter objects that will be used with the ADO.NET command object to thread the query parameters together.
To do the parameter list, it will be necessary to use a mutable list structure that can be added to and that holds the parameter names and values. Also needed will be something that generates unique parameter names. These parameters need to be generated on an on-demand basis while parsing is taking place. The paramEnumerator
is a utility that can be used to act as something of a dispenser for parameters that are guaranteed to be unique to the query. The nextParam()
function executes this generator, as is used by queryString
as it is doing its work. paramList
is then used to accumulate these parameters as queryString
does its work.
The real work of queryString
is to handle the various node cases, and either just generate SQL (in the case of an EqualNode
), or — in the case of AndNode
or OrNode
— take the result of recursive queryString
calls and compound the results together into the appropriate syntax.
This is all brought together in the full implementation of ParseCriteria
:
//takes a predicate and generates a tuple composed of // parameterized sql * parameters static member private ParseCriteria<'a> (criteria:Expr<'a -> bool>) = let rec predicateParser expr = match expr with | SpecificCall <@ (=) @> (optionExpr, types, exprs) -> Some(EqualNode(types,exprs)) | SpecificCall <@ (&&) @> (optionExpr, types, exprs) -> Some(AndNode(predicateParser(exprs.[0]),predicateParser(exprs.[1]))) | SpecificCall <@ (||) @> (optionExpr, types, exprs) -> Some(OrNode(predicateParser(exprs.[0]),predicateParser(exprs.[1]))) | Patterns.IfThenElse (left, middle, right) -> match middle with | Patterns.Call(optionExpr, types, exprs) -> Some(AndNode(predicateParser(left), predicateParser(middle))) | Patterns.Value(value, valueType) -> Some(OrNode(predicateParser(left), predicateParser(right))) | _ -> None | _ -> None let parsedResult = match criteria with | Patterns.Lambda(var,lambda) -> predicateParser lambda | _ -> None let paramEnumerator = let paramNames = 1 |> Seq.unfold (fun i -> Some(i+1,i)) |> Seq.map( fun i -> sprintf "param%i" i) paramNames.GetEnumerator() let nextParam() = paramEnumerator.MoveNext() |> ignore paramEnumerator.Current let paramList = new System.Collections.Generic.List<string * obj>() let rec queryString (treeNode:ParseNode option) = match treeNode with | Some(node) -> match node with | EqualNode (left,right) -> match (right.[0],right.[1]) with | (Patterns.PropertyGet (option,property,someList), Patterns.Value(value, valueType)) -> let p = nextParam() do (p,value) |> paramList.Add sprintf "%s = @%s" property.Name p | _ -> raise(new InvalidOperationException()) | AndNode (left,right) -> sprintf "(%s and %s)" (queryString left) (queryString right) | OrNode (left,right)
-> sprintf "(%s or %s)" (queryString left) (queryString right) | None -> "" queryString parsedResult, paramList.ToArray()
Of course, a couple more utility functions are needed to make the context work. A connection needs to be opened if it isn't open yet. (Remember, we can run multiple queries from the same context.) Also needed is a way to easily take the SQL and command objects and execute them:
member private context.OpenConnectionIfNeeded() = if (context.connection.State <> Data.ConnectionState.Open) then context.connection.Open() //core routine that takes sql + parameters and yields readers that // eventually get composed into records we want to work with member private context.DoQuery query (rawParameters:array<string * #obj>) = use command = MakeCommand query context.connection do rawParameters |> Array.map( fun r -> new SqlParameter(fst r,snd r) ) |> Array.iter( fun p -> command.Parameters.Add p |> ignore ) do context.OpenConnectionIfNeeded() let reader = command.ExecuteReader() seq { while reader.Read() do yield reader } member private context.DoCommand command (rawParameters:array<string * #obj>) = use command = MakeCommand command context.connection do rawParameters |> Array.map( fun r -> new SqlParameter(fst r,snd r) ) |> Array.iter( fun p -> command.Parameters.Add p |> ignore ) do context.OpenConnectionIfNeeded() command.ExecuteNonQuery()
These methods use similar techniques that were used when doing raw ADO for creating and using connections and commands. The only difference is they do so in a much more generic fashion. With these utility methods in place, the next step is to go forth and implement the query interface:
//core routine that composes and executes the query member private context.SequenceFrom<'a> ((whereClause:string),(parameters:array<string * #obj>)) = let tableName = typeof<'a>.Name let memberNameSelector (m:#MemberInfo) = m.Name let propertyNames = typeof<'a>.GetProperties() |> Seq.map memberNameSelector let query = (makeSimpleSelect tableName propertyNames) + " " + whereClause let creator = Reflection.FSharpValue.PreComputeRecordConstructor( typeof<'a>,BindingFlags.Public) let data = context.DoQuery query parameters let readObjectsFromReaderByField (reader:SqlDataReader) (keys:seq<string>) = keys |> Seq.map( fun(k) -> reader.[k] ) |> Seq.toArray data |> Seq.map(
fun r -> creator(readObjectsFromReaderByField r propertyNames) :?> 'a ) //Performs queries where you have (for now) simple object.Property = someValue member context.SequenceFrom<'a> (criteria:Expr<'a -> bool>) = let queryAndParams = criteria |> ForDataContext.ParseCriteria context.SequenceFrom<'a> ( (sprintf "where %s" (fst queryAndParams)),(snd queryAndParams))
The public version takes the quoted predicate criteria, generates a SQL query and parameter set, and calls the private SequenceFrom
, which expects a where
clause in string form, as well as an array of parameter name and object tuples that will be used for parameters.
In the private overload of SequenceFrom
, the rest of the query will be built. F# reflection is used to build a mechanism (creator
, in the preceding example) for creating F# records based on results we get returned from the SqlDataReader
.
When a creator is set up, the next step is to call context.DoQuery
that yields a DataReader
that, when iterated, advances through the records. What is returned is a sequence iterator that converts each state of the SqlDataReader
into the appropriate record type. What is notable about this is that it returns a seq<'a>
for a reason. A user of the library could choose to take only the top five elements, and in doing so, will use the built in forward-only cursor functionality of MS SQL Server to avoid taking the entire set of rows over the wire for only the top five elements.
Wrapping up, for the convenience of people who want to read all the records, this overload is added:
//simple case where we are getting all the rows from a table member context.SequenceFrom<'a>() = context.SequenceFrom<'a>(String.Empty,Array.empty)
Other operations are, thankfully, much simpler to implement. Take create for example:
member context.Create someObject = let tableName = someObject.GetType().Name let columnsAndValues = someObject.GetType().GetProperties() |> Seq.map( fun p -> (p.Name,p.GetValue(someObject,Array.empty))) let columnNames columnsValuePair = columnsValuePair |> Seq.map( fun pair -> fst pair) |> ConvertToCommaSeparatedString let convertToValueParameterBucket columnsValuePair = columnsValuePair |> Seq.map( fun pair -> sprintf "@%s" (fst pair)) |> ConvertToCommaSeparatedString let query = sprintf "insert into %s (%s) values (%s)" tableName
(columnsAndValues |> columnNames) (columnsAndValues |> convertToValueParameterBucket) let parameters = columnsAndValues |> Seq.toArray context.DoCommand query parameters
The common pattern for nonquery operations where a parse tree need not be analyzed is simply to do some reflection over the generic type, read the values from the provided record in some manner that matches a convention, construct a SQL command, and execute it. The implementations of delete and update differ only by the SQL that is generated; nearly everything else is the same.
FAR, as a library, is meant to be a simple, minimalist, and illustrative implementation that covers a large, but not exhaustive, set of cases for how F# applications might access data. It does so in a manner that allows the library to be independent of the domain, avoiding inheritance or any other special "things" that need to be done to make a record work with FAR. If you have a record, and you have a table and columns that match up, FAR can work.
In this chapter, we covered ways for F# programs to interact with data, particular in ways that will feel idiomatic to F# developers. You may choose to use raw ADO.NET, use F# Active Record, or use one of the more object-oriented frameworks. What is important, across all of these, is to minimize the amount of plumbing code you write, so you can focus on doing something with data, rather than simply gathering it.