SQLProvider


Composable Queries

Definition of Composable Queries

Basicly composable methods are those that you can chain together to build the desired functionality out of smaller parts. By passing functions as parameters you are able to generate higher-order query operations. Therefore composable query means that you can do logics to compose just one database-SQL-query from multiple queryables. By using composable queries you can shorten the Database transactions and keep the connection open a minimum of time.

One common anti-pattern is a trial to solve all the problems in the world by a single code. So when you use this kind of features to achieve "common logics", keep in mind the software mainainability. One solution is to add a database view and query that from SQLProvider.

Generate composable queries by using Linq IQueryable

With composable queries you can combine two queries in multiple ways, and one query can be used as the building block for the other query. To see how this works, let’s look at a simple query:

1: 
2: 
3: 
4: 
5: 
let query1 =
    query {
      for customers  in ctx.Main.Customers do
      where (customers.ContactTitle = "USA")
      select (customers)}

The variable that is returned from the query is sometimes called a computation. If you write evaluate (e.g. a foreach loop or |> Seq.toList) and display the address field from the customers returned by this computation, you see the following output:

1: 
2: 
3: 
4: 
5: 
GREAL|Great Lakes Food Market|Howard Snyder|Marketing Manager|2732 Baker Blvd.|Eugene|OR|97403|USA|(503) 555-7555|
HUNGC|Hungry Coyote Import Store|Yoshi Latimer|Sales Representative|City Center Plaza 516 Main St.|Elgin|OR|97827|USA|(503) 555-6874|(503) 555-2376
...
TRAIH|Trail's Head Gourmet Provisioners|Helvetius Nagy|Sales Associate|722 DaVinci Blvd.|Kirkland|WA|98034|USA|(206) 555-8257|(206) 555-2174
WHITC|White Clover Markets|Karl Jablonski|Owner|305 - 14th Ave. S. Suite 3B|Seattle|WA|98128|USA|(206) 555-4112|(206) 555-4115

You can now write a second query against the result of this query:

1: 
2: 
3: 
4: 
5: 
6: 
let query2 =
    query {
      for customers in query1 do
      where (customers.CompanyName = "The Big Cheese")
      select customers}
    |> Seq.toArray

Notice that the last word in the first line of this query is the computation returned from the previous query. This second query produces the following output:

1: 
THEBI|The Big Cheese|Liz Nixon|Marketing Manager|89 Jefferson Way Suite 2|Portland|OR|97201|USA|(503) 555-3612|

SQLProvider to Objects queries are composable because they operate on and usually return variables of type IQueryable<T>. In other words, SQLProvider queries typically follow this pattern:

1: 
2: 
3: 
4: 
5: 
6: 
7: 
8: 
let (qry:IQueryable<'T>) =
    query {
        //for is like C# foreach
        for x in (xs:IQueryable<'T>) do
        select x
    }

//

This is a simple mechanism to understand, but it yields powerful results. It allows you to take complex problems, break them into manageable pieces, and solve them with code that is easy to understand and easy to maintain.

Generate composable queries by using .NET LINQ functions with IQueryable.

The difference between IEnumerable and IQueryable is basically that IEnumerable is executed in the IL while IQueryable can be translated as en expression tree to some other context (like a database query). They are both lazy by nature, meaning they aren’t evaluated until you enumerate over the results.

Here an example:

First you have to add .NET LINQ:

1: 
open System.Linq

Then you can define a composable query outside the main query

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
type CustomersEntity = sql.dataContext.``main.CustomersEntity``

let companyNameFilter inUse =

    let myFilter2 : IQueryable<CustomersEntity> -> IQueryable<CustomersEntity> = fun x -> x.Where(fun i -> i.CustomerId = "ALFKI")

    let queryable:(IQueryable<CustomersEntity> -> IQueryable<CustomersEntity>) =
        match inUse with
        |true ->
            (fun iq -> iq.Where(fun (c:CustomersEntity) -> c.CompanyName = "The Big Cheese"))
        |false -> 
            myFilter2
    queryable

(Let's asume that your inUse some complex data: E.g. Your sub-queries would come from other functions. Basic booleans you can just include to your where-clause)

Then you can create the main query

1: 
2: 
3: 
4: 
5: 
let query1 =
    query {
        for customers  in ctx.Main.Customers do
        where (customers.ContactTitle = "USA")
        select (customers)}

and now call you are able to call the second query like this

1: 
let res = companyNameFilter true query1 |> Seq.toArray

Generate composable queries by using FSharp.Linq.ComposableQuery

The SQLProvider also supports composable queries by integrating following library FSharpLinqComposableQuery. You can read more about that library here: FSharp.Linq.ComposableQuery

Because it is implemented in the SQLProvider you dont need to add FSharpComposableQuery in your script.

Example for using FSharpComposableQuery

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
14: 
15: 
let qry1 =
    query { for u in dbContext.Users do
            select (u.Id, u.Name, u.Email)
    }

let qry2 =
    query { for c in dbContext.Cars do
            select (c.UserId, c.Brand, c.Year)
    }

query { for (i,n,e) in qry1 do
        join (u,b,y) in qry2 on (i = u)
        where (y > 2015)
        select (i,n,e,u,b,y)
    } |> Seq.toArray

Nested Select Where Queries

You can create a query like SELECT * FROM xs WHERE xs.x IN (SELECT y FROM ys)) with either LINQ Contains or custom operators: in |=| and not-in |<>| This is done by not saying |> Seq.toArray to the first query:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
11: 
12: 
13: 
open System.Linq

let nestedQueryTest =
    let qry1 = query {
        for emp in ctx.Hr.Employees do
        where (emp.FirstName.StartsWith("S"))
        select (emp.FirstName)
    }
    query {
        for emp in ctx.Hr.Employees do
        where (qry1.Contains(emp.FirstName))
        select (emp.FirstName, emp.LastName)
    } |> Seq.toArray

Using non-strongly-typed .GetColumn "name"

All the entities inherit from SqlEntity which has GetColumn-method. So you can use non-strongly-typed columns like this:

1: 
2: 
3: 
4: 
5: 
6: 
let qry = 
    query {
        for x in query1 do
        where ((x.GetColumn<string> "CustomerId") = "ALFKI")
        select (x.GetColumn<string> "CustomerId")
    } |> Seq.head

However this is not recommended as one of the SQLProvider's key benefits is strong typing.

Generate composable queries from quotations

You can also construct composable queries using the F# quotation mechanism. For example, if you need to select a filter function at runtime, you could write the filters as quotations, and then include them into query like that:

 1: 
 2: 
 3: 
 4: 
 5: 
 6: 
 7: 
 8: 
 9: 
10: 
let johnFilter = <@ fun (employee : sql.dataContext.``main.EmployeesEntity``) -> employee.FirstName = "John" @>
let pamFilter = <@ fun (employee : sql.dataContext.``main.EmployeesEntity``) -> employee.FirstName = "Pam" @>

let runtimeSelectedFilter = if 1 = 1 then johnFilter else pamFilter
let employees =
    query {
        for emp in ctx.Main.Employees do
        where ((%runtimeSelectedFilter) emp) 
        select emp
    } |> Seq.toArray

Quotations are AST representations which the Linq-to-SQL translator can use.

Multiple items
type LiteralAttribute =
  inherit Attribute
  new : unit -> LiteralAttribute

--------------------
new : unit -> LiteralAttribute
val connectionString : string
val resolutionPath : string
namespace System
namespace System.Linq
Multiple items
namespace FSharp

--------------------
namespace Microsoft.FSharp
Multiple items
namespace FSharp.Data

--------------------
namespace Microsoft.FSharp.Data
namespace FSharp.Data.Sql
type sql = obj
type SqlDataProvider


<summary>Typed representation of a database</summary>
                    <param name='ConnectionString'>The connection string for the SQL database</param>
                    <param name='ConnectionStringName'>The connection string name to select from a configuration file</param>
                    <param name='DatabaseVendor'> The target database vendor</param>
                    <param name='IndividualsAmount'>The amount of sample entities to project into the type system for each SQL entity type. Default 1000.</param>
                    <param name='UseOptionTypes'>If true, F# option types will be used in place of nullable database columns. If false, you will always receive the default value of the column's type even if it is null in the database.</param>
                    <param name='ResolutionPath'>The location to look for dynamically loaded assemblies containing database vendor specific connections and custom types.</param>
                    <param name='Owner'>Oracle: The owner of the schema for this provider to resolve. PostgreSQL: A list of schemas to resolve, separated by spaces, newlines, commas, or semicolons.</param>
                    <param name='CaseSensitivityChange'>Should we do ToUpper or ToLower when generating table names?</param>
                    <param name='TableNames'>Comma separated table names list to limit a number of tables in big instances. The names can have '%' sign to handle it as in the 'LIKE' query (Oracle and MSSQL Only)</param>
                    <param name='ContextSchemaPath'>The location of the context schema previously saved with SaveContextSchema. When not empty, will be used to populate the database schema instead of retrieving it from then database.</param>
                    <param name='OdbcQuote'>Odbc quote characters: Quote characters for the table and column names: `alias`, [alias]</param>
                    <param name='SQLiteLibrary'>Use System.Data.SQLite or Mono.Data.SQLite or select automatically (SQLite only)</param>
                    
namespace FSharp.Data.Sql.Common
type DatabaseProviderTypes =
  | MSSQLSERVER = 0
  | SQLITE = 1
  | POSTGRESQL = 2
  | MYSQL = 3
  | ORACLE = 4
  | MSACCESS = 5
  | ODBC = 6
  | FIREBIRD = 7
  | MSSQLSERVER_DYNAMIC = 8
Common.DatabaseProviderTypes.SQLITE: Common.DatabaseProviderTypes = 1
type CaseSensitivityChange =
  | ORIGINAL = 0
  | TOUPPER = 1
  | TOLOWER = 2
Common.CaseSensitivityChange.ORIGINAL: Common.CaseSensitivityChange = 0
val ctx : obj
val query1 : IQueryable<obj>
val query : Linq.QueryBuilder
val customers : obj
custom operation: where (bool)

Calls Linq.QueryBuilder.Where
custom operation: select ('Result)

Calls Linq.QueryBuilder.Select
val query2 : obj []
Multiple items
module Seq

from FSharp.Data.Sql

--------------------
module Seq

from Microsoft.FSharp.Collections
val toArray : source:seq<'T> -> 'T []
val qry : IQueryable<obj>
Multiple items
type IQueryable =
  inherit IEnumerable
  member ElementType : Type
  member Expression : Expression
  member Provider : IQueryProvider

--------------------
type IQueryable<'T> =
  inherit IEnumerable<'T>
  inherit IEnumerable
  inherit IQueryable
val x : obj
type CustomersEntity = obj
val companyNameFilter : inUse:bool -> (IQueryable<CustomersEntity> -> IQueryable<CustomersEntity>)
val inUse : bool
val myFilter2 : (IQueryable<CustomersEntity> -> IQueryable<CustomersEntity>)
val x : IQueryable<CustomersEntity>
(extension) System.Collections.Generic.IEnumerable.Where<'TSource>(predicate: System.Func<'TSource,bool>) : System.Collections.Generic.IEnumerable<'TSource>
(extension) System.Collections.Generic.IEnumerable.Where<'TSource>(predicate: System.Func<'TSource,int,bool>) : System.Collections.Generic.IEnumerable<'TSource>
(extension) IQueryable.Where<'TSource>(predicate: Expressions.Expression<System.Func<'TSource,bool>>) : IQueryable<'TSource>
(extension) IQueryable.Where<'TSource>(predicate: Expressions.Expression<System.Func<'TSource,int,bool>>) : IQueryable<'TSource>
val i : CustomersEntity
val queryable : (IQueryable<CustomersEntity> -> IQueryable<CustomersEntity>)
val iq : IQueryable<CustomersEntity>
val c : CustomersEntity
val query1 : IQueryable<CustomersEntity>
val customers : CustomersEntity
val res : CustomersEntity []
val qry1 : IQueryable<obj * obj * obj>
val u : obj
val qry2 : IQueryable<obj * obj * int>
val c : obj
val i : obj
val n : obj
val e : obj
custom operation: join var in collection on (outerKey = innerKey). Note that parentheses are required after 'on'

Calls Linq.QueryBuilder.Join
val b : obj
val y : int
val nestedQueryTest : (obj * obj) []
val qry1 : IQueryable<obj>
val emp : obj
(extension) System.Collections.Generic.IEnumerable.Contains<'TSource>(value: 'TSource) : bool
(extension) IQueryable.Contains<'TSource>(item: 'TSource) : bool
(extension) System.Collections.Generic.IEnumerable.Contains<'TSource>(value: 'TSource, comparer: System.Collections.Generic.IEqualityComparer<'TSource>) : bool
(extension) IQueryable.Contains<'TSource>(item: 'TSource, comparer: System.Collections.Generic.IEqualityComparer<'TSource>) : bool
val qry : obj
val x : CustomersEntity
Multiple items
val string : value:'T -> string

--------------------
type string = System.String
val head : source:seq<'T> -> 'T
val johnFilter : Quotations.Expr<(obj -> bool)>
val employee : obj
val pamFilter : Quotations.Expr<(obj -> bool)>
val runtimeSelectedFilter : Quotations.Expr<(obj -> bool)>
val employees : obj []
Fork me on GitHub