JSON Type Provider

This article demonstrates how to use the JSON Type Provider to access JSON files in a statically typed way. We first look at how the structure is inferred and then demonstrate the provider by parsing data from WorldBank and Twitter.

The JSON Type Provider provides statically typed access to JSON documents. It takes a sample document as an input (or a document containing a JSON array of samples). The generated type can then be used to read files with the same structure. If the loaded file does not match the structure of the sample, a runtime error may occur (but only when accessing e.g. non-existing element).

Introducing the provider

The type provider is located in the FSharp.Data.dll assembly and namespace:

open FSharp.Data

Inferring types from the sample

The JsonProvider<...> takes one static parameter of type string. The parameter can be either a sample string or a sample file (relative to the current folder or online accessible via http or https). It is not likely that this could lead to ambiguities.

The following sample passes a small JSON string to the provider:

type Simple = JsonProvider<""" { "name":"John", "age":94 } """>
let simple = Simple.Parse(""" { "name":"Tomas", "age":4 } """)

You can see that the generated type has two properties - Age of type int and Name of type string. The provider successfully infers the types from the sample and exposes the fields as properties (with PascalCase name to follow standard naming conventions).

Inferring numeric types

In the previous case, the sample document simply contained an integer and so the provider inferred the type int. Sometimes, the types in the sample document (or a list of samples) may not match exactly. For example, a list may mix integers and floats:

type Numbers = JsonProvider<""" [1, 2, 3, 3.14] """>
let nums = Numbers.Parse(""" [1.2, 45.1, 98.2, 5] """)
let total = nums |> Seq.sum

When the sample is a collection, the type provider generates a type that can be used to store all values in the sample. In this case, the resulting type is decimal, because one of the values is not an integer. In general, the provider supports (and prefers them in this order): int, int64, decimal and float.

Other primitive types cannot be combined into a single type. For example, if the list contains numbers and strings. In this case, the provider generates two methods that can be used to get values that match one of the types:

type Mixed = JsonProvider<""" [1, 2, "hello", "world"] """>
let mixed = Mixed.Parse(""" [4, 5, "hello", "world" ] """)

mixed.Numbers |> Seq.sum
mixed.Strings |> String.concat ", "

As you can see, the Mixed type has properties Numbers and Strings that return only int and string values from the collection. This means that we get type-safe access to the values, but not in the original order (if order matters, then you can use the mixed.JsonValue property to get the underlying JsonValue and process it dynamically as described in the documentation for JsonValue.

Inferring record types

Now let's look at a sample JSON document that contains a list of records. The following example uses two records - one with name and age and the second with just name. If a property is missing, then the provider infers it as optional.

If we want to just use the same text used for the schema at runtime, we can use the GetSamples method:

type People = JsonProvider<"""
  [ { "name":"John", "age":94 },
    { "name":"Tomas" } ] """>

for item in People.GetSamples() do
  printf "%s " item.Name
  item.Age |> Option.iter (printf "(%d)")
  printfn ""

The inferred type for items is a collection of (anonymous) JSON entities - each entity has properties Name and Age. As Age is not available for all records in the sample data set, it is inferred as option<int>. The above sample uses Option.iter to print the value only when it is available.

In the previous case, the values of individual properties had common types - string for the Name property and numeric type for Age. However, what if the property of a record can have multiple different types? In that case, the type provider behaves as follows:

type Values = JsonProvider<""" [{"value":94 }, {"value":"Tomas" }] """>

for item in Values.GetSamples() do
  match item.Value.Number, item.Value.String with
  | Some num, _ -> printfn "Numeric: %d" num
  | _, Some str -> printfn "Text: %s" str
  | _ -> printfn "Some other value!"

Here, the Value property is either a number or a string, The type provider generates a type that has an optional property for each possible option, so we can use simple pattern matching on option<int> and option<string> values to distinguish between the two options. This is similar to the handling of heterogeneous arrays.

Note that we have a GetSamples method because the sample is a JSON list. If it was a JSON object, we would have a GetSample method instead.

More complex object type on root level

If you want the root type to be an object type, not an array, but you need more samples at root level, you can use the SampleIsList parameter. Applied to the previous example this would be:

type People2 = JsonProvider<"""
  [ { "name":"John", "age":94 },
    { "name":"Tomas" } ] """, SampleIsList=true>

let person = People2.Parse("""{ "name":"Gustavo" }""")

Loading WorldBank data

Now let's use the type provider to process some real data. We use a data set returned by the WorldBank, which has (roughly) the following structure:

[ { "page": 1, "pages": 1, "total": 53 },
  [ { "indicator": {"value": "Central government debt, total (% of GDP)"},
      "country": {"id":"CZ","value":"Czech Republic"},
    { "indicator": {"value": "Central government debt, total (% of GDP)"},
      "country": {"id":"CZ","value":"Czech Republic"},
      "value":"16.6567773464055","decimal":"1","date":"2010"} ] ]

The response to a request contains an array with two items. The first item is a record with general information about the response (page, total pages, etc.) and the second item is another array which contains the actual data points. For every data point, we get some information and the actual value. Note that the value is passed as a string (for some unknown reason). It is wrapped in quotes, so the provider infers its type as string (and we need to convert it manually).

The following sample generates type based on the data/WorldBank.json file and loads it:

type WorldBank = JsonProvider<"../data/WorldBank.json", ResolutionFolder=__SOURCE_DIRECTORY__>
let doc = WorldBank.GetSample()

Note that we can also load the data directly from the web both in the Load method and in the type provider sample parameter, and there's an asynchronous AsyncLoad method available too:

let wbReq =
  "http://api.worldbank.org/country/cz/indicator/" +

let docAsync =

The doc is an array of heterogeneous types, so the provider generates a type that can be used to get the record and the array, respectively. Note that the provider infers that there is only one record and one array. We can print the data set as follows:

// Print general information
let info = doc.Record
printfn "Showing page %d of %d. Total records %d"
  info.Page info.Pages info.Total

// Print all data points
for record in doc.Array do
  record.Value |> Option.iter (fun value ->
    printfn "%d: %f" record.Date value)

When printing the data points, some of the values might be missing (in the input, the value is null instead of a valid number). This is another example of a heterogeneous type - the type is either Number or some other type (representing null value). This means that record.Value has a Number property (when the value is a number) and we can use it to print the result only when the data point is available.

Parsing Twitter stream

We now look on how to parse tweets returned by the Twitter API. Tweets are quite heterogeneous, so we infer the structure from a list of inputs rather than from just a single input. To do that, we use the file data/TwitterStream.json (containing a list of tweets) and pass an optional parameter SampleIsList=true which tells the provider that the sample is actually a list of samples:

type Tweet = JsonProvider<"../data/TwitterStream.json", SampleIsList=true, ResolutionFolder=__SOURCE_DIRECTORY__>
let text = (omitted)
let tweet = Tweet.Parse(text)

printfn "%s (retweeted %d times)\n:%s"
  tweet.User.Value.Name tweet.RetweetCount.Value tweet.Text.Value

After creating the Tweet type, we parse a single sample tweet and print some details about the tweet. As you can see, the tweet.User property has been inferred as optional (meaning that a tweet might not have an author?) so we unsafely get the value using the Value property. The RetweetCount and Text properties may be also missing, so we also access them unsafely.

Getting and creating GitHub issues

In this example we will now also create JSON in addition to consuming it. Let's start by listing the 5 most recently updated open issues in the FSharp.Data repo.

// GitHub.json downloaded from
// https://api.github.com/repos/fsharp/FSharp.Data/issues
// to prevent rate limit when generating these docs
type GitHub = JsonProvider<"../data/GitHub.json", ResolutionFolder=__SOURCE_DIRECTORY__>

let topRecentlyUpdatedIssues =
    |> Seq.filter (fun issue -> issue.State = "open")
    |> Seq.sortBy (fun issue -> System.DateTimeOffset.Now - issue.UpdatedAt)
    |> Seq.truncate 5

for issue in topRecentlyUpdatedIssues do
    printfn "#%d %s" issue.Number issue.Title

And now let's create a new issue. We look into the documentation at http://developer.github.com/v3/issues/#create-an-issue and we see that we need to post a JSON value similar to this:

let issueSample = """
  "title": "Found a bug",
  "body": "I'm having a problem with this.",
  "assignee": "octocat",
  "milestone": 1,
  "labels": [

This JSON is different from what we got for each issue in the previous API call, so we'll define a new type based on this sample, create an instance, and send a POST request:

type GitHubIssue = JsonProvider<issueSample, RootName="issue">

let newIssue =
    ( "Test issue",
      "This is a test issue created in FSharp.Data documentation",
      assignee = "",
      labels = [| |],
      milestone = 0)
newIssue.JsonValue.Request "https://api.github.com/repos/fsharp/FSharp.Data/issues"

Using JSON provider in a library

You can use the types created by JSON type provider in a public API of a library that you are building, but there is one important thing to keep in mind - when the user references your library, the type provider will be loaded and the types will be generated at that time (the JSON provider is not currently a generative type provider). This means that the type provider will need to be able to access the sample JSON. This works fine when the sample is specified inline, but it won't work when the sample is specified as a local file (unless you distribute the samples with your library).

For this reason, the JSON provider lets you specify samples as embedded resources using the static parameter EmbeddedResource (don't forget then to include the file as EmbeddedResource in the project file). If you are building a library MyLib.dll, you can write:

type WB = JsonProvider<"../data/WorldBank.json",
                       EmbeddedResource="MyLib, MyLib.data.worldbank.json",

You still need to specify the local path, but this is only used when compiling MyLib.dll. When a user of your library references MyLib.dll later, the JSON Type Provider will be able to load MyLib.dll and locate the sample worldbank.json as a resource of the library. When this succeeds, it does not attempt to find the local file and so your library can be used without providing a local copy of the sample JSON files.

Related articles

Multiple items
namespace FSharp

namespace Microsoft.FSharp
Multiple items
namespace FSharp.Data

namespace Microsoft.FSharp.Data
type Simple = JsonProvider<...>
type JsonProvider =
<summary>Typed representation of a JSON document.</summary> <param name='Sample'>Location of a JSON sample file or a string containing a sample JSON document.</param> <param name='SampleIsList'>If true, sample should be a list of individual samples for the inference.</param> <param name='RootName'>The name to be used to the root type. Defaults to `Root`.</param> <param name='Culture'>The culture used for parsing numbers and dates. Defaults to the invariant culture.</param> <param name='Encoding'>The encoding used to read the sample. You can specify either the character set name or the codepage number. Defaults to UTF8 for files, and to ISO-8859-1 the for HTTP requests, unless `charset` is specified in the `Content-Type` response header.</param> <param name='ResolutionFolder'>A directory that is used when resolving relative file references (at design time and in hosted execution).</param> <param name='EmbeddedResource'>When specified, the type provider first attempts to load the sample from the specified resource (e.g. 'MyCompany.MyAssembly, resource_name.json'). This is useful when exposing types generated by the type provider.</param> <param name='InferTypesFromValues'>If true, turns on additional type inference from values. (e.g. type inference infers string values such as "123" as ints and values constrained to 0 and 1 as booleans.)</param>
val simple : JsonProvider<...>.Root
JsonProvider<...>.Parse(text: string) : JsonProvider<...>.Root
Parses the specified JSON string
property JsonProvider<...>.Root.Age: int with get
property JsonProvider<...>.Root.Name: string with get
type Numbers = JsonProvider<...>
val nums : decimal []
JsonProvider<...>.Parse(text: string) : decimal []
Parses the specified JSON string
val total : decimal
module Seq from Microsoft.FSharp.Collections
<summary>Contains operations for working with values of type <see cref="T:Microsoft.FSharp.Collections.seq`1" />.</summary>
val sum : source:seq<'T> -> 'T (requires member ( + ) and member get_Zero)
<summary>Returns the sum of the elements in the sequence.</summary>
<remarks>The elements are summed using the <c>+</c> operator and <c>Zero</c> property associated with the generated type.</remarks>
<param name="source">The input sequence.</param>
<returns>The computed sum.</returns>
type Mixed = JsonProvider<...>
val mixed : JsonProvider<...>.Root
property JsonProvider<...>.Root.Numbers: int [] with get
property JsonProvider<...>.Root.Strings: string [] with get
module String from Microsoft.FSharp.Core
<summary>Functional programming operators for string processing. Further string operations are available via the member functions on strings and other functionality in <a href="http://msdn2.microsoft.com/en-us/library/system.string.aspx">System.String</a> and <a href="http://msdn2.microsoft.com/library/system.text.regularexpressions.aspx">System.Text.RegularExpressions</a> types. </summary>
<category>Strings and Text</category>
val concat : sep:string -> strings:seq<string> -> string
<summary>Returns a new string made by concatenating the given strings with separator <c>sep</c>, that is <c>a1 + sep + ... + sep + aN</c>.</summary>
<param name="sep">The separator string to be inserted between the strings of the input sequence.</param>
<param name="strings">The sequence of strings to be concatenated.</param>
<returns>A new string consisting of the concatenated strings separated by the separation string.</returns>
<exception cref="T:System.ArgumentNullException">Thrown when <c>strings</c> is null.</exception>
type People = JsonProvider<...>
val item : JsonProvider<...>.Root
JsonProvider<...>.GetSamples() : JsonProvider<...>.Root []
val printf : format:Printf.TextWriterFormat<'T> -> 'T
<summary>Print to <c>stdout</c> using the given format.</summary>
<param name="format">The formatter.</param>
<returns>The formatted result.</returns>
property JsonProvider<...>.Root.Age: Option<int> with get
module Option from Microsoft.FSharp.Core
<summary>Contains operations for working with options.</summary>
val iter : action:('T -> unit) -> option:'T option -> unit
<summary><c>iter f inp</c> executes <c>match inp with None -&gt; () | Some x -&gt; f x</c>.</summary>
<param name="action">A function to apply to the option value.</param>
<param name="option">The input option.</param>
<example><code> None |&gt; Option.iter (printfn "%s") // does nothing Some "Hello world" |&gt; Option.iter (printfn "%s") // prints "Hello world" </code></example>
<returns>Unit if the option is None, otherwise it returns the result of applying the predicate to the option value.</returns>
val printfn : format:Printf.TextWriterFormat<'T> -> 'T
<summary>Print to <c>stdout</c> using the given format, and add a newline.</summary>
<param name="format">The formatter.</param>
<returns>The formatted result.</returns>
type Values = JsonProvider<...>
property JsonProvider<...>.Root.Value: JsonProvider<...>.IntOrString with get
property JsonProvider<...>.IntOrString.Number: Option<int> with get
property JsonProvider<...>.IntOrString.String: Option<string> with get
union case Option.Some: Value: 'T -> Option<'T>
<summary>The representation of "Value of type 'T"</summary>
<param name="Value">The input value.</param>
<returns>An option representing the value.</returns>
val num : int
val str : string
type People2 = JsonProvider<...>
val person : JsonProvider<...>.Root
type WorldBank = obj
val doc : obj
val wbReq : string
val docAsync : obj
val info : obj
val record : obj
val value : float
type Tweet = obj
val text : string
""" {"in_reply_to_status_id_str":null,"text":"\u5927\u91d1\u6255\u3063\u3066\u904a\u3070\u3057\u3066\u3082\u3089\u3046\u3002\u3082\u3046\u3053\u306e\u4e0a\u306a\u3044\u8d05\u6ca2\u3002\u3067\u3082\uff0c\u5b9f\u969b\u306b\u306f\u305d\u306e\u8d05\u6ca2\u306e\u672c\u8cea\u3092\u6e80\u55ab\u3067\u304d\u308b\u4eba\u306f\u9650\u3089\u308c\u3066\u308b\u3002\u305d\u3053\u306b\u76ee\u306b\u898b\u3048\u306a\u3044\u968e\u5c64\u304c\u3042\u308b\u3068\u304a\u3082\u3046\u3002","in_reply_to_user_id_str":null,"retweet_count":0,"geo":null,"source":"web","retweeted":false,"truncated":false,"id_str":"263290764686155776","entities":{"user_mentions":[],"hashtags":[],"urls":[]},"in_reply_to_user_id":null,"in_reply_to_status_id":null,"place":null,"coordinates":null,"in_reply_to_screen_name":null,"created_at":"Tue Oct 30 14:46:24 +0000 2012","user":{"notifications":null,"contributors_enabled":false,"time_zone":"Tokyo","profile_background_color":"FFFFFF","location":"Kodaira Tokyo Japan","profile_background_tile":false,"profile_image_url_https":"https:\/\/si0.twimg.com\/profile_images\/1172376796\/70768_100000537851636_3599485_q_normal.jpg","default_profile_image":false,"follow_request_sent":null,"profile_sidebar_fill_color":"17451B","description":"KS(Green62)\/WasedaUniv.(Schl Adv Sci\/Eng)\/SynBio\/ChronoBio\/iGEM2010-2012\/Travel\/Airplane\/ \u5bfa\u30fb\u5ead\u3081\u3050\u308a","favourites_count":17,"screen_name":"Merlin_wand","profile_sidebar_border_color":"000000","id_str":"94788486","verified":false,"lang":"ja","statuses_count":8641,"profile_use_background_image":true,"protected":false,"profile_image_url":"http:\/\/a0.twimg.com\/profile_images\/1172376796\/70768_100000537851636_3599485_q_normal.jpg","listed_count":31,"geo_enabled":true,"created_at":"Sat Dec 05 13:07:32 +0000 2009","profile_text_color":"000000","name":"Marin","profile_background_image_url":"http:\/\/a0.twimg.com\/profile_background_images\/612807391\/twitter_free1.br.jpg","friends_count":629,"url":null,"id":94788486,"is_translator":false,"default_profile":false,"following":null,"profile_background_image_url_https":"https:\/\/si0.twimg.com\/profile_background_images\/612807391\/twitter_free1.br.jpg","utc_offset":32400,"profile_link_color":"ADADAD","followers_count":426},"id":263290764686155776,"contributors":null,"favorited":false} """
val tweet : obj
type GitHub = obj
val topRecentlyUpdatedIssues : seq<obj>
val filter : predicate:('T -> bool) -> source:seq<'T> -> seq<'T>
<summary>Returns a new collection containing only the elements of the collection for which the given predicate returns "true". This is a synonym for Seq.where.</summary>
<remarks>The returned sequence may be passed between threads safely. However, individual IEnumerator values generated from the returned sequence should not be accessed concurrently. Remember sequence is lazy, effects are delayed until it is enumerated.</remarks>
<param name="predicate">A function to test whether each item in the input sequence should be included in the output.</param>
<param name="source">The input sequence.</param>
<returns>The result sequence.</returns>
<exception cref="T:System.ArgumentNullException">Thrown when the input sequence is null.</exception>
val issue : obj
val sortBy : projection:('T -> 'Key) -> source:seq<'T> -> seq<'T> (requires comparison)
<summary>Applies a key-generating function to each element of a sequence and yield a sequence ordered by keys. The keys are compared using generic comparison as implemented by <see cref="M:Microsoft.FSharp.Core.Operators.compare" />.</summary>
<remarks>This function returns a sequence that digests the whole initial sequence as soon as that sequence is iterated. As a result this function should not be used with large or infinite sequences. The function makes no assumption on the ordering of the original sequence and uses a stable sort, that is the original order of equal elements is preserved.</remarks>
<param name="projection">A function to transform items of the input sequence into comparable keys.</param>
<param name="source">The input sequence.</param>
<returns>The result sequence.</returns>
<exception cref="T:System.ArgumentNullException">Thrown when the input sequence is null.</exception>
namespace System
Multiple items
[<Struct>] type DateTimeOffset = new : dateTime: DateTime -> unit + 5 overloads member Add : timeSpan: TimeSpan -> DateTimeOffset member AddDays : days: float -> DateTimeOffset member AddHours : hours: float -> DateTimeOffset member AddMilliseconds : milliseconds: float -> DateTimeOffset member AddMinutes : minutes: float -> DateTimeOffset member AddMonths : months: int -> DateTimeOffset member AddSeconds : seconds: float -> DateTimeOffset member AddTicks : ticks: int64 -> DateTimeOffset member AddYears : years: int -> DateTimeOffset ...
<summary>Represents a point in time, typically expressed as a date and time of day, relative to Coordinated Universal Time (UTC).</summary>

System.DateTimeOffset ()
System.DateTimeOffset(dateTime: System.DateTime) : System.DateTimeOffset
System.DateTimeOffset(dateTime: System.DateTime, offset: System.TimeSpan) : System.DateTimeOffset
System.DateTimeOffset(ticks: int64, offset: System.TimeSpan) : System.DateTimeOffset
System.DateTimeOffset(year: int, month: int, day: int, hour: int, minute: int, second: int, offset: System.TimeSpan) : System.DateTimeOffset
System.DateTimeOffset(year: int, month: int, day: int, hour: int, minute: int, second: int, millisecond: int, offset: System.TimeSpan) : System.DateTimeOffset
System.DateTimeOffset(year: int, month: int, day: int, hour: int, minute: int, second: int, millisecond: int, calendar: System.Globalization.Calendar, offset: System.TimeSpan) : System.DateTimeOffset
property System.DateTimeOffset.Now: System.DateTimeOffset with get
<summary>Gets a <see cref="T:System.DateTimeOffset" /> object that is set to the current date and time on the current computer, with the offset set to the local time's offset from Coordinated Universal Time (UTC).</summary>
<returns>A <see cref="T:System.DateTimeOffset" /> object whose date and time is the current local time and whose offset is the local time zone's offset from Coordinated Universal Time (UTC).</returns>
val truncate : count:int -> source:seq<'T> -> seq<'T>
<summary>Returns a sequence that when enumerated returns at most N elements.</summary>
<param name="count">The maximum number of items to enumerate.</param>
<param name="source">The input sequence.</param>
<returns>The result sequence.</returns>
<exception cref="T:System.ArgumentNullException">Thrown when the input sequence is null.</exception>
Multiple items
type LiteralAttribute = inherit Attribute new : unit -> LiteralAttribute
<summary>Adding this attribute to a value causes it to be compiled as a CLI constant literal.</summary>

new : unit -> LiteralAttribute
val issueSample : string = " { "title": "Found a bug", "body": "I'm having a problem with this.", "assignee": "octocat", "milestone": 1, "labels": [ "Label1", "Label2" ] } "
type GitHubIssue = JsonProvider<...>
val newIssue : JsonProvider<...>.Issue
property Runtime.BaseTypes.IJsonDocument.JsonValue: JsonValue with get
member JsonValue.Request : url:string * ?httpMethod:string * ?headers:seq<string * string> -> HttpResponse
type WB = obj