Header menu logo FSharp.Data

BinderScriptNotebook

JSON Parser

The F# JSON Type Provider is built on top of an efficient JSON parser written in F#.

When working with well-defined JSON documents, it is easier to use the type provider, but in a more dynamic scenario or when writing quick and simple scripts, the parser might be a simpler option.

Loading JSON documents

To load a sample JSON document, we first need to reference the FSharp.Data package.

open FSharp.Data

The FSharp.Data namespace contains the JsonValue type that can be used to parse strings formatted using JSON as follows:

let info =
    JsonValue.Parse(
        """
    { "name": "Tomas", "born": 1985,
      "siblings": [ "Jan", "Alexander" ] } """
    )
val info: JsonValue =
  {
  "name": "Tomas",
  "born": 1985,
  "siblings": [
    "Jan",
    "Alexander"
  ]
}

The parsed value can be processed using pattern matching - the JsonValue type is a discriminated union with cases such as Record, Collection and others that can be used to examine the structure.

Using JSON extensions

We do not cover this technique in this introduction. Instead, we look at a number of extensions that become available after opening the JsonExtensions module. Once opened, we can write:

Methods that may need to parse a numeric value or date (such as AsFloat and AsDateTime) receive an optional culture parameter.

The following example shows how to process the sample JSON value:

open FSharp.Data.JsonExtensions

// Print name and birth year
let n = info?name
printfn "%s (%d)" (info?name.AsString()) (info?born.AsInteger())

// Print names of all siblings
for sib in info?siblings do
    printfn "%s" (sib.AsString())
Tomas (1985)
Jan
Alexander
val n: JsonValue = "Tomas"
val it: unit = ()

Note that the JsonValue type does not actually implement the IEnumerable<'T> interface (meaning that it cannot be passed to Seq.xyz functions). It only has the GetEnumerator method, which makes it possible to use it in sequence expressions and with the for loop.

Parsing WorldBank response

To look at a more complex example, consider a sample document data/WorldBank.json which was obtained as a response to a WorldBank request (you can access the WorldBank data more conveniently using a type provider). The document looks as follows:

[ { "page": 1, "pages": 1, "total": 53 },
  [ { "indicator": {"value": "Central government debt, total (% of GDP)"},
      "country": {"id":"CZ","value":"Czech Republic"},
      "value":null,"decimal":"1","date":"2000"},
    { "indicator": {"value": "Central government debt, total (% of GDP)"},
      "country": {"id":"CZ","value":"Czech Republic"},
      "value":"16.6567773464055","decimal":"1","date":"2010"} ] ]

The document is formed by an array that contains a record as the first element and a collection of data points as the second element. The following code reads the document and parses it:

let value = JsonValue.Load(__SOURCE_DIRECTORY__ + "../../data/WorldBank.json")

Note that we can also load the data directly from the web, and there's an asynchronous version available too:

let wbReq =
    "http://api.worldbank.org/country/cz/indicator/"
    + "GC.DOD.TOTL.GD.ZS?format=json"

let valueAsync = JsonValue.AsyncLoad(wbReq)
val wbReq: string =
  "http://api.worldbank.org/country/cz/indicator/GC.DOD.TOTL.GD."+[14 chars]
val valueAsync: Async<JsonValue>

To split the top-level array into the first record (with overall information) and the collection of data points, we use pattern matching and match the value against the JsonValue.Array constructor:

match value with
| JsonValue.Array [| info; data |] ->
    // Print overall information
    let page, pages, total = info?page, info?pages, info?total
    printfn "Showing page %d of %d. Total records %d" (page.AsInteger()) (pages.AsInteger()) (total.AsInteger())

    // Print every non-null data point
    for record in data do
        if record?value <> JsonValue.Null then
            printfn "%d: %f" (record?date.AsInteger()) (record?value.AsFloat())
| _ -> printfn "failed"
Showing page 1 of 1. Total records 53
2010: 35.142297
2009: 31.034880
2008: 25.475164
2007: 24.193320
2006: 23.708055
2005: 22.033462
2004: 20.108379
2003: 18.267725
2002: 15.425565
2001: 14.874434
2000: 13.218869
1999: 11.356696
1998: 10.178780
1997: 10.153566
1996: 10.520301
1995: 12.707834
1994: 14.781808
1993: 16.656777
val it: unit = ()

The value property of a data point is not always available - as demonstrated above, the value may be null. In that case, we want to skip the data point. To check whether the property is null we simply compare it with JsonValue.Null.

The date values will be parsed as DateTimeOffset if there is an offset present. However, for a mixed collection of DateTime (that is, without the offset) and DateTimeOffset values, the type of the collection will be collection of DateTime after parsing. Also note that the date and value properties are formatted as strings in the source file (e.g. "1990") instead of numbers (e.g. 1990). When you try accessing the value as an integer or float, the JsonValue automatically parses the string into the desired format. In general, the API attempts to be as tolerant as possible when parsing the file.

Related articles

Multiple items
namespace FSharp

--------------------
namespace Microsoft.FSharp
Multiple items
namespace FSharp.Data

--------------------
namespace Microsoft.FSharp.Data
val info: JsonValue
type JsonValue = | String of string | Number of decimal | Float of float | Record of properties: (string * JsonValue) array | Array of elements: JsonValue array | Boolean of bool | Null member Request: url: string * [<Optional>] ?httpMethod: string * [<Optional>] ?headers: (string * string) seq -> HttpResponse member RequestAsync: url: string * [<Optional>] ?httpMethod: string * [<Optional>] ?headers: (string * string) seq -> Async<HttpResponse> member ToString: saveOptions: JsonSaveOptions * ?indentationSpaces: int -> string + 2 overloads member WriteTo: w: TextWriter * saveOptions: JsonSaveOptions * ?indentationSpaces: int -> unit static member AsyncLoad: uri: string * [<Optional>] ?encoding: Encoding -> Async<JsonValue> static member Load: stream: Stream -> JsonValue + 2 overloads static member Parse: text: string -> JsonValue static member ParseMultiple: text: string -> JsonValue seq static member TryParse: text: string -> JsonValue option
<summary> Represents a JSON value. Large numbers that do not fit in the Decimal type are represented using the Float case, while smaller numbers are represented as decimals to avoid precision loss. </summary>
static member JsonValue.Parse: text: string -> JsonValue
Multiple items
module JsonExtensions from FSharp.Data
<summary> Provides the dynamic operator for getting a property of a JSON object </summary>

--------------------
type JsonExtensions = static member AsArray: x: JsonValue -> JsonValue array static member AsBoolean: x: JsonValue -> bool static member AsDateTime: x: JsonValue * [<Optional>] ?cultureInfo: CultureInfo -> DateTime static member AsDateTimeOffset: x: JsonValue * [<Optional>] ?cultureInfo: CultureInfo -> DateTimeOffset static member AsDecimal: x: JsonValue * [<Optional>] ?cultureInfo: CultureInfo -> decimal static member AsFloat: x: JsonValue * [<Optional>] ?cultureInfo: CultureInfo * [<Optional>] ?missingValues: string array -> float static member AsGuid: x: JsonValue -> Guid static member AsInteger: x: JsonValue * [<Optional>] ?cultureInfo: CultureInfo -> int static member AsInteger64: x: JsonValue * [<Optional>] ?cultureInfo: CultureInfo -> int64 static member AsString: x: JsonValue * [<Optional>] ?cultureInfo: CultureInfo -> string ...
<summary> Extension methods with operations on JSON values </summary>
val n: JsonValue
val printfn: format: Printf.TextWriterFormat<'T> -> 'T
val sib: JsonValue
static member JsonExtensions.AsString: x: JsonValue * [<System.Runtime.InteropServices.Optional>] ?cultureInfo: System.Globalization.CultureInfo -> string
val value: JsonValue
static member JsonValue.Load: reader: System.IO.TextReader -> JsonValue
static member JsonValue.Load: stream: System.IO.Stream -> JsonValue
static member JsonValue.Load: uri: string * [<System.Runtime.InteropServices.Optional>] ?encoding: System.Text.Encoding -> JsonValue
val wbReq: string
val valueAsync: Async<JsonValue>
static member JsonValue.AsyncLoad: uri: string * [<System.Runtime.InteropServices.Optional>] ?encoding: System.Text.Encoding -> Async<JsonValue>
union case JsonValue.Array: elements: JsonValue array -> JsonValue
val data: JsonValue
val page: JsonValue
val pages: JsonValue
val total: JsonValue
static member JsonExtensions.AsInteger: x: JsonValue * [<System.Runtime.InteropServices.Optional>] ?cultureInfo: System.Globalization.CultureInfo -> int
val record: JsonValue
union case JsonValue.Null: JsonValue
static member JsonExtensions.AsFloat: x: JsonValue * [<System.Runtime.InteropServices.Optional>] ?cultureInfo: System.Globalization.CultureInfo * [<System.Runtime.InteropServices.Optional>] ?missingValues: string array -> float

Type something to start searching.