category feedAeson cookbookCookbookseditdelete

This category is a work in progress






A collection of recipes for processing JSON with aeson, the de-facto JSON library for Haskell.

All examples have been tested with aeson-1.4.2.0 (2019-01-06).

Table of contents

Basic operations

Automatic encoding/decoding

  • Overview – encoding and decoding standard Haskell types.

  • Objects – encoding and decoding Haskell records as JSON objects. Covers common cases like mismatching field names and optional fields.

  • Enums – encoding and decoding Haskell enums. Covers common cases like mismatching constructor names. Also explains how to encode enums to objects instead of strings.

  • Newtypes – common problems and pitfalls when encoding newtypes specifically.

  • ADTs – goes deep into all possible Aeson options for encoding arbitrary ADTs (types with several constructors and possibly several fields per constructor).

Writing custom encoders/decoders

  • The Value type
  • TODO

Generating JSON

  • TODO

Miscellaneous topics

  • JSONPath
  • Encoding
  • FromJSONKey
  • withEmbeddedJSON
  • Data.Aeson.QQ
  • Template Haskell
  • Pretty-printing
  • Lenses
edit description
or press Ctrl+Enter to savemarkdown supported
#
Encoding
other
move item up move item down edit item info delete item
Summary edit summary

Prerequisites: none.

Encoding to JSON

To encode a piece of data to JSON, use encode.

-- Import the main Aeson module
> import Data.Aeson

-- Encode a list to JSON
> encode [1, 2, 3]
"[1,2,3]" :: ByteString

encode works for all types with a ToJSON instance. Instances for common types (lists, maps, strings, numbers, timestamps, etc) are provided out of the box.

-- | Efficiently serialize a JSON value as a lazy ByteString.
encode :: ToJSON a => a -> ByteString

Using different string types for output

Sometimes you might need to encode to types other than lazy ByteString. Use the following snippets:

import Data.Aeson
import qualified Data.Aeson.Text as A
import qualified Data.ByteString.Lazy as BL
import qualified Data.Text.Lazy as TL

encode                              -- encode to ByteString (lazy)
BL.toStrict . encode                -- encode to ByteString (strict)
A.encodeToLazyText                  -- encode to Text (lazy)
TL.toStrict . A.encodeToLazyText    -- encode to Text (strict)
TL.unpack . A.encodeToLazyText      -- encode to String
Summary quit editing summary
#
Decoding
other
move item up move item down edit item info delete item
Summary edit summary

Prerequisites: none.

Decoding from JSON

To decode a piece of data, use decode. If it's not clear from the context what the resulting type will be, you'll have to specify it explicitly.

-- Allow string literal syntax for 'ByteString'
> :set -XOverloadedStrings

-- Import the main Aeson module
> import Data.Aeson

-- Decode a list of Ints from JSON
> decode "[1,2,3]" :: Maybe [Int]
Just [1,2,3]

-- Try to decode a malformed list of Ints from JSON
> decode "[1,2,null]" :: Maybe [Int]
Nothing

decode works for all types with a FromJSON instance. Instances for common types (lists, maps, strings, numbers, timestamps, etc) are provided out of the box.

-- | Efficiently deserialize a JSON value from a lazy ByteString. If
-- this fails due to incomplete or invalid input, Nothing is returned.
--
-- The input must consist solely of a JSON document, with no trailing
-- data except for whitespace.
decode :: FromJSON a => ByteString -> Maybe a

Using different string types as input

Sometimes you might need to decode from types other than ByteString. Use the following snippets:

import Data.Aeson
import qualified Data.ByteString.Lazy as BL
import qualified Data.Text.Encoding as T
import qualified Data.Text.Lazy as TL
import qualified Data.Text.Lazy.Encoding as TL

decode                              -- decode from ByteString (lazy)
decodeStrict                        -- decode from ByteString (strict)
decode . TL.encodeUtf8              -- decode from Text (lazy)
decodeStrict . T.encodeUtf8         -- decode from Text (strict)
decode . TL.encodeUtf8 . TL.pack    -- decode from String
Summary quit editing summary
#
Decoding with error reporting
other
move item up move item down edit item info delete item
Summary edit summary

Prerequisites: Decoding.

Conversion errors

eitherDecode reports an error message when JSON is valid but doesn't have the right "shape". The path in the error message uses standard JSONPath syntax.

-- Allow string literal syntax for 'ByteString'
> :set -XOverloadedStrings

-- Import the main Aeson module
> import Data.Aeson

-- Decode an [[Int]] list from malformed JSON
> eitherDecode "[[1,2],[null,3]]" :: Either String [[Int]]
Left "Error in $[1][0]: expected Int, encountered Null"
-- | Like 'decode' but returns an error message when decoding fails.
eitherDecode :: FromJSON a => ByteString -> Either String a

Extracting the conversion error path

To access the error path reported by eitherDecode, you will need to use Aeson's internal functions:

-- Allow string literal syntax for 'ByteString'
> :set -XOverloadedStrings

-- Import Aeson's internals
> import Data.Aeson.Internal
> import Data.Aeson.Parser.Internal

-- Decode an [[Int]] list from malformed JSON,
-- getting back the JSONPath with the error position
> eitherDecodeWith jsonEOF ifromJSON "[[1,2],[null,3]]" :: Either (JSONPath, String) [[Int]]
Left ([Index 1,Index 0],"expected Int, encountered Null")
Summary quit editing summary
#
Encoding/decoding objects
other
move item up move item down edit item info delete item
Summary edit summary

Prerequisites: Basic encoding/decoding.

Default encoding

The simplest way to encode/decode objects is to add deriving Generic to your data declaration, and write empty FromJSON and ToJSON instances:

{-# LANGUAGE DeriveGeneric #-}

import Data.Aeson
import GHC.Generics

data User = User { userName :: Maybe String, userId :: Int }
    deriving (Show, Generic)

instance FromJSON User
instance ToJSON User

Now you can use encode and decode on the data type. Aeson will assume that field names in the data type definition are the same as in JSON.

-- Encoding a JSON object
> encode (User (Just "Joe") 13)
"{\"userName\":\"Joe\",\"userId\":13}"

-- Decoding a JSON object
> decode "{\"userName\":\"Joe\",\"userId\":13}" :: Maybe User
Just (User {userName = Just "Joe", userId = 13})

The next sections show how you can customize Aeson's Options to change the way objects are encoded.

Renaming fields

Removing prefixes

You can remove the "user" prefix by customizing fieldLabelModifier. This will affect both encoding and decoding.

// The encoding we want to get
{"name":"Joe","id":13}
-- How we get it
import Data.Char

lower1 :: String -> String
lower1 (c:cs) = toLower c : cs
lower1 [] = []

instance FromJSON User where
    parseJSON = genericParseJSON defaultOptions { fieldLabelModifier = lower1 . drop 4 }

instance ToJSON User where
    toJSON = genericToJSON defaultOptions { fieldLabelModifier = lower1 . drop 4 }

Snake-casing field names

Or you can use Aeson's camelTo2 function to switch to snake-case encoding:

// The encoding we want to get
{"user_name":"Joe","user_id":13}
-- How we get it
instance FromJSON User where
    parseJSON = genericParseJSON defaultOptions { fieldLabelModifier = camelTo2 '_' }

instance ToJSON User where
    toJSON = genericToJSON defaultOptions { fieldLabelModifier = camelTo2 '_' }

Arbitrary field names

In fact, you can pick any field names you want:

// The encoding we want to get
{"display_name":"Joe","entity_id":13}
-- How we get it
toLegacy :: String -> String
toLegacy "userName" = "display_name"
toLegacy "userId" = "entity_id"
toLegacy s = s

instance FromJSON User where
    parseJSON = genericParseJSON defaultOptions { fieldLabelModifier = toLegacy }

instance ToJSON User where
    toJSON = genericToJSON defaultOptions { fieldLabelModifier = toLegacy }

TODO link to custom encoder example

Omitting empty fields

If you set userName to Nothing, by default it will be encoded as null:

// By default 'Nothing' is encoded to 'null'
{"userName":null,"userId":13}

You can customize omitNothingFields to omit such fields instead:

// The encoding we want to get instead
{"userId":13}
-- How we get it
instance FromJSON User where
    parseJSON = genericParseJSON defaultOptions { omitNothingFields = True }

instance ToJSON User where
    toJSON = genericToJSON defaultOptions { omitNothingFields = True }

When decoding, omitNothingFields has no effect – Aeson will always treat null the same as the absence of a field.

Summary quit editing summary
#
Encoding/decoding enums
other
move item up move item down edit item info delete item
Summary edit summary

Prerequisites: Basic encoding/decoding.

Default encoding

The simplest way to encode/decode enums is to add deriving Generic to your data declaration, and write empty FromJSON and ToJSON instances:

{-# LANGUAGE DeriveGeneric #-}

import Data.Aeson
import GHC.Generics

data Color = ColorRed | ColorBlue | ColorGreen
    deriving (Show, Generic)

instance FromJSON Color
instance ToJSON Color

Now you can use encode and decode on the data type. Aeson will encode it as a string.

-- Encoding an enum
> encode ColorGreen
"\"ColorGreen\""

-- Decoding an enum
> decode "\"ColorGreen\"" :: Maybe Color
Just ColorGreen

Aeson is case-sensitive and will not decode a string that does not exactly correspond to one of the constructors:

-- Decoding "colorGreen" fails
> eitherDecode "\"colorGreen\"" :: Either String Color
Left "Error in $: The key \"colorGreen\" was not found"

The next sections show how you can customize Aeson's Options to change the way enums are encoded.

Renaming constructors

Removing prefixes

You can remove the "Color" prefix by customizing constructorTagModifier. This will affect both encoding and decoding.

// The encoding we want to get
"Green"
-- How we get it
instance FromJSON Color where
    parseJSON = genericParseJSON defaultOptions { constructorTagModifier = drop 5 }

instance ToJSON Color where
    toJSON = genericToJSON defaultOptions { constructorTagModifier = drop 5 }

Snake-casing

Or you can use Aeson's camelTo2 function to switch to snake-case encoding:

// The encoding we want to get
"color_green"
-- How we get it
instance FromJSON Color where
    parseJSON = genericParseJSON defaultOptions { constructorTagModifier = camelTo2 '_' }

instance ToJSON Color where
    toJSON = genericToJSON defaultOptions { constructorTagModifier = camelTo2 '_' }

Arbitrary constructor names

In fact, you can pick any constructor names you want:

// The encoding we want to get for 'ColorGreen'
"grass"
-- How we get it
toLegacy :: String -> String
toLegacy "ColorRed" = "blood"
toLegacy "ColorGreen" = "grass"
toLegacy "ColorBlue" = "sky"
toLegacy s = s

instance FromJSON Color where
    parseJSON = genericParseJSON defaultOptions { constructorTagModifier = toLegacy }

instance ToJSON Color where
    toJSON = genericToJSON defaultOptions { constructorTagModifier = toLegacy }

TODO link to custom encoder example

Encoding to an object

It is also possible to encode enums to an object with a single field by customizing allNullaryToStringTag:

// The encoding we want to get
{"tag":"ColorGreen"}
-- How we get it
instance FromJSON Color where
    parseJSON = genericParseJSON defaultOptions { allNullaryToStringTag = False }

instance ToJSON Color where
    toJSON = genericToJSON defaultOptions { allNullaryToStringTag = False }

The field name ("tag" in our case) can be changed by customizing sumEncoding:

// The encoding we want to get
{"value":"ColorGreen"}
-- How we get it
instance FromJSON Color where
    parseJSON = genericParseJSON defaultOptions
        { allNullaryToStringTag = False
        , sumEncoding = TaggedObject "value" "" }

instance ToJSON Color where
    toJSON = genericToJSON defaultOptions
        { allNullaryToStringTag = False
        , sumEncoding = TaggedObject "value" "" }
Summary quit editing summary
#
Encoding/decoding newtypes
other
move item up move item down edit item info delete item
Summary edit summary

Prerequisites: Basic encoding/decoding.

Unwrapping newtypes

If you want the newtype to be encoded just like the value contained in it, you should use -XGeneralizedNewtypeDeriving instead of defining an instance:

{-# LANGUAGE DerivingStrategies #-}
{-# LANGUAGE GeneralizedNewtypeDeriving #-}

import Data.Aeson

newtype X = X { getX :: Int }
    deriving (Show)
    deriving newtype (FromJSON, ToJSON)
> encode (X 13)
"13"

The -XDerivingStrategies extension (available since GHC 8.2) allows the deriving newtype syntax. Without it, GHC will use the Generic instance if -XDeriveAnyClass is enabled, which would lead to hard-to-spot bugs.

Unwrapping single-field data

If the type is a data and not a newtype, the method described in this section will not work. Instead it is recommended to write custom instances (TODO link):

import Data.Aeson

data X = X { getX :: Int }
    deriving (Show)

instance ToJSON X where
    toJSON = toJSON . getX

instance FromJSON X where
    parseJSON = fmap X . parseJSON
> encode (X 13)
"13"

The Options approach

You can also create custom Options to do the same. Specifically, you will have to tweak the unwrapUnaryRecords option:

{-# LANGUAGE DeriveGeneric #-}

import Data.Aeson
import GHC.Generics

data X = X { getX :: Int }
    deriving (Show, Generic)

instance FromJSON X where
    parseJSON = genericParseJSON defaultOptions { unwrapUnaryRecords = True }

instance ToJSON X where
    toJSON = genericToJSON defaultOptions { unwrapUnaryRecords = True }
> encode (X 13)
"13"

Encoding newtypes as objects

If the newtype has a named field, Aeson will represent it as an object automatically.

If the field is unnamed, the easiest way to encode it like an object is to write custom instances (TODO link):

{-# LANGUAGE OverloadedStrings #-}

import Data.Aeson

newtype X = X Int
    deriving (Show)

instance ToJSON X where
    toJSON (X a) = object ["getX" .= a]

instance FromJSON X where
    parseJSON = withObject "X" $ \o -> X <$> o .: "getX"
> encode (X 13)
"{\"getX\":13}"

The Options approach

You can also create custom Options to do the same, though in a much more cumbersome way.

First you need to force Aeson to treat your type like a sum type by tweaking tagSingleConstructors. Then sumEncoding and constructorTagModifier can be set to treat it like an object:

{-# LANGUAGE DeriveGeneric #-}

import Data.Aeson
import GHC.Generics

newtype X = X Int
    deriving (Show, Generic)

singleFieldOptions :: String -> Options
singleFieldOptions fieldName = defaultOptions
    { tagSingleConstructors = True
    , sumEncoding = ObjectWithSingleField
    , constructorTagModifier = const fieldName
    }

instance FromJSON X where
    parseJSON = genericParseJSON (singleFieldOptions "getX")

instance ToJSON X where
    toJSON = genericToJSON (singleFieldOptions "getX")
> encode (X 13)
"{\"getX\":13}"
Summary quit editing summary
#
Encoding/decoding ADTs
other
move item up move item down edit item info delete item
Summary edit summary

TODO

Summary quit editing summary