PostgreSQL JSON Functions

PostgreSQL jsonb_set() Function

Summary: in this tutorial, you will learn how to use the PostgreSQL jsonb_set() function to replace an existing value specified by a path with a new value in a JSON document.

Introduction to the PostgreSQL jsonb_set() function

The jsonb_set() function allows you to replace an existing value specified by a path with a new value in a JSON document of the JSONB type.

More specifically, the jsonb_set() function allows you to replace an array element or key/value in a JSON object, or nested combinations of them.

Here’s the syntax of the jsonb_set() function:

jsonb_set(
   target jsonb, 
   path text[], 
   new_value jsonb
   [, create_missing boolean]
)Code language: SQL (Structured Query Language) (sql)

In this syntax:

  • target: This is the original JSON document of the JSONB type that you want to modify.
  • path: This is an array of text elements that specifies the path to the key where you want to insert or update the data.
  • new_value is the new JSONB value that you want to set at the specified path.
  • create_missing: This is an optional boolean parameter indicating whether you want to create missing keys if they do not exist. It defaults to true, meaning that the function will create a new key if you attempt to set it for a key that does not exist.

The jsonb_set() function returns the modified JSON document with the new_value set at a specified path.

PostgreSQL jsonb_set() function examples

Let’s explore some examples of using the PostgreSQL jsonb_set() function

1) Updating an existing element in a JSON array

The following example uses the jsonb_set() function to update an existing element in a JSON array:

 SELECT jsonb_set('[1,2,3]', '{0}', '-1');Code language: SQL (Structured Query Language) (sql)

Output:

 jsonb_set
------------
[-1, 2, 3]
(1 row)

In this example:

  • The original array is [1,2,3].
  • The path {0} indicates the first element of the array.
  • The number -1 is the new value.

The jsonb_set() function sets the first element of the array to -1 and returns the modified document.

To insert the number 4 after the 3rd element, you use a non-existing path to the 4th element as follows:

 SELECT jsonb_set('[1,2,3]', '{4}', '4');Code language: SQL (Structured Query Language) (sql)

Output:

  jsonb_set
--------------
[1, 2, 3, 4]
(1 row)

2) Updating an element in a nested JSON array

The following example uses the jsonb_set() function to update an element in a nested array:

SELECT 
jsonb_set(
'[1,2,[4,5],6]', '{2,0}', '3'
);

Output:

     jsonb_set
-------------------
[1, 2, [3, 5], 6]
(1 row)

In this example:

  • The original array is [1,2,[4,5],6].
  • The path {2, 0}, 2 specifies the second element of the array which is the nested array [4,5], and 0 specifies the first element of the nested array.
  • 3 is the new value.

Therefore the jsonb_set() function changes the number 4 as the first element of the nested array [4,5] to 3.

3) Updating data in a JSON object

The following example uses the jsonb_set() to update the value of a key in a JSON object:

SELECT 
jsonb_set('{"name": "Jane Doe"}', '{name}', '"Jane Smith"');
Code language: SQL

Output:

       jsonb_set
------------------------
{"name": "Jane Smith"}
(1 row)

In this example:

  • {“name”: “Jane Doe”} is the original object.
  • {name} is the path that indicates the name property (or key).
  • “Jane Smith” is the new value to update.

Therefore, the jsonb_set() set the value of the name key in the JSON object to “Jane Smith”.

Note that if you attempt to set a key that does not exist, you’ll get an error, the jsonb_set will insert it. For example:

SELECT jsonb_set('{"name": "Jane Doe"}', '{age}', '25');Code language: JavaScript (javascript)

Output:

            jsonb_set
---------------------------------
{"age": 25, "name": "Jane Doe"}
(1 row)

But if you set the create_missing parameter to false, the function will not insert a new key/value pair:

SELECT 
jsonb_set(
'{"name": "Jane Doe"}', '{age}',
'25',
false
);

Output:

      jsonb_set
----------------------
{"name": "Jane Doe"}
(1 row)

4) Updating a value in a nested JSON object

The following example uses the jsonb_set() to modify a key/value pair in a nested JSON object:

SELECT 
jsonb_set(
'{"name":"John Doe", "address" : { "city": "San Francisco"}}',
'{address,city}', '"San Jose"'
);

Output:

                       jsonb_set
-------------------------------------------------------
{"name": "John Doe", "address": {"city": "San Jose"}}
(1 row)

In this example:

  • {"name":"John Doe", "address" : { "city": "San Francisco"}} is the original JSON object.
  • {address, city} is a path that specifies the address key whose value is an object and the city is the key of the address object that will be modified.
  • "San Jose" is the value of the city key.

Therefore, the jsonb_set() function updates the city with the value San Jose in the address object of the JSON document.

5) Updating an element in an array of a nested object

The following example uses the jsonb_set() to update an element in an array of a nested object

SELECT 
jsonb_set(
'{"name": "John", "skills" : ["PostgreSQL", "API"]}',
'{skills,1}',
'"Web Dev"'
);

Output:

                       jsonb_set
-------------------------------------------------------
{"name": "John", "skills": ["PostgreSQL", "Web Dev"]}
(1 row)

In this example:

  • {"name": "John", "skills" : ["PostgreSQL", "API"]} is the original JSON object.
  • {skills,1} is a path that specifies the skills key, which is an array, and 1 specifies the second element of the array.
  • "Web Dev" is the new value to update.

The jsonb_set() function sets the second element of the skills array to "Web Dev".

6) Using the PostgreSQL jsonb_set() function with table data

We’ll show you how to use the jsonb_set() function to insert a new value into a JSON document and update it back to a table.

First, create a new table called employee_skills:

CREATE TABLE employee_skills(
id INT PRIMARY KEY,
data JSONB
);

Second, insert rows into the employee_skills table:

INSERT INTO employee_skills(id, data)
VALUES
(1, '{"name": "John", "skills" : ["PostgreSQL", "API"]}'),
(2, '{"name": "Jane", "skills" : ["SQL","Java"]}')
RETURNING *;

Output:

 id |                       data
----+---------------------------------------------------
1 | {"name": "John", "skills": ["PostgreSQL", "API"]}
2 | {"name": "Jane", "skills": ["SQL", "Java"]}
(2 rows)

Third, replace the first skill in the skills array of the employee id 1 with the new skill "Web Dev":

UPDATE 
  employee_skills
SET 
  data = jsonb_set(
    data, '{skills,0}', '"Web Dev"'
  ) 
WHERE 
  id = 1 
RETURNING *;Code language: SQL (Structured Query Language) (sql)

Output:

 id |                      data
----+------------------------------------------------
1 | {"name": "John", "skills": ["Web Dev", "API"]}
(1 row)

Summary

  • Use the jsonb_set() function to update a JSON document of the type JSONB.

mirror=> SELECT id, name, properties
FROM products;
id | name | properties
—-+———————–+—————————————————
1 | Ink Fusion T-Shirt | {“size”: [“S”, “M”, “L”, “XL”], “color”: “white”}
2 | ThreadVerse T-Shirt | {“size”: [“S”, “M”, “L”, “XL”], “color”: “black”}
3 | Design Dynamo T-Shirt | {“size”: [“S”, “M”, “L”, “XL”], “color”: “blue”}
(3 rows)

mirror=> UPDATE products SET properties = jsonb_set(properties, ‘{color}’, ‘”purple”‘) WHERE id = 3 RETURNING *;
id | name | properties
—-+———————–+—————————————————-
3 | Design Dynamo T-Shirt | {“size”: [“S”, “M”, “L”, “XL”], “color”: “purple”}
(1 row)

UPDATE 1

How to query a JSON column in PostgreSQL

source
Querying JSON in PostgreSQL

PostgreSQL is a powerful relational database management system. One of its standout features is its ability to handle unstructured data by allowing you to store it in a JSON column. This means you can enjoy the benefits of a structured relational database while leveraging the flexibility of JSON for certain data types. Below are some common ways to

Using PostgreSQL JSON to query a column

Retrieving a Specific JSON Key as Text

If you have a table named **events and you want to retrieve the value associated with the key name from the JSON column params, you can use the following query:

SELECT params->>'name' FROM events;

This will return the value of params.name as text from the events table.

Filtering rows based on a specific JSON key value

If you want to find all events with a specific name, for instance, ‘Click Button’, you can use:

SELECT * FROM events WHERE params->>'name' = 'Click Button';

This will return all rows from the events table where the name key in the params JSON column has the value ‘Click Button’.

Accessing an element from a JSON array

If your JSON column contains arrays and you want to retrieve the first element (index 0) of the array associated with the key ids from the params column, you can use:

SELECT params->'ids'->0 FROM events;

This will return the first element of the ids array from the params column in the events table.

Filtering rows based on a nested JSON key

Sometimes, your JSON might have nested structures. For instance, if you have a table named users with a JSON column preferences and you want to find users where the nested key beta is set to true, you can use:

SELECT preferences->'beta' FROM users WHERE (preferences->>'beta')::boolean IS TRUE;

This query first type casts the value of preferences.beta from JSON to boolean and then filters the rows where it’s true.

Querying a JSONb column in PostgreSQL

In PostgreSQL, jsonb is a data type used to store JSON (JavaScript Object Notation) data in a more efficient and optimized binary format. It is an extension of the json data type. jsonb stands for JSON binary. It provides several advantages over the standard json type, especially when it comes to querying and indexing JSON data.

You can queryjsonb columns using various JSON functions and operators provided by PostgreSQL. Some commonly used functions and operators include:

  • ->: Extracts a JSON element by key or array index.
  • ->>: Extracts a JSON element as text.
  • #>: Extracts a JSON sub-object at a specified path.
  • #>>: Extracts a JSON sub-object as text.
  • @>: Checks if a JSON document contains another JSON document.
  • <@: Checks if a JSON document is contained within another JSON document.
  • jsonb_array_elements(): Expands a JSON array into a set of rows.
  • jsonb_each(): Expands a JSON object into key-value pairs.

Suppose you have a table called employees with a jsonb column namedemployee_data.

-- Extract the employee's name
SELECT employee_data->>'name' AS employee_name
FROM employees;

-- Check if the employee has a skill in "Sales"
SELECT *
FROM employees
WHERE employee_data->'skills' @> '["Sales"]';

-- Find employees in the "Marketing" department
SELECT *
FROM employees
WHERE employee_data->>'department' = 'Marketing';

jsonb is a powerful tool for working with JSON data in PostgreSQL, especially when you need to query and manipulate complex JSON structures.

Fixing issues in querying JSON columns

Troubleshooting JSON column querying in PostgreSQL can involve identifying and addressing issues related to data integrity, query performance, and syntax errors. Troubleshooting JSON column querying in PostgreSQL often requires a combination of SQL knowledge, understanding of JSON data structures, and careful query optimization. By addressing these common issues and best practices, you can improve your ability to work effectively with JSON data in PostgreSQL. Here are some common troubleshooting steps and potential issues to watch out for when querying JSON columns.

Nested JSON structures

As shown in the tutorial, querying JSON columns is fairly straightforward. However, it can get a bit difficult to query nested JSON structures. It is important to use appropriate JSON operators and functions to navigate and query nested JSON objects and arrays. Functions like->->>#>, and#>> can help access nested elements. The-> operator returns a JSON object, and->> returns the value as text. By chaining these operators, you can navigate through nested JSON structures to retrieve the desired information.

Incorrect JSON path

While it seems obvious, more often than you would want, specifying the wrong JSON path in your queries results in incorrect output or failed queries. Examples of such error messages are – cannot extract elements from a scalar or JSON path not found. Double-check the JSON path you’re using in your queries, especially when dealing with nested structures. Use tools like JSON viewers to visualize the JSON structure.

Error handling

Data quality is an industry wide problem. While we fight this issue daily, lack of error handling in queries can cause errors. Poor quality data results in random missing keys. A single missing key in a large query can disrupt query execution and raise an error. To ensure your queries don’t fail entirely for a few missing keys, implement error handling in your queries. To handle such situations more gracefully, you can use the COALESCE function or conditional logic to provide a default value when a JSON key is missing. Instead of complete failure, the query will then return "Uncategorized" and you still get the values for which keys are present.

Database version

Some JSON functions and operators may not be available in older PostgreSQL versions.

Ensure that you’re using a PostgreSQL version that supports the JSON functionality you need. Consider upgrading, if necessary.

Performance bottlenecks

The reason for slow query performance can be inefficient JSON queries. To resolve this, profile your queries using tools like EXPLAIN to identify potential bottlenecks. Consider optimizing queries by creating appropriate indexes, rewriting queries, or denormalizing data where necessary.

Manage unstructured data like a pro

PostgreSQL’s ability to seamlessly integrate structured relational data with the flexibility of unstructured JSON data offers developers a unique advantage. By understanding how to query JSON columns effectively, one can harness the full potential of Postgres, making it easier to manage, retrieve, and analyze diverse datasets. Whether you’re accessing specific JSON keys, filtering based on specific values, or diving into nested structures, PostgreSQL provides the tools to do so with precision and efficiency. As data continues to evolve and become more complex, mastering these techniques will be invaluable for any database professional.

Going beyond JSON querying

Not that you have queried JSON columns successfully, it is time to use the results to gain further insights. Filtering this data is one of the ways to refine the results of JSON querying. Take a look at the tutorial How to Use Filter to Have Multiple Counts in PostgreSQL.

FAQs

1. How do I extract a specific JSON key from a JSON column?

You can use the -> or->>operators to extract JSON keys.` `-> returns the value as JSON, while->> returns the value as text.

2. Can I query nested JSON structures in PostgreSQL?

Yes, PostgreSQL allows you to query and extract data from nested JSON structures using nested -> or ->> operators.

3. How can I filter rows based on JSON data criteria?

Use the WHERE clause to filter rows based on JSON data criteria. For example,WHERE json_column->>’key’ = ‘value’.

4. What’s the difference between the json and jsonb data types for querying JSON in PostgreSQL?

jsonb is a binary JSON data type optimized for querying and indexing, while json is a plain text JSON data type. jsonb is recommended for querying JSON data.

5. How do I handle missing JSON keys or errors when querying JSON columns?

You can use theCOALESCE function or conditional logic to provide default values or handle missing keys when querying JSON columns.

How to Query Deeply Nested JSON Data in PSQL

source
First let’s start by creating a table, and specifying that we need a PSQL database. And right away let’s add some data we can work with later.

CREATE TABLE jsonb_exp (
  id serial primary key,
  data jsonb
);

INSERT INTO jsonb_exp (data) VALUES 
  ('{"data":{"1":{"items":[{"name":"John"},{"name":"Jack"}]},"2":{"items":[{"name":"John Dorian"},{"name":"Jack Sparrow"}]},"3":{"items":[{"name":"Tom Cruise"},{"name":"somerandomtext"}]},"5":{"items":[{"name":"Tom Cruise"}]}},"property":"Some string"}'),
  ('{"data":{"4":{"items":[{"name":"Maria"},{"name":"Jack"}]},"6":{"items":[{"name":"Jack Jackson"},{"name":"Thomas ---"}]},"7":{"items":[{"name":"-"},{"name":"somerandomtext"}]},"15":{"items":[{"name":"hello"}]}},"property":"Some string"}'),
  ('{"a": "b", "c": [1, 2, 3]}'

Usually, you wouldn’t have a different structure inside your JSON across records, but it’s just for the tests. We can now select and see what we’ve created in a table format.

SELECT * from jsonb_exp;

First, let’s get the basics out of the way.

-> returns jsonb
->> returns text

Not sure, what type you’re working with, you can use pg_typeof.

SELECT data->>'data' as data, pg_typeof(data->'data'), pg_typeof(data->>'data') from jsonb_exp where data->>'data' IS NOT NULL;

Querying from deeply nested JSONB

In this scenario, as you can see in the example above: We have an object, it has a defined structure, but some of the keys are like array indices. Not a common thing, maybe. But it happens.

To the rescue comes the following method:

jsonb_path_query()

It’s in the documentation, but it wasn’t clear to me right away how it works. For the first argument, you pass the object, next you can pass the query. Let’s say we need to query particular items from the object, from ‘items’ property, and only those that contain ‘Jack’.

SELECT id, data, item FROM jsonb_exp t, jsonb_path_query(data->'data', '$.*.items[*]') as item
 WHERE data->'data' IS NOT NULL AND item->>'name'='Jack';

Unpacking. Let’s first look at the jsonb_path_query query, everything else is just the support around it to demonstrate what we did.

Reading ‘$..items[]’, here is how you can read it. $ – the object itself, it can contain a key we don’t know – mark it with an asterisk. Inside of that, we want to look for property items, which is an array, and using [*] we say, that we want every item inside that array.

The comma in the query is just an implicit cross-join, not super relevant to JSONB methods.

Remove the ‘WHERE’ Clause and filter right away

Here is the code, you can add conditions inside the query.

SELECT id, data, jsonb_path_query(data->'data', '$.*.items[*] ? (@.name == $name)', '{"name": "John"}') FROM jsonb_exp
 WHERE data->'data' IS NOT NULL;

As you can see here, you can even add the third parameter where you can keep the variables to be used in the query. It’s just for demo purposes here, you can absolutely remove it and compare it with the string without using additional variables.

Notice here, you use double equal signs and not one as you would outside of this query.

You also can’t just write LIKE as your comparison operator. But you can use like_regex. Let’s say we want to find every item, where the name starts with ‘John’.

SELECT id, data, jsonb_path_query(data->'data', '$.*.items[*] ? (@.name like_regex "^John")') FROM jsonb_exp
 WHERE data->'data' IS NOT NULL;

Dealing with nested JSON objects in PostgreSQL

source
Over the past few weeks I’ve been having to deal with nested JSON columns in Postgres. It was turning into me just throwing queries at it until something stuck. Well, as it would turn out, it’s actually very simple! I just needed to take the time to sit down and learn about a few operators and what they’re actually doing.

With that being said, here is a condensed version of some things I learned that helped me navigate through JSON columns in Postgres.

First, let’s create a very basic table containing a json column

create table sample_table (json_data jsonb);

Now, let’s seed this table with some data about a few different vehicles

insert into sample_table
values 
    ('{ "year": "2011", "make":"Toyota", "model":"Camry", "misc": {"color": "Gray", "doors": "4"}}'),
    ('{ "year": "2017", "make":"Honda", "model":"Civic", "misc": {"color": "White", "doors": "4"}}'),
    ('{ "year": "2017", "make":"Toyota", "model":"Camry", "misc": {"color": "Red", "doors": "2"}}'),
    ('{ "year": "2023", "make":"Honda", "model":"Accord"}'),
    ('{ "year": "1908", "make":"Ford", "model":"T", "misc": {"doors": "2"}}')
;

Now that we have our data, let’s go over some of the basics.

If you’ve read my post about the Bacon Cannon, then you know that I am a firm believer that unique operators deserve unique names. So, as I cover some unique operators, I will be throwing in their nicknames, at no extra charge to you!

The -> Operator

Using the -> (Stabby) operator will return JSON data. So here we can see a few examples of how we would get the json values out of the column:

select json_data -> 'make' from sample_table;
select json_data -> 'model' from sample_table;
select json_data -> 'year' from sample_table;
select json_data -> 'misc' from sample_table;

image

As you can see, ‘misc’ is a nested JSON object. If you want to go another layer deeper into a nested JSON object, we can simply add another ->

select json_data -> 'misc' -> 'color' as color from sample_table;

image

The ->> Operator

Now, as I said before, this isn’t giving us text values. These quotations are an indicator that ->is returning JSON. In order to grab the text value we need to use the ->> (Double Stabby) operator. This is equivalent to adding ::text afterwards, typecasting it to a text.

select json_data -> 'misc' ->> 'color' as color from sample_table;

image

The #> and #>> operators

A simpler way to do nested JSON attributes is to use the #> (Waffle Cone) operator with an array of the path you want to follow.

select json_data #> Array['misc','color'] as color from sample_table;

Just like before, a single > returns JSON, so if we wanted to grab the actual text value, we’ll need to use the #>> (Double Waffle Cone) operator.

select json_data #>> Array['misc','color'] as color from sample_table;

The ?, ?|, and ?& operator

Okay, but what if you want the entire row that matches a certain criteria? We’ll move this into a where clause, and then use the ? operator.

A good way to think of this operator is to replace it with the word “contains”.

For example the next query would read in english as

“select all from sample_table where json_data’s misc doors contains the value 4”

This query will return to us every row where there is a 4 door car

select * from sample_table
where json_data #> Array['misc', 'doors'] ? '4'
;

image

This query will return to us every row containing a Toyota or a Honda

select * from sample_table
where json_data -> 'make' ?| Array['Toyota','Honda']
;

image

This query will return to us every row containing both color and doors keys inside of misc.

select * from sample_table
where json_data -> 'misc' ?& Array['color', 'doors']
;

image

(Notice this doesn’t contain the row that has just {“misc”: {“doors”: “2”}}

The @> and <@ operators

If you are looking for a specific JSON object inside of another, you want to use the @> *(Penguin)*operator.

For example, if were looking specifically for {"key": "value"}:

select * from sample_table
where json_data @> '{"make": "Toyota"}'
;

As you might have guessed, we can look for nested JSON objects

This will return to us every row containing red, 2-door vehicles”

select * from sample_table
where json_data @> '{"misc": {"color":"Red", "doors":"2"}}'
;

There is also the <@ (Ice Cream Cone or Reverse Penguin) operator that behaves the same, only in reverse.

select * from sample_table
where '{"misc": {"color":"Red", "doors":"2"}}' <@ json_data
;

In conclusion

I hope this helps clear up any confusion you might have had about querying nested JSON objects in Postgres.

If you want to learn more about these operators or if you want to see what else you can do, here is a link to the docs for JSON functions in Postgres.

Storing and Using JSON Within PostgreSQL 

https://www.percona.com/blog/storing-and-using-json-within-postgresql-part-one/
https://www.percona.com/blog/storing-and-using-json-within-postgresql-part-two

Recap & What’s Next

A few quick takeaways:

Recapping part 1:

  • Using JSONB is probably going to be your best option in most use cases.
  • Be very careful of type conversions and making assumptions on the data within your JSON/JSONB columns.  You may get errors or odd results.
  • Use the available indexes, generated columns, and expression indexes to gain substantial performance benefits.

What we learned in part 2:

  • The more straightforward the JSON, the easier it will be to pull out and interact with the data you need.
  • Nested JSON data can be pulled out in a few different ways. jsonb_to_rescordset is the easiest way I found to pull out the data I need.  However, using this function or others is very susceptible to the data structure in your document (data types matter!).
  • JSON data whose format changes ( elements added or removed ) may make using some functions difficult, if not impossible, to use.
  • JSON within a well-built, designed application and database can be wonderful and offer many great benefits.  JSON just dumped into databases won’t scale.  Database design still matters.

Now that we have covered the basics and a few nuances of JSON with PostgreSQL, next up, it is time to look at the same functionality and data within MongoDB before finally comparing the three databases. Stay tuned!