Technical tutorial for using Jinja2 within the Certainly Platform

Jinja2 is one of the most-used template engines for Python. Its syntax is heavily inspired by Django and Python. It has full Unicode support, optional automatic HTML escaping, and more. In a Certainly chatbot, Jinja can be used for processing a Webhook's response and request, assigning values of Custom Variables, and checking conditions in Module connections.

In our case, data is always JSON, gets processed by Jinja code, and produces a text result. The following picture illustrates the main idea behind the Jinja template engine. 

In this article, we describe the elements of Jinja2 that are relevant for use within the Certainly Platform, as noted below. Template hierarchy, blocks, custom filters, and other aspects not relevant to our chatbot platform are left out of scope.

Below, we'll cover:

• Jinja basics
• Expressions
• Execution flow control 
• Standard filters (pipe | operator)
• Certainly custom filters
• Tests (is operator)
• Global functions (range, cycler, joiner, lipsum)
• Useful Jinja code snippets
• Troubleshooting Jinja code
• Best practices

Jinja basics

In this section, we'll address the following:

• Jinja delimiters (tags)
• Data types
• none, null, defined, undefined
• Jinja operators overview (is, in, pipe etc)
• Whitespace control

Jinja delimiters (tags)

Jinja tags are used to identify Jinja code. All text outside the tags is given as output without change.

Variables

Similar to Python, Jinja variables are case-sensitive, so MyVar and myVar are two different variables.
A variable name can contain alpha-numerical, underscore "_", and dash "-" characters but not special characters like [, ], {, }, \, ", or '.

To access the attribute of a variable (dict) you can use a dot . or “subscript” syntax [].

The following lines do the same thing:

{{ foo.bar }}
{{ foo['bar'] }} {# Useful if 'bar' is reserved word and can't be accessed via dot (.) #}

It’s important to know that the outer double-curly braces are not part of the variable, but the print statement. If you access variables inside tags, don’t put the braces around them.

If a variable or attribute does not exist, you will get back an undefined value. The default behavior is to evaluate to an empty string if printed or iterated over, and to fail for every other operation.

Data types

Hello
"World"

To determine the variable type, you can use a number of built-in "is" tests. While obvious for types like integers and booleans, it becomes trickier for strings and lists because the string is basically a list of characters that can be accessed by index.

The following table shows how to identify each Jinja data type. For example {{ myString is string }} will render to True.

null, none, undefined, defined

It may be hard at first to understand the difference between these four words because they all mean the absence of value. Not to worry - we'll explain each in greater depth here!

The table here demonstrates the difference between these keywords:

Jinja operators overview

Whitespace control

A minus sign "-" in the opening Jinja tag removes spaces and newlines before the tag. A minus sign "-" in the closing tag removes spaces and newlines after the tag.

The following example demonstrates how the whitespace control works. The dot "·" here represents the space character. 

··{%- if true -%}···Text····{%- endif %}

Text

··{% if true -%}···Text····{%- endif %} 

··Text

··{% if true %}···Text····{%- endif %}

·····Text

··{% if true %}···Text····{% endif %}

·····Text····

Expressions

Here, we'll share information on:

• Mathematical expressions
• Comparison expressions (<. ==, >)
• Logic (boolean) expressions (and, or, not)
• Logical expressions with non-boolean values

Mathematical expressions

Note that + and * operators can be used with strings.

Comparison expressions

Jinja inherits the comparison operators from Python. 

• == - true if both operands are equal
• != - true if both operands are not equal
• > - true if the left operand is greater than the right operand
• >= - true if the left operand is greater than or equal to the right operand
• < - true if the left operand is lower than the right operand
• <= - true if the left operand is lower than or equal to the right operand

Comparisons can be chained arbitrarily. For example, x < y < z is equivalent to x < y and y < z, except that y is evaluated only once. Note that in both cases, z is not evaluated at all when x < y is found to be false. More information can be found in the Python documentation for comparison operations.

Logic (boolean) expressions

Logic operators are inherited from Python, the same as comparisons. In Python, the left operand is always evaluated before the right operand. 
Python uses short circuiting when evaluating expressions involving the and or or operators. When using those operators, Python does not evaluate the second operand unless it is necessary to resolve the result. That allows statements such as if (s != None) and (len(s) < 10): ... to work reliably.

• or - true if one of the operands is true. If the left operand is true, then stops and returns true. If the left operand is false, then the right operand is checked.
• and - true if both operands are true. If the left operand is false, then stops and returns false. If the left operand is true, then the right operand is checked.
• not - negates a statement. Returns true if the right operand is false.

Logical expressions with non-boolean values

The same as Python, Jinja allows non-boolean values in logical expressions. For example, to output the value of variable myVar or a default value if it is empty, you can use:

{{ myVar or "default value" }}

In this example, the or operator returns myVar if it has a truthy value, or returns a "default value" if not. This is useful for variable output or assignments that need fallback values.

Non-boolean values are considered true (also known as truthy), or false (also known as falsy) based on their value. Basically, all empty values are considered false, and all other values are considered true.

Here is the list of most important values that are falsy in a boolean context:

• undefined
• none
• 0 - integer and float zero
• "" - an empty string
• [] - an empty list
• {} - an empty dict
• () - an empty tuple

More information can be found in the Python documentation for truth value testing.

To better understand how it works, we can rewrite the logical expression with conditional expressions.

The following example shows one of the practical uses of the and operator. To remove an element from the list, we need to make sure that list is not empty.

myList and myList.pop()

Execution flow control

Below, we'll explore:

• if statement
• Inline if expression
• for loops (conditional. over dictionary, sorted, items())

• Accessing variables across the scopes (inside loop)

if statement

The if statement in Jinja is comparable with the Python if statement.

{% if 5 <= hour < 12 %} 
Good Morning! 
{% elif 12 <= hour < 21 %} 
Good Afternoon! 
{% else %}
Good night!
{% endif %}

Inline if expression

This is how to write the if condition in one line. For print Jinja tag the syntax looks as follows:

{{ myVar if myVar else "default value" }}

In this example, the value of myVar is printed if it is not empty. Otherwise, "default value" is returned. The else part is optional, but it is recommended to specify a default value to handle exceptions. If not provided, the else block implicitly evaluates into an undefined object.

Below, we show how to set myVar to 0 when it is <0 and leave it untouched otherwise.

{% set myVar = 0 if myVar < 0 %}{# WRONG! myVar is set to undefind when >= 0 #}
{% set myVar = 0 if myVar < 0 else myVar %}{# RIGHT! myVar is untouched when >= 0 #}

for loops

In this section, we'll address:

• else clause in for loop
• Looping with a condition and special variable loop
• Looping over the dictionary

else clause in for loop

Jinja has a special clause else that can be used in loops. The code in this block is executed when no iteration took place in for loop. That can happen because the sequence was empty or the filtering removed all the items from the sequence.

{% for item in [1, "two", {"name":"three"}] %} 
  {{ item }} 
{% else %} 
  The list is empty 
{% endfor %}

Looping with the condition and special variable loop

You can filter the sequence during iteration by using an inline if expression. Inside of a for-loop block, you can access a special loop variable like loop.index for the number of the current iteration, loop.first for detecting the first iteration, and so on.

The following example will show all elements of the list that are greater than 1 and will detect iteration over the list's first and last elements:

{% for item in [1,2,3,4,5] if item > 1 -%} 
{{ item }} is
{%- if loop.first %} First item 
{%  elif loop.last %} Last item 
{%  else %} Middle item 
{%  endif %} 
{%- endfor %}

This is the result:

2 is First item 
3 is Middle item 
4 is Middle item 
5 is Last item 

Looping over the dictionary 

In all the following three examples, we will use this foodDict dictionary.

"foodDict": {
"carrot":{"cost": 1, "category": "vegetable"},
"banana": {"cost": 3, "category": "fruit"},
"apple": {"cost": 2.5, "category": "fruit"},
"tomato": {"cost": 2, "category": "vegetable"}
}

Getting keys only

While looping through dict, the item will have the only key of the key-value pair.

{% for item in foodDict -%}
{{ item }}
{% endfor %}

In the result, only keys are printed:

carrot
banana
apple
tomato

Getting keys and values by using items() method

To get both keys and values, we need to use the items() method.

{% for key, val in foodDict.items() -%}
The {{ key }}'s cost is {{ val.cost }} and it is a {{ val.category }}
{% endfor %}

Result:

The carrot's cost is 1 and it is a vegetable
The banana's cost is 3 and it is a fruit
The apple's cost is 2.5 and it is a fruit
The tomato's cost is 2 and it is a vegetable

Iterating through the sorted dictionary

Usually, you need to have the output sorted, which you can do with dictsort filter. By default, it sorts by key, but you can change it to values. You can also change the direction of the sort and adjust case sensitivity. The values should be either numbers or strings, not dictionaries.

By default, the dictsort filter sorts by key, case insensitive, and ascending.

{% for key, val in foodDict | dictsort -%}
The {{ key }}'s value is {{ val }}
{% endfor %}

In the result, the output is sorted by key:

The apple's value is {'cost': 2.5, 'category': 'fruit'}
The banana's value is {'cost': 3, 'category': 'fruit'}
The carrot's value is {'cost': 1, 'category': 'vegetable'}
The tomato's value is {'cost': 2, 'category': 'vegetable'}

You can also combine sorting and filtering. In the following example, we output only vegetables, descending by key.

{% for key, val in (foodDict | dictsort(reverse=true)) if val.category=="vegetable" -%}
The {{ key }}'s cost is {{ val.cost }}
{% endfor %}

In this result, the output is sorted by key and filtered by vegetable.

The tomato's cost is 2  
The carrot's cost is 1  

Accessing variables across the scopes (inside loop)

Jinja has very strict variable scoping. If you have an assignment in a loop, it won't work because you don't have access to outer variables inside the loop scope. The solution is to use a special loop variable or use a namespace object to allow changes across the scopes (v2.10+).

The following example shows that the assignment of the found2 inside for-loop fails while the assignment of ns.found succeeds.

{% set ns = namespace(found=false) -%} {# Creates ns dict with attribute found set to false #}
{% set found2 = false -%} 
{% for item in [1,2,3,4] -%} 
{%- if item == 2 %}{% set ns.found = true %}{% set found2 = true %}{% endif -%} 
    {{ item }},  
{%- endfor %} 
ns.found = {{ ns.found }} 
found2   = {{ found2 }}

Here is the result:

1,2,3,4, 
ns.found = True 
found2   = False 

Standard filters (pipe | operator)

We've seen the use of filters in previous examples. Filters are essentially the functions that are called with a pipe operator | and can take arguments. Multiple filters can be chained. In this case, the output of one filter is applied to the next one.

We highlight some of the most useful filters here:

Below, we'll cover:

• Select or reject elements of a sequence
• safe
• tojson

Select or reject elements of a sequence

The following filters are based on the use of Jinja tests described later in this article. These filters apply a test to each object in a sequence, and select or reject the objects depending on the test's result. If no test is specified, each object will be evaluated as a boolean.

Consider {% set myList = ["between", 10, "and", 20] %} 

You may be wondering, what if the element of the sequence is a dictionary? Then we need to use selectattr/rejectattr filters to check the attribute (field) of the dictionary.

In the following example, we have a list of dictionaries (myLOD) where we select or reject objects based on the value of the "city" attribute.

{% set myLOD = [{"id":1, "city":"Madrid"}, {"id":2, "city":"Copenhagen"}, {"id":3, "city":None}] %}
{{ myLOD | selectattr("city", "none") | list }}
{{ myLOD | rejectattr("city", "none") | list }}

The result is:

[{'id': 3, 'city': None}]
[{'id': 1, 'city': 'Madrid'}, {'id': 2, 'city': 'Copenhagen'}]

safe

Turns off automatic HTML escaping. In the Certainly Platform, all the bot's messages are in plain text, so no HTML escaping is needed. Consider using the safe filter if your data contains apostrophes like in the word "don't".

The following is an example of using escape and safe filters:

{{ "Don't" | escape }} 
{{ "Don't" | safe }} 

Don't
Don't 

In this table are the most important characters that get replaced by HTML escaping (Jinja filter escape):

tojson

Serializes input to JSON text. It escapes special characters in strings according to JSON specifications.

Below is an example of serializing strings and a dictionary. Note that the strings automatically get surrounded by double quotes, and the keyword None in Jinja changes to null in JSON.

{{ "Don't" | tojson }} 
{{ 'I "knew" that' | tojson }} 
{{ {"myKey": None} | tojson }}

"Don\u0027t"
"I \"knew\" that" 
{"myKey": null} 

A JSON string must be double-quoted according to the specs. It cannot be single-quoted.

Here, you can see the most important characters that get escaped by Unicode escape sequences:

Certainly custom filters

Below are descriptions of custom filters available on the Certainly Platform. Note that you will not be able to use these filters in online Jinja parsers, as they are only available within the Certainly Platform.

• date_time()
• time_to_unix()
• loadjson()
• tohash()

date_time()

Returns the current date and time as a string if the input contains a formatting string or Unix Time in milliseconds if the input contains the reserved word "unix".

{{ "%d-%m-%Y %H:%M" | date_time(timezone='Europe/Copenhagen', delta_days=0) }}
{{ "%w"| date_time }}

• "%d-%m-%Y %H:%M" and "%w" are the format strings, where:
  • %d  = Day of the month as a decimal number [01, 31]
  • %m = Month as a decimal number [01, 12]
  • %Y = Year with century as a decimal number
  • %H = Hour (24-hour clock) as a decimal number [00, 23]
  • %M = Minute as a decimal number [00, 59]
  • %w = Weekday as a decimal number [0=Sunday, 6=Saturday]
    
    A full list of date format values can be found in the Python documentation for time.strftime.

• The arguments of the filter are:
  • timezone - The time zone you want to use. The default is 'Europe/Copenhagen'. Example: 'Europe/Madrid'. You can also explore a full list of time zones.
  • delta_days - Number of days to add to or subtract from the current date. It also accepts a float number if you need a time shift of less than a day. For example, -0.041 6 will result in an hour before (1/24 = 0.0416). 

Below are a few examples:

One example of a use for the date_time() filter is checking a store's opening hours. Note that the weekday is returned as a decimal number, where 0=Sunday, 1=Monday, and so on.

{%- set time = "%H%M"| date_time("Europe/Amsterdam")|int -%}
{%- set day = "%w"| date_time("Europe/Amsterdam")|int -%}
{%- if ((900<=time<=1800 and 1<=day<=4) or (900<=time<=1600 and day==5)) -%}
Open
{%- else -%}
Closed
{%- endif -%}

time_to_unix()

Converts date-time string to a Unix Time in milliseconds (standard Unix Time is a number of seconds since 1970-01-01 00:00:00 UTC). It is helpful for date manipulation like calculating the difference between two dates (order or ticket age).

The syntax looks like:

{{"2022-09-08T09:44:24" | time_to_unix(datetime_format="%Y-%m-%dT%H:%M:%S", timezone="Europe/Copenhagen") }}

Where:

• "2022-09-08T09:44:24" - Date-time as a string that we want to convert to Unix Time
• datetime_format="%Y-%m-%dT%H:%M:%S" - The format of the date-time string. See the list of possible values in date_time() filter description above.
• timezone="Europe/Copenhagen" - Timezone, defaults to "Europe/Copenhagen"

Below are a few examples:

{{ "2022-09-01" | time_to_unix('%Y-%m-%d') }}

{{"01/11/2021T05" | time_to_unix('%d/%m/%YT%H','Europe/Madrid'}}

{{"2022-09-01T10:50:32"|time_to_unix('%Y-%m-%dT%H:%M:%S','Europe/Madrid')}}

To calculate the age of some event that happend on start_date you can use the following Jinja code

{{ (("unix"|date_time - start_date|time_to_unix("%Y-%m-%dT%H:%M:%S"))/1000/60/60/24)|int }}

That code will return number of full days since start_date until now. The start_date should be in "%Y-%m-%dT%H:%M:%S" format. The timezone of the start_date and current date-time are defaults to Copenhagen.

loadjson()

Deserializes string to Python variable of the appropriate type. JSON objects become Python dictionaries, arrays become lists, and null becomes None in Python. Strings, int, and boolean are the same. 

The following example shows how to access properties of different JSON objects:

{% set myJSON = "{\"myKey\":123}" %}
{{ (myJSON | loadjson).myKey }}

{% set myList = "[1,null,\"three\"]" | loadjson %}
{{ myList[0] }}
{{ myList[1] }}
{{ myList[2] }}

123 


1 
None 
three

tohash()

Applies one of the hash algorithms to the input string. By default, it uses the sha256 algorithm to hash a string. If a different algorithm is needed, it must be passed as an argument. Here is the list of currently supported algorithms:

'sha3_512', 'blake2b', 'sha3_256', 'sha3_224', 'shake_128', 'sha384', 'blake2s', 'md5', 'sha256', 'sha1', 'sha512', 'sha224', 'sha3_384', 'shake_256'

In the example below, we apply different hash algorithms to the same string. Note that the non-existent algorithm "md57" results in the error message.

sha256: {{ "Certainly is the best" | tohash }}
md5:    {{ "Certainly is the best" | tohash("md5") }}
error:  {{ "Certainly is the best" | tohash("md57") }}

Result:

sha256: 9856c460829f711cce1d9ea335dcfeb65c9e170b32f600c1b16be9a1a03e8dd0 
md5:    b104732ffa676ac4bf11b37ef876b82a
error:  Hash function currently not supported

Tests (is operator)

The test operator tests left operand against the test provided as right operand. The result is boolean True or False.

Here, we highlight the most useful filters. Consider {% set myVar = none %}:

Global functions

The functions noted here are available in the global scope by default. In this section, we'll provide information on the following functions:

• range([start, ]stop[, step])
• cycler(\*items)
• joiner(sep=',')
• lipsum(n=5, html=True, min=20, max=100)

range([start, ]stop[, step])

Returns a list containing an arithmetic progression of integers: range(i, j) returns [i, i+1, i+2, ..., j-1]. By default, the progression starts from 0. When the step is given, it specifies the increment (or decrement). For example, range(4) and range(0, 4, 1) return [0, 1, 2, 3]. The end point is omitted. These are exactly the valid indices for a list of four elements.

cycler(items)

This is a helper function that cycles through items, then restarts once the end is reached.

joiner(sep=',')

This is a helper function that will return sep string every time it’s called except the first time, in which case it returns an empty string.

Example of use of range, joiner, and cycler functions.

{% set myJoiner = joiner("-") -%}
{% for e in range(4) -%}
{{ myJoiner() }}{{ e -}}
{% endfor %}

0-1-2-3 

{% set myJoiner = joiner("-") -%}
{% set myCycler = cycler("a", "b") -%}
{% for e in range(4) -%}
{{ myJoiner() }}{{ e }}{{ myCycler.next() -}}
{% endfor %}

0a-1b-2a-3b

lipsum(n=5, html=True, min=20, max=100)

Generates some lorem ipsum text. By default, five paragraphs of HTML are generated, with each paragraph between 20 and 100 words. If html is False, regular text is returned. This is useful to generate simple content for layout testing.

Useful Jinja code snippets 

In this section, you can find information on:

• String manipulation
• Number manipulation
• Date manipulation

String manipulation

It is also possible to use the standard Python function for string manipulation. 

Below is a table of some of the most useful string functions. Consider {% set myStr = "Hello world!" %}:

{% set myStr = "order no is #123" %}
{{ myStr.partition("#")[0] }}
{{ myStr.partition("#")[1] }}
{{ myStr.partition("#")[2] }}

Number manipulation

Date manipulation

There is no special data type for dates. They are usually represented as strings. Presentation in a format that allows string comparison (such as ISO 8601 date representation standard) is preferred. It will allow filters like "sort" to work.

In the example below, we show how to extract the latest element from a list of dictionaries with a timestamp field.

"myList": [ 
{"d":"2021-08-10 07:40:23", "a":"work"}, 
{"d":"2021-08-22 07:40:23", "a":"golf"}, 
{"d":"2021-08-05 07:40:23", "a":"bar"}
]  

This code will sort the array in reverse order (greatest element first) and show the field "a" of the first element:

{{ (myList | sort(reverse=true, attribute="d"))[0].a }} 

For the result, the latest element is "golf":

golf

Troubleshooting Jinja code

Jinja's print tag {{ }} produces an empty string if: the variable inside it is not defined, the index of a list is outside of the defined range, or you're accessing a non-existing key of a dictionary.

However, if you try to access an element of an undefined list or undefined dictionary, you'll get the following error: "Rendering error: 'undefList' is undefined".

In a Python environment or an online Jinja parser, you'll have a clear error message. In the Certainly Platform debugger (tester), you'll see the Jinja code instead of a value, as shown in the following image:
If this occurs, it means your Jinja code has either:

1. A variable that was not defined (assigned a value) previously in the chat flow, or
2. Syntax or other error

If the first case, you just need to ensure you spelled the variable name correctly. Remember that in Jinja variables' names are case-sensitive.

If the variable is supposed to be undefined at that point of the chat flow and your Jinja code does not have any other errors, it is ok to leave it as it is. The Certainly Platform will try to render your Jinja code even if it has undefined variables.

In the following table, we provide examples of errors and fixes for them.

If you want to have something instead of an empty string in cases where a variable is not defined, you can use the following constructions:

{{ undfVar or "Default value" }} for undefined variables

{{ undefList[0] if undefList else "Default value" }} for undefined lists

{{ undefDict.key1 if undefDict else "Default value" }} for undefined dictionaries

Best practices

As a final note, we'd like to share a few best practices with you:

1. Use Jinja live parsers online for debugging, such as the TTL255 Jinja2 parser or the https://cryptic-cliffs-32040.herokuapp.com.
2. Always convert to the proper data type prior to using any filters. For example:
   

   {{ set value = "123" }} 
   {{ value > 100 }} given the value is of "string" type, make sure you convert to integer first 
   {{ (value | int) > 123 }} 

3. Remember that boolean expressions inside {{ }} tag  are rendered as True / False strings. 
   So, you should use

   {{ time < 18 }} 

   instead of

   {% if time < 18 %}True{% else %}False{% endif %}

Have questions about the info provided here? Feel free to contact our Customer Success team.