Keras backends

Keras is a model-level library, offers high-level building blocks that are useful to develop deep learning models. Instead of supporting low-level operations such as tensor products, convolutions, etc. itself, it depends upon the backend engine that is well specialized and optimized tensor manipulation library. It doesn't pick just one library of a tensor to implement Keras tied to that particular library. It handles the situation in a modular way by seamlessly plugging many distinct back-end engines to Keras.

Following are the three available backend implementations, which are as follows;

TensorFlow: This Google-developed framework for symbolic tensor manipulation is open-source.
Theano: It is also an open-source framework for symbolic manipulation of a tensor is developed at Universite de Montreal by LISA Lab.
CNTK: It is developed by Microsoft, which is also an open-source deep-learning toolkit.

Switching from one backend to another

You will probably find the Keras configuration file at:

$HOME/.keras/keras.json

In case you face a problem finding it there, then you may create one!

Note: Especially for the Windows user, you have to replace $HOME with %USERPROFILE%.

Following is the default configuration;

{
    "image_data_format": "channels_last",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "tensorflow"
}

Here, you just have to change the backend field to "theano", "tensorflow" or "cntk", and then Keras will make use of the modified configuration when you will run any Keras code.

Once you define the KERAS_BACKEND environment variable, it will override whatsoever defined inside your config file:

KERAS_BACKEND=tensorflow python -c "from keras import backend"
Using TensorFlow backend.

Possibly you can load many more backends in Keras then "tensorflow", "theano" or "cntk" as it can easily make use of external backends. This can be done by changing keras.json and "backend" setting. Let's suppose you have a Python module named as my_module to be used as an external backend; then, in that case, the keras.json file may undergo some changes, which is as follows;

{
    "image_data_format": "channels_last",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "my_package.my_module"
}

In order to use an external backend, it must be validated and encompass functions like placeholder, variable, and function.

If the external backend is not valid then, it may generate an error which may contain all the missing entries.

keras.json details

Following are the settings contained in the keras.json file:

{
    "image_data_format": "channels_last",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "tensorflow"
}

The settings can be simply modified by editing $HOME/.keras/keras.json.

image_data_format: It can be defined as a string, either of "channels_last" or "channels_first", specifying the convention data format followed by Keras. (It is returned by backend.image_data_format()).
For any two-dimensional data such as an image, the "channels_last" will assume (rows, cols, channels), whereas "channels_first" will assume (channels, rows, cols).
For any three-dimensional data, the "channels_last" will relate to (conv_dim1, conv_dim2, conv_dim3, channels), whereas "channels_first" will relate to (channels, conv_dim1, conv_dim2, conv_dim3).
epsilon: It refers to a float, which is a fuzzy numeric constant utilized for avoiding the division by zero in some operations.
floatx: It indicates a string of "float16", "float32", or "float64". By default, it is float precision.
backend: It refers to a string of "tensorflow", "theano", or "cntk".

Usage of abstract Keras backend for writing new code

With the help of abstract Keras backend API, you can make your written Keras module more compatible with both Theano (th) and TensorFlow (tf). Following is an intro to it;

The backend module can be import thru:

An input placeholder will be instantiated by the code given below, which is equal to tf.placeholder() or th.tensor.matrix(), th.tensor.tensor3(), etc.

inputs = K.placeholder(shape=(2, 4, 5))
# also works:
inputs = K.placeholder(shape=(None, 4, 5))
# also works:
inputs = K.placeholder(ndim=3)

A variable will be instantiated by incorporating the following code, which in return is equal to tf.Variable() or th.shared().

import numpy as np
val = np.random.random((3, 4, 5))
var = K.variable(value=val)

# all-zeros variable:
var = K.zeros(shape=(3, 4, 5))
# all-ones:
var = K.ones(shape=(3, 4, 5))

Most of the tensor operations that you may require will be performed in a similar way as you will do in TensorFlow or Theano are as follows:

 # Initializing Tensors with Random Numbers
b = K.random_uniform_variable(shape=(3, 4), low=0, high=1) # Uniform distribution
c = K.random_normal_variable(shape=(3, 4), mean=0, scale=1) # Gaussian distribution
d = K.random_normal_variable(shape=(3, 4), mean=0, scale=1)

# Tensor Arithmetic
a = b + c * K.abs(d)
c = K.dot(a, K.transpose(b))
a = K.sum(b, axis=1)
a = K.softmax(b)
a = K.concatenate([b, c], axis=-1)
# etc...

Backend functions

backend

The backend function is used to revert back the current backend name.

Returns

It returns a string that relates to the current name of the backing being used.

Example

>>> keras.backend.backend()
'tensorflow'

symbolic

It can be defined as a decorator, which is utilized in TensorFlow 2.0 for entering the Keras graph.

Arguments

func: It refers to a function that is used to decorate.

Returns

It returns a decorated function.

eager

It can be defined as a decorator, which is utilized in TensorFlow 2.0 for exiting the Keras graph.

Arguments

func: It refers to a function that is used to decorate.

Returns

It returns a decorated function.

get_uid

It provides a unique UID that gives a string prefix.

Arguments

prefix: It refers to a string.

Returns

This backend function returns an integer.

Example

>>> keras.backend.get_uid('dense')
1
>>> keras.backend.get_uid('dense')
2

This function is used for setting up the manual variable initialization flags. The flag is indicated as a Boolean that governs for a variable to be initialized or the user has to handle the initialization because they are self-instantiated by default.

Arguments

value: It refers to Python's Boolean value.

epsilon

It is used to return the fuzz factor value, which is being utilized in the numeric expressions.

Returns

It returns a float.

Example

>>> keras.backend.epsilon()
1e-07

reset_uids

It is used to reset the graph identifiers.

epsilon

It outputs a fuzz factor value, which is utilized in the numeric expressions.

Returns

It returns a float value.

Example

>>> tf.keras.backend.epsilon()
1e-07

set_epsilon

It is used to set the fuzz factor value, which is being used in the numeric expressions.

Arguments

e: It can be defined as a float value that represents the epsilon's new value.

Example

>>> from keras import backend as K
>>> K.epsilon()
1e-07
>>> K.set_epsilon(1e-05)
>>> K.epsilon()
1e-05

floatx

It is used to output a string of float type, such as 'float16', 'float32', 'float64'.

Returns

It returns a string of the current default float type.

Example

>>> keras.backend.floatx()
'float32'

set_floatx

It is used to set the default float type value.

Arguments

floatx: It refers to a string of float type, such as 'float16', 'float32', or 'float64'.

Example

>>> from keras import backend as K
>>> K.floatx()
'float32'
>>> K.set_floatx('float16')
>>> K.floatx()
'float16'

Raises

ValueError: Whenever there is an invalid value, then ValueError will be generated.

cast_to_floatx

It is used for casting a Numpy array to the default Keras float type.

Arguments

x: It refers to the Numpy array.

Returns

It returns the same Numpy array that is being casted to its new type.

Example

>>> from keras import backend as K
>>> K.floatx()
'float32'
>>> arr = numpy.array([1.0, 2.0], dtype='float64')
>>> arr.dtype
dtype('float64')
>>> new_arr = K.cast_to_floatx(arr)
>>> new_arr
array([ 1.,  2.], dtype=float32)
>>> new_arr.dtype
dtype('float32')

image_data_format

It is used to returns the default image data format convention.

Returns

It returns a string either of 'channels_first' or 'channels_last'

Example

>>> keras.backend.image_data_format()
'channels_first'

set_image_data_format

This function is used for setting up the data format convention's value.

Arguments

data_format: It can be defined as a string either of 'channels_first' or 'channels_last'.

Example

>>> from keras import backend as K
>>> K.image_data_format()
'channels_first'
>>> K.set_image_data_format('channels_last')
>>> K.image_data_format()
'channels_last'

Raises

ValueError: Whenever there is an invalid data_format value, then it will generate a ValueError.

learning_phase

It outputs the flag of a learning phase, which refers to a bool tensor (0 = test, 1 = train) to be passed as an input to any of the Keras function that utilizes a distinct behavior both at training and testing time.

Returns

It returns a scalar integer tensor or Python integer of the learning phase.

set_learning_phase

It is used to set a fixed value to the learning phase.

Arguments

value: It can be defined as an integer that represents the learning phase value to be either 0 or 1.

Raises

ValueError: It is raised if the value is neither 0 nor 1.

clear_session

It is used for resetting each and every state that is produced by Keras. The global state that is utilized for the implementation of the Functional model-building API as well as to uniquify auto-generated layer names is handled by Keras.

When multiple models are built in a loop, then an increasing amount of memory over a certain time period will be consumed by the global state, which you will wish to clear it.

It is used for destroying the current graph of Keras and creating a new one. It is very useful as it avoids clutter from old models/layers.

Example1: calling clear_session() while creating models in a loop.

for _ in range(100):
  # Without `clear_session()`, each iteration of this loop will
  # slightly increase the size of the global state managed by Keras
  model = tf.keras.Sequential([tf.keras.layers.Dense(10) for _ in range(10)])

for _ in range(100):
  # With `clear_session()` called at the beginning,
  # Keras starts with a blank state at each iteration
  # and memory consumption is constant over time.
  tf.keras.backend.clear_session()
  model = tf.keras.Sequential([tf.keras.layers.Dense(10) for _ in range(10)])

Example2: resetting the layer name generation counter.

>>> import tensorflow as tf
>>> layers = [tf.keras.layers.Dense(10) for _ in range(10)]
>>> new_layer = tf.keras.layers.Dense(10)
>>> print(new_layer.name)
dense_10
>>> tf.keras.backend.set_learning_phase(1)
>>> print(tf.keras.backend.learning_phase())
1
>>> tf.keras.backend.clear_session()
>>> new_layer = tf.keras.layers.Dense(10)
>>> print(new_layer.name)
dense

is_sparse

It is used to return whether a tensor is a sparse tensor.

Arguments

tensor: It refers to an instance of tensor.

Returns

It returns a Boolean.

Example

>>> from keras import backend as K
>>> a = K.placeholder((2, 2), sparse=False)
>>> print(K.is_sparse(a))
False
>>> b = K.placeholder((2, 2), sparse=True)
>>> print(K.is_sparse(b))
True

to_dense

It is used in conversion of a sparse tensor to a dense tensor and returns it.

Arguments

tensor: It refers to an instance of a tensor(potentially sparse).

Returns

It returns a dense tensor.

Example

>>> from keras import backend as K
>>> b = K.placeholder((2, 2), sparse=True)
>>> print(K.is_sparse(b))
True
>>> c = K.to_dense(b)
>>> print(K.is_sparse(c))
False

variable

It helps in instantiating a variable and returning it.

Arguments

value: It can be defined as a numpy array that represents tensor's initial value.
dtype: It refers to the type of a Tensor.
name: For a tensor it indicates a string name.
constraint: It refers to an optional projection function that is implemented on the variable after updating an optimizer.

Returns

It returns an instance of a variable that comprising of a Keras metadata.

Example

>>> from keras import backend as K
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val, dtype='float64', name='example_var')
>>> K.dtype(kvar)
'float64'
>>> print(kvar)
example_var
>>> K.eval(kvar)
array([[ 1.,  2.],
       [ 3.,  4.]])

is_variable

constant

It lead to the creation of a unique tensor.

Arguments

value: It refers to constant value or a list.
dtype: It refers to the type of a Tensor.
name: For a tensor it indicates a string name.
shape: It can be defined as a dimensionality of the resulting tensor, which is an optional.

Returns

It also return a unique Tensor.

is_keras_tensor

It outputs whether x is a Keras tensor or not. A "Keras tensor" is a tensor that was returned by a Keras layer, (Layer class) or by Input.

Arguments

x: It refers to a candidate tensor.

Returns

It returns a Boolean that represents whether the argument is a Keras tensor or not.

Raises

It raises a ValueError if x is not a symbolic tensor.

Example

>>> from keras import backend as K
>>> from keras.layers import Input, Dense
>>> np_var = numpy.array([1, 2])
>>> K.is_keras_tensor(np_var) # A numpy array is not a symbolic tensor.
ValueError
>>> k_var = tf.placeholder('float32', shape=(1,1))
>>> # A variable indirectly created outside of keras is not a Keras tensor.
>>> K.is_keras_tensor(k_var)
False
>>> keras_var = K.variable(np_var)
>>> # A variable created with the keras backend is not a Keras tensor.
>>> K.is_keras_tensor(keras_var)
False
>>> keras_placeholder = K.placeholder(shape=(2, 4, 5))
>>> # A placeholder is not a Keras tensor.
>>> K.is_keras_tensor(keras_placeholder)
False
>>> keras_input = Input([10])
>>> K.is_keras_tensor(keras_input) # An Input is a Keras tensor.
True
>>> keras_layer_output = Dense(10)(keras_input)
>>> # Any Keras layer output is a Keras tensor.
>>> K.is_keras_tensor(keras_layer_output)
True

is_tensor

placeholder

keras.backend.placeholder(shape=None, ndim=None, dtype=None, sparse=False, name=None)

It helps in instantiating a placeholder tensor and returning it.

Arguments

shape: It can be defined as a tuple integer, which may incorporate None entries helps in representing the placeholder's Shape.
ndim: It refers to the number of tensor's axes, which specifies at least one of {shape, ndim}. The shape is used, if both are specified.
dtype: It defines the type of Placeholder.
sparse: It can be defined as a Boolean that represents whether or not the placeholder to have a sparse type.
name: It is an optional argument that defines a string for the placeholder's name.

Returns

It returns an instance of a Tensor by including a Keras metadata.

Example

>>> from keras import backend as K
>>> input_ph = K.placeholder(shape=(2, 4, 5))
>>> input_ph._keras_shape
(2, 4, 5)
>>> input_ph
<tf.Tensor 'Placeholder_4:0' shape=(2, 4, 5) dtype=float32>

is_placeholder

It returns if x is a placeholder or not.

Arguments

x: It can be defined as a candidate placeholder.

Returns

It returns a Boolean.

shape

It outputs the symbolic shape of a tensor or variable.

Arguments

x: It refers to a tensor or variable.

Returns

It returns a tensor of symbolic shape.

Examples

# TensorFlow example
>>> from keras import backend as K
>>> tf_session = K.get_session()
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> inputs = keras.backend.placeholder(shape=(2, 4, 5))
>>> K.shape(kvar)
<tf.Tensor 'Shape_8:0' shape=(2,) dtype=int32>
>>> K.shape(inputs)
<tf.Tensor 'Shape_9:0' shape=(3,) dtype=int32>
# To get integer shape (Instead, you can use K.int_shape(x))
>>> K.shape(kvar).eval(session=tf_session)
array([2, 2], dtype=int32)
>>> K.shape(inputs).eval(session=tf_session)
array([2, 4, 5], dtype=int32)

int_shape

It can be defined as a tuple of int or None entries that outputs the tensor or a variable's shape.

Arguments

x: It refers to either a tensor or variable.

Returns

It either returns a tuple of integers or None entries.

Example

>>> from keras import backend as K
>>> inputs = K.placeholder(shape=(2, 4, 5))
>>> K.int_shape(inputs)
(2, 4, 5)
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> K.int_shape(kvar)
(2, 2)

Numpy implementation

def int_shape(x):
    return x.shape

ndim

It refers to an integer that are returned as number of axes within a tensor.

Arguments

x: It can be either defined as a tensor or variable.

Returns

It outputs the number of axes as an integer value.

Example

>>> from keras import backend as K
>>> inputs = K.placeholder(shape=(2, 4, 5))
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> K.ndim(inputs)
3
>>> K.ndim(kvar)
2

Numpy implementation

def ndim(x):
    return x.ndim

size

It outputs the tensor size.

Arguments

x: It can either be defined as a tensor or variable.
name: It is an optional keyword argument that represents the operation's name.

Returns

It returns the tensor's size.

Example

>>> from keras import backend as K
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> K.size(inputs)
<tf.Tensor: id=9, shape=(), dtype=int32, numpy=4>

dtype

It can be defined as a string, which is returned as a dtype of a Keras tensor or variable.

Arguments

x: It can be either defined as a tensor or variable.

Returns

For x it returns its dtype.

Example

>>> from keras import backend as K
>>> K.dtype(K.placeholder(shape=(2,4,5)))
'float32'
>>> K.dtype(K.placeholder(shape=(2,4,5), dtype='float32'))
'float32'
>>> K.dtype(K.placeholder(shape=(2,4,5), dtype='float64'))
'float64'
# Keras variable
>>> kvar = K.variable(np.array([[1, 2], [3, 4]]))
>>> K.dtype(kvar)
'float32_ref'
>>> kvar = K.variable(np.array([[1, 2], [3, 4]]), dtype='float32')
>>> K.dtype(kvar)
'float32_ref'

Numpy implementation

def dtype(x):
    return x.dtype.name

eval

It helps in evaluating tensor value.

Arguments

x: It can be defined as a tensor.

Returns

It outputs a Numpy array.

Example

>>> from keras import backend as K
>>> kvar = K.variable(np.array([[1, 2], [3, 4]]), dtype='float32')
>>> K.eval(kvar)
array([[ 1.,  2.],
       [ 3.,  4.]], dtype=float32)

Numpy implementation

def eval(x):
    return x

zeros

It helps in instantiation of those variables that are all-zeros followed by returning it.

Arguments

shape: It can be defined as a tuple of integers that represents the returned Keras variable's shape.
dtype: It refers to a string that corresponds to the returned Keras variable's data type.
name: It refers to the string that represent the returned Keras variable's name.

Returns

It returns a variable that includes the Keras metadata, which is filled with 0.0. It should be noted that if it is symbolic n shape, then a variable cannot be returned rather a dynamic-shaped tensor will be returned.

Example

>>> from keras import backend as K
>>> kvar = K.zeros((3,4))
>>> K.eval(kvar)
array([[ 0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.]], dtype=float32)

Numpy implementation

def zeros(shape, dtype=floatx(), name=None):
    return np.zeros(shape, dtype=dtype)

ones

It helps in Instantiation of an all-ones variable followed by returning it.

Arguments

shape: It can be defined as a tuple of integers that represents the returned Keras variable's shape.
dtype: It refers to a string that corresponds to the returned Keras variable's data type.
name: It refers to the string that represent the returned Keras variable's name.

Returns

It returns a Keras variable, which is filled with 0.0. It should be noted that if it is symbolic n shape, then a variable cannot be returned rather a dynamic-shaped tensor will be returned.

Example

>>> from keras import backend as K
>>> kvar = K.ones((3,4))
>>> K.eval(kvar)
array([[ 1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.]], dtype=float32)

Numpy implementation

def ones(shape, dtype=floatx(), name=None):
    return np.ones(shape, dtype=dtype)

eye

It helps in the instantiation of an identity matrix followed by returning it.

Arguments

size: It can be defined either as a tuple defining the number of rows and columns or an integer that represents the number of rows.
dtype: It refers to a string that corresponds to the returned Keras variable's data type.
name: It refers to the string that represent the returned Keras variable's name.

Returns

It outputs a Keras variable that represents an identity matrix.

Example

>>> from keras import backend as K
>>> K.eval(K.eye(3))
array([[ 1.,  0.,  0.],
       [ 0.,  1.,  0.],
       [ 0.,  0.,  1.]], dtype=float32)
>>> K.eval(K.eye((2, 3)))
array([[1., 0., 0.],
       [0., 1., 0.]], dtype=float32)

Numpy implementation

def eye(size, dtype=None, name=None):
    if isinstance(size, (list, tuple)):
        n, m = size
    else:
        n, m = size, size
    return np.eye(n, m, dtype=dtype)

zeros_like

It helps in instantiating the similar shape variable that are all-zeros as another tensor.

Arguments

x: It can be defined either as Keras variable or Keras tensor.
dtype: It refers to a string that corresponds to the returned Keras variable's data type. Here the None relates to the usage of x dtype.
name: It refers to the string that represent the returned Keras variable's name.

Returns

It returns a variable of Keras filled with all zeros that constitutes a shape of x.

Example

>>> from keras import backend as K
>>> kvar = K.variable(np.random.random((2,3)))
>>> kvar_zeros = K.zeros_like(kvar)
>>> K.eval(kvar_zeros)
array([[ 0.,  0.,  0.],
       [ 0.,  0.,  0.]], dtype=float32)

Numpy implementation

def zeros_like(x, dtype=floatx(), name=None):
    return np.zeros_like(x, dtype=dtype)

ones_like

It helps in instantiating the similar shape variable that are all-ones as another tensor.

Arguments

x: It can be defined either as Keras variable or Keras tensor.
dtype: It refers to a string that corresponds to the returned Keras variable's data type. Here the None relates to the usage of x dtype.
name: It refers to the string that represent the returned Keras variable's name.

Returns

It returns a variable of Keras filled with all zeros that constitutes a shape of x.

Example

>>> from keras import backend as K
>>> kvar = K.variable(np.random.random((2,3)))
>>> kvar_ones = K.ones_like(kvar)
>>> K.eval(kvar_ones)
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.]], dtype=float32)

Numpy implementation

def ones_like(x, dtype=floatx(), name=None):
    return np.ones_like(x, dtype=dtype)

identity

It outputs a tensor having a similar content as that of the input tensor.

Arguments

x: It refers to the input tensor.
name: It refers to the string that represent the name of the variable, which has to be created.

Returns

It returns a tensor that has same shape, type as well as content.

random_uniform_variable

keras.backend.random_uniform_variable(shape, low, high, dtype=None, name=None, seed=None)

It put an emphasis on the instantiation a variable that have its values drawn from a uniform distribution.

Arguments

shape: It can be defined as a tuple of integers that represents the returned Keras variable's shape.
low: It indicates to a float value that represents the output interval's lower boundary.
high: It refers to a float value, which represents the output interval's upper boundary.
dtype: It refers to a string that corresponds to the returned Keras variable's data type.
name: It can be defined as a string that relates to the returned Keras variable's name.
seed: It can be defined as an integer that represents a random seed.

Returns

It outputs a Keras variable that has been filled with drawn samples.

Example

# TensorFlow example
>>> kvar = K.random_uniform_variable((2,3), 0, 1)
>>> kvar
<tensorflow.python.ops.variables.Variable object at 0x10ab40b10>
>>> K.eval(kvar)
array([[ 0.10940075,  0.10047495,  0.476143  ],
       [ 0.66137183,  0.00869417,  0.89220798]], dtype=float32)

Numpy implementation

def random_uniform_variable(shape, low, high, dtype=None, name=None, seed=None):
    return (high - low) * np.random.random(shape).astype(dtype) + low

random_normal_variable

keras.backend.random_normal_variable(shape, mean, scale, dtype=None, name=None, seed=None)

It helps in the instantiation of a variable whose values are drawn from a normal distribution.

Arguments

shape: It can be defined as a tuple of integers that represents the returned Keras variable's shape
mean: It refers to a float that represents the mean of the normal distribution.
scale: It refers to a float that represents the normal distribution's standard deviation.
dtype: It can be defined as string that represents the returned Keras variable's dtype.
name: It refers to a String that represents the returned Keras variable's name.
seed: It refer to an integer that represents the random seed.

Returns

It outputs a Keras variable that has been filled with drawn samples.

Example

# TensorFlow example
>>> kvar = K.random_normal_variable((2,3), 0, 1)
>>> kvar
<tensorflow.python.ops.variables.Variable object at 0x10ab12dd0>
>>> K.eval(kvar)
array([[ 1.19591331,  0.68685907, -0.63814116],
       [ 0.92629528,  0.28055015,  1.70484698]], dtype=float32)

Numpy implementation

def random_normal_variable(shape, mean, scale, dtype=None, name=None, seed=None):
    return scale * np.random.randn(*shape).astype(dtype) + mean

count_params

It outputs the constant number of components residing within a Keras variable or tensor.

Arguments

x: It refers to a Keras variable or tensor.

Returns

It results in an integer, which depicts the total number of elements present in x, i.e., the product of the static dimensions of an array.

Example

>>> kvar = K.zeros((2,3))
>>> K.count_params(kvar)
6
>>> K.eval(kvar)
array([[ 0.,  0.,  0.],
       [ 0.,  0.,  0.]], dtype=float32)

Numpy implementation

def count_params(x):
    return x.size

cast

It helps in casting a tensor to a distinct dtype followed by returning it. In case, if you cast a Keras variable then also it will result in a Keras tensor.

Arguments

x: It can be defined as Keras tensor or variable.
dtype: It refers to a string either of 'float16', 'float32', or 'float64'.

Returns

It output a Keras tensor with dtype dtype.

Example

>>> from keras import backend as K
>>> input = K.placeholder((2, 3), dtype='float32')
>>> input
<tf.Tensor 'Placeholder_2:0' shape=(2, 3) dtype=float32>
# It doesn't work in-place as below.
>>> K.cast(input, dtype='float16')
<tf.Tensor 'Cast_1:0' shape=(2, 3) dtype=float16>
>>> input
<tf.Tensor 'Placeholder_2:0' shape=(2, 3) dtype=float32>
# you need to assign it.
>>> input = K.cast(input, dtype='float16')
>>> input
<tf.Tensor 'Cast_2:0' shape=(2, 3) dtype=float16>

update

It helps in updating the value of x to new_x.

Arguments

x: It refers to a variable.
new_x: It can be defined as a tensor having similar shape as that of x.

Returns

It results in the updated x variable

update_add

It adds an increment, which helps to update the value of x.

Arguments

x: It refers to a variable.
increment: It can be defined as a tensor having a similar shape as that of x.

Returns

It returns the updated x variable.

update_sub

It subtracts the decrement so as to update the value of x.

Arguments

x: It can be defined as a variable.
decrement: It refers to a tensor that have a similar shape as that of x.

Returns

It returns the updated x variable.

moving_average_update

For a variable it computes its moving average.

Arguments

x: It refers to a variable.
value: It can be defined as a tensor that have a same shape as that of x.
momentum: It refers to a static average momentum.

Returns

It outputs an operation, which is utilized for updating the variable.

dot

It returns a tensor by either multiplying 2 tensors or variable.

While multiplying an nD tensor to another nD tensor, a Theano behavior is reproduced. (e.g. (2, 3) * (4, 3, 5) -> (2, 4, 5))

Arguments

x: It refers to a tensor or variable.
y: It refers to a tensor or variable.

Returns

It returns a tensor, which is produced after undergoing a dot product between x and y.

Examples

# dot product between tensors
>>> x = K.placeholder(shape=(2, 3))
>>> y = K.placeholder(shape=(3, 4))
>>> xy = K.dot(x, y)
>>> xy
<tf.Tensor 'MatMul_9:0' shape=(2, 4) dtype=float32>
# dot product between tensors
>>> x = K.placeholder(shape=(32, 28, 3))
>>> y = K.placeholder(shape=(3, 4))
>>> xy = K.dot(x, y)
>>> xy
<tf.Tensor 'MatMul_9:0' shape=(32, 28, 4) dtype=float32>
# Theano-like behavior example
>>> x = K.random_uniform_variable(shape=(2, 3), low=0, high=1)
>>> y = K.ones((4, 3, 5))
>>> xy = K.dot(x, y)
>>> K.int_shape(xy)
(2, 4, 5)

Numpy implementation

def dot(x, y):
    return np.dot(x, y)

batch_dot

batch_dot is useful in computing batchwise dot product between x and y, where x and y are data inside batches (i.e. in a shape of (batch_size, :)). It either outputs a tensor or variable that encompass less dimensions than the input. If we reduce the number of dimensions to 1, then we can use expand_dims, which ensure the ndim to be atleast 2.

Arguments

x: It refers to either the Keras tensor or variable that have ndim greater than or equals to 2.
y: It refers to the Keras tensor or variable that has ndim greater than or equals to 2.
axes: It can be defined as an int or tuple(int, int) that puts emphasis on the dimensions of the target to be reduced.

Returns

It returns a tensor that has a shape identical to the concatenation of x's shape and y's shape (). Here the shape of x relates to the less the dimension that was summed over and y signifies less the batch dimension and the dimension that was summed over. However, it is reshaped to (batch_size, 1) if the final rank is 1.

Examples

Assume x = [[1, 2], [3, 4]] and y = [[5, 6], [7, 8]] batch_dot(x, y, axes=1) = [[17], [53]] which is the main diagonal of x.dot(y.T), although we never have to calculate the off-diagonal elements.

Pseudocode:

inner_products = []
for xi, yi in zip(x, y):
    inner_products.append(xi.dot(yi))
result = stack(inner_products)

Shape inference: Let x's shape be (100, 20) and y's shape be (100, 30, 20). If axes is (1, 2), to find the output shape of resultant tensor, loop through each dimension in x's shape and y's shape:

shape[0] : 100 : append to output shape
shape[1] : 20 : do not append to output shape, dimension 1 of x has been summed over. (dot_axes[0] = 1)
shape[0] : 100 : do not append to output shape, always ignore first dimension of y
shape[1] : 30 : append to output shape
shape[2] : 20 : do not append to output shape, dimension 2 of y has been summed over. (dot_axes[1] = 2) output_shape = (100, 30)

>>> x_batch = K.ones(shape=(32, 20, 1))
>>> y_batch = K.ones(shape=(32, 30, 20))
>>> xy_batch_dot = K.batch_dot(x_batch, y_batch, axes=(1, 2))
>>> K.int_shape(xy_batch_dot)
(32, 1, 30)

transpose

It is used to transpose a tensor followed by returning it.

Arguments

x: It can either be a tensor or variable.

Returns

It returns a tensor.

Examples

>>> var = K.variable([[1, 2, 3], [4, 5, 6]])
>>> K.eval(var)
array([[ 1.,  2.,  3.],
       [ 4.,  5.,  6.]], dtype=float32)
>>> var_transposed = K.transpose(var)
>>> K.eval(var_transposed)
array([[ 1.,  4.],
       [ 2.,  5.],
       [ 3.,  6.]], dtype=float32)
>>> inputs = K.placeholder((2, 3))
>>> inputs
<tf.Tensor 'Placeholder_11:0' shape=(2, 3) dtype=float32>
>>> input_transposed = K.transpose(inputs)
>>> input_transposed
<tf.Tensor 'transpose_4:0' shape=(3, 2) dtype=float32>

Numpy implementation

def transpose(x):
    return np.transpose(x)

gather

It helps in the retrieval of indices indices elements within the reference of tensor.

Arguments

reference: It refers to a tensor.
indices: It can be defined as an integer that represents the tensor of indices.

Returns

It outputs a tensor of same type as that of the reference.

Numpy implementation

def gather(reference, indices):
    return reference[indices]

max

It calculates tensor's maximum value.

Arguments

x: It can be defined as a tensor or variable.
axis: It refers to an integer or integers list present inside [-rank(x), rank(x)), the axis that is used to compute maximum values. If it is set to None (default), then it calculates the maximum overall dimensions.
keepdims: It is a Boolean that decides either to retain the dimensions or not. If keepdims is set to False, then tensor's rank will be reduced by 1. Else if keepdims is set to True, then the reduced dimension will be preserved with length 1.

Returns

It returns a tensor that represents the maximum values of x.

Numpy implementation

def max(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.max(x, axis=axis, keepdims=keepdims)

min

It computes the minimum value inside a tensor.

Arguments

x: It can be defined as a tensor or variable.
axis: It refers to an integer or integers list present inside [-rank(x), rank(x)), the axis that is used to compute minimum values. If it is set to None (default), then it calculates the minimum overall dimensions.
keepdims: It is a Boolean that decides either to retain the dimensions or not. If keepdims is set to False, then tensor's rank will be reduced by 1. Else if keepdims is set to True, then the reduced dimension will be preserved with length 1.

Returns

It returns a tensor that represents the minimum values of x.

Numpy implementation

def min(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.min(x, axis=axis, keepdims=keepdims)

sum

It outputs the summation of values within a tensor, along with the specified axis.

Arguments

x: It can be defined as a tensor or variable.
axis: It refers to an integer or integers list present inside [-rank(x), rank(x)), the axis that is used to compute the sum. If it is set to None (default), then it calculates the sum overall dimensions.
keepdims: It is a Boolean that decides either to retain the dimensions or not. If keepdims is set to False, then tensor's rank will be reduced by 1. Else if keepdims is set to True, then the reduced dimension will be preserved with length 1.

Returns

It returns a tensor encompassing sum of x.

Numpy implementation

def sum(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.sum(x, axis=axis, keepdims=keepdims)

prod

In conjunction with the specific axis, it computes the multiplication of values inside a tensor.

Arguments

x: It can be defined as a tensor or variable.
axis: It refers to an integer or integers list present inside [-rank(x), rank(x)), the axis that is used to compute the product. If it is set to None (default), then it calculates the overall product dimensions.
keepdims: It is a Boolean that decides either to retain the dimensions or not. If keepdims is set to False, then tensor's rank will be reduced by 1. Else if keepdims is set to True, then the reduced dimension will be preserved with length 1.

Returns

It returns a tensor encompassing product of elements within the x.

Numpy implementation

def prod(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.prod(x, axis=axis, keepdims=keepdims)

cumsum

In conjunction with the specific axis, it computes the cumulative sum of values inside a tensor.

Arguments

x: It can be defined as a tensor or variable.
axis: It refers to an integer, which is the axis that is used to compute the sum.

Returns

It returns a tensor encompassing the cumulative sum of values of x along an axis.

Numpy implementation

def cumsum(x, axis=0):
    return np.cumsum(x, axis=axis)

cumprod

In conjunction with the specific axis, it computes the cumulative product of values inside a tensor.

Arguments

x: It can be defined as a tensor or variable.
axis: It refers to an integer, which is the axis that is used to compute the product.

Returns

It returns a tensor encompassing cumulative product of values of x along an axis.

Numpy implementation

def cumprod(x, axis=0):
    return np.cumprod(x, axis=axis)

var

In conjunction with the specific axis, it computes the tensor's variance.

Arguments

x: It can be defined as a tensor or variable.
axis: It refers to an integer or integers list present inside [-rank(x), rank(x)), the axis that is used to compute the variance. If it is set to None (default), then it calculates the overall variance dimensions.
keepdims: It is a Boolean that decides either to retain the dimensions or not. If keepdims is set to False, then tensor's rank will be reduced by 1. Else if keepdims is set to True, then the reduced dimension will be preserved with length 1.

Returns

It returns tensor's variance of elements residing in x.

Numpy implementation

def var(x, axis=None, keepdims=False):
    if isinstance(axis, list):
        axis = tuple(axis)
    return np.var(x, axis=axis, keepdims=keepdims)

rnn

tf.keras.backend.rnn(
    step_function,
    inputs,
    initial_states,
    go_backwards=False,
    mask=None,
    constants=None,
    unroll=False,
    input_length=None,
    time_major=False,
    zero_output_for_mask=False,
)

It is useful for reiterating above the tensor dimension.

Arguments

step_function: It is known as RNN step function. It includes the following argument that is given below:
- input: It includes a tensor having a shape of (samples, ...) that represent batch sample's input at a particular time step. It does not include the time dimension.
- states: It can be defined as a tensor's list.
- new_states: It can be defined as a list of tensor that constitutes the same shape as well as length as that of the states, such that the initial state has to be the output tensor of the previous timestep in the list.

Returns

It outputs a tensor of shape (samples, output_dim)

inputs: It either refers to a tensor of temporal data to be atleast three-dimensional that constitute a shape of (samples, time, ...) or a nested tensor, such that each has a shape of (samples, time, ...).
initial_states: It can be defined as a tensor of shape (samples, state_size) that encompass the state's initial values to be utilized in the step function. When the state_size has a nested shape, then the nested structure will also be followed by the initial_states.
go_backwards: It can be defined as a Boolean, and if it set to True, then an interation will be performed above the time dimension in a reverse order followed by returning a reversed sequence.
mask: It refers to a binary tenor having a shape of (samples, time, 1) including a zero each and every element that has been masked.
constants: It can be defined as a constant values list, which is distributed at every single step.
unroll: It illustrates if the RNN should be unrolled or a symbolic while-loop to be used.
input_length: It can be defined as an integer or one-dimensional tensor based on the time dimension if it has fixed length or not. If it is set to variable-length input when there is no specified mask, then it will be used for masking.
time_major: It can be defined as a Boolean. If it set to true, then the shape of input as well output will be (timesteps batch, ...), else (batch, timesteps, ...) if set to false. To use time_major = True is quite an efficient task as transpose is avoided at the commencement as well as the culmination of RNN calculation. But, mostly, the TensorFlow data exists batch-major due to which, by default, this function accepts input and emits output in the form of batch-major.
zero_output_for_mask: It refers to a Boolean for which if it is set to true, then it will masked timestep output will be zeros, else the previous step output will be returned.

Returns

It returns a tuple of shape (last_output, outputs, new_states), where last_output relates to rnn's most recent output consisting a shape of (samples, …), outputs refer to a tensor of shape (samples, time, …), such that each entry outputs[s, t] corresponds to the step function's output for sample s and time t and new_states can be defined as a tensor list that represents the newest states, which are reverted by the step function of shape encompassing a shape of (samples, …).

Raises

ValueError: A value error is generated if the dimension of input is less than 3.
ValueError: It can also be raised in case if unroll is set to True, whereas input timestep isn't a static number.
ValueError: In case if the mask is provided but is not set to None and state isn't provided (i.e., len(states) ==0), then it is also generated.

Next TopicKeras Models

← prev next →