It is important that we understand some of the terminology used within the Microsoft CNTK toolkit. Let’s look at some of that terminology now:

• Tensors: All CNTK inputs, outputs and parameters are organized as tensors. It should also be noted that minibatches are tensors as well.
• Rank: Each tensor has a rank. Scalars are tensors with a rank of 0, vectors are tensors and have a rank of 1, and matrices are tensors with a rank of 2.
• Static axis: The dimensions listed in 2 are referred to as axes. Every tensor has static and dynamic axes. A Static axis has the same length throughout its entire life.
• Dynamic axis: Dynamic axes, however, can vary their length from instance to instance. Their length is typically not known before each minibatch is presented. Additionally, they may be ordered.
• Minibatch: A minibatch is also a tensor. It has a dynamic axis, which is called the batch axis. The length of this axis can change from minibatch to minibatch.

At the time of writing of CNTK supports one additional single additional dynamic axis, also known as the sequence axis. This axis allows the user to work with sequences in a more abstract, higher-level manner. The beauty of sequence is that whenever an operation is performed on a sequence, the CNTK toolkit does a type-checking operation to determine safety: