The word architecture refers to the overall structure of the neural network, including how many layers it can have and how units in the layers should be connected to each other (for instance, units across successive layers can be fully connected, partially connected, or may even skip the next layer altogether and then make connections to a layer at a much higher level in the network). With the availability of modular deep learning frameworks, such as Caffe, Torch, and TensorFlow, complex neural network designs have been revolutionized. Now we can compare neural network designs to Lego blocks, where you can build almost any structure that you can imagine. However, these designs are not just random guesses. The intuitions behind these designs are usually driven by the domain knowledge the designer has about the problem, along with some trial and error to fine-tune the final design.