In this chapter, we will continue with our exploration into RabbitMQ clients, but this time in Python.
Python provides an almost perfect environment into this work, because it is dynamic in nature and over the (many) years, it has established itself as an almost de-facto toolset for backend engineers and data scientists. It is only fitting that during earlier time, it has allowed great tooling to evolve, and we'll explore them here.
In this chapter, you will learn the following topics:
- Using RabbitMQ from Python with the help of a library called Pika
- Using Pika to implement a sample use case project
- Exploring how Pika will help you tackle almost any AMQP-related task
- Understanding Celery—a powerful background task library
- Understanding how the work we did so far adapts into Celery, why it is better, and how to use its more-advanced usage
For this study, we'll choose the world of web scraping. Python is universally known for this use, because if you're a python programmer, you're one of the following:
- A full-stack Python programmer, developing client and server applications
- A data scientist, using NumPy, scikit-learn, and so on
- A systems programmer, as Python comes standard with most, if not all, Linux distributions
- A big-data programmer, utilizing Hadoop and Python-Hadoop streaming or Python for building map-reduce jobs
A successful web scraping infrastructure for a successful product generates a lot of data (a big data programmer) that we can learn from (a data scientist); it is rather taxying to properly build at scale (a systems programmer), and somehow, you'd want to turn all this into a product (a full stack Python programmer).
As you see, once you cover web scraping, you can build a lot of products; you can also get into a lot of technological domains in the same programming language and platform. And if this is not enough, any of these domains are a hot recruitment targets today.
I hope this is enough motivation to explore Python and how RabbitMQ fits into this world.
One of the easy things do in Python is getting dependencies—the idea of using the libraries that you've chosen to integrate into your code, their dependencies, and well, you get the picture.
Python comes with a few tools to do this, and you can pick what you wish based on what works for you the best. Some of these are as follows:
pip: This is a Python dependency managereasy_install: This is yet another Python dependency managerapt-get: This is available on Debian and Ubuntu Linux; the standard package manager can also install Python packages