In this chapter, we looked into Google Cloud Dataproc and how to create a customized Dataproc cluster. We also learned how to submit jobs and how to manage a cluster, and, finally, the management part included scaling, graceful decommissioning, and deleting the cluster.
- Title Page
- Copyright and Credits
- Packt Upsell
- Contributors
- Preface
- The Case for Cloud Computing
- Introduction to Google Cloud Platform
- Compute Choices – VMs and the Google Compute Engine
- GKE, App Engine, and Cloud Functions
- GKE
- Creating a Kubernetes cluster and deploying a WordPress container
- Using the features of GKE
- Google App Engine – flexible
- Google App Engine – standard
- Hosted web apps with App Engine Standard
- Typical App Engine architecture
- Deploying and running on App Engine Standard
- Traffic splitting
- Serverless compute with cloud functions
- Cloud Functions triggered by HTTP
- Cloud Functions triggered by Pub/Sub
- Cloud functions triggered by GCS object notifications
- Summary
- Google Cloud Storage – Fishing in a Bucket
- Knowing when (and when not) to use GCS
- Serving Static Content with GCS Buckets
- Storage classes–Regional, multi-regional, nearline, and coldline
- Working with GCS buckets
- Creating buckets
- Transferring data in and out of buckets
- Use case – Object Versioning
- Use case – object life cycle policies
- Use case – restricting access with both ACLs and IAM
- Use case – signed and timed URLs
- Use case – reacting to object changes
- Use case – using customer supplied encryption keys
- Use case – auto-syncing folders
- Use case – mounting GCS using gcsfuse
- Use case – offline ingestion options
- Summary
- Relational Databases
- NoSQL Databases
- BigQuery
- Identity and Access Management
- Managing Hadoop with Dataproc
- Load Balancing
- Networking in GCP
- Logging and Monitoring
- Infrastructure Automation
- Security on the GCP
- Pricing Considerations
- Effective Use of the GCP
- Eat the Kubernetes frog
- Careful that you don't get nickel-and-dimed
- Pay for what you allocate not what you use
- Make friends with the gsuite admins
- Try to find reasons to use network peering
- Understand how sustained use discounts work
- Read the fine print on GCS pricing
- Use BigQuery unless you have a specific reason not to
- Use pre-emptible instances in your Dataproc clusters
- Keep your Dataproc clusters stateless
- Understand the unified architecture for batch and stream
- Understand the main choices for ML applications
- Understand the differences between snapshots and images
- Don't be Milton!
- Summary
- Other Books You May Enjoy
Summary
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.