Cartesian join happens when every row of one DataFrame is joined with every row of the other DataFrame. This can explode the number of rows very fast. Two DataFrames, each with a modest 100 K rows, can result in 10 billion rows with a cartesian join.
- www.PacktPub.com
- Preface
- Getting Started with Apache Spark
- Introduction
- Leveraging Databricks Cloud
- Deploying Spark using Amazon EMR
- Installing Spark from binaries
- Building the Spark source code with Maven
- Launching Spark on Amazon EC2
- Deploying Spark on a cluster in standalone mode
- Deploying Spark on a cluster with Mesos
- Deploying Spark on a cluster with YARN
- Understanding SparkContext and SparkSession
- Understanding resilient distributed dataset - RDD
- Developing Applications with Spark
- Introduction
- Exploring the Spark shell
- Developing a Spark applications in Eclipse with Maven
- Developing a Spark applications in Eclipse with SBT
- Developing a Spark application in IntelliJ IDEA with Maven
- Developing a Spark application in IntelliJ IDEA with SBT
- Developing applications using the Zeppelin notebook
- Setting up Kerberos to do authentication
- Enabling Kerberos authentication for Spark
- Spark SQL
- Understanding the evolution of schema awareness
- Understanding the Catalyst optimizer
- Inferring schema using case classes
- Programmatically specifying the schema
- Understanding the Parquet format
- Loading and saving data using the JSON format
- Loading and saving data from relational databases
- Loading and saving data from an arbitrary source
- Understanding joins
- Analyzing nested structures
- Working with External Data Sources
- Spark Streaming
- Getting Started with Machine Learning
- Supervised Learning with MLlib — Regression
- Supervised Learning with MLlib — Classification
- Unsupervised Learning
- Recommendations Using Collaborative Filtering
- Graph Processing Using GraphX and GraphFrames
- Optimizations and Performance Tuning
The cartesian join
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.