You can also check which execution strategy is being used using explain:
scala> mydf.explain
scala> mydf.queryExecution.executedPlan
- www.PacktPub.com
- Preface
- Getting Started with Apache Spark
- Introduction
- Leveraging Databricks Cloud
- Deploying Spark using Amazon EMR
- Installing Spark from binaries
- Building the Spark source code with Maven
- Launching Spark on Amazon EC2
- Deploying Spark on a cluster in standalone mode
- Deploying Spark on a cluster with Mesos
- Deploying Spark on a cluster with YARN
- Understanding SparkContext and SparkSession
- Understanding resilient distributed dataset - RDD
- Developing Applications with Spark
- Introduction
- Exploring the Spark shell
- Developing a Spark applications in Eclipse with Maven
- Developing a Spark applications in Eclipse with SBT
- Developing a Spark application in IntelliJ IDEA with Maven
- Developing a Spark application in IntelliJ IDEA with SBT
- Developing applications using the Zeppelin notebook
- Setting up Kerberos to do authentication
- Enabling Kerberos authentication for Spark
- Spark SQL
- Understanding the evolution of schema awareness
- Understanding the Catalyst optimizer
- Inferring schema using case classes
- Programmatically specifying the schema
- Understanding the Parquet format
- Loading and saving data using the JSON format
- Loading and saving data from relational databases
- Loading and saving data from an arbitrary source
- Understanding joins
- Analyzing nested structures
- Working with External Data Sources
- Spark Streaming
- Getting Started with Machine Learning
- Supervised Learning with MLlib — Regression
- Supervised Learning with MLlib — Classification
- Unsupervised Learning
- Recommendations Using Collaborative Filtering
- Graph Processing Using GraphX and GraphFrames
- Optimizations and Performance Tuning
How to do it...
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.