last

Gets the last record in the RelationalGroupedDataset.

The last API has several implementations as follows. The exact API used depends on the specific use case:

def last(columnName: String): Column
Aggregate function: returns the last value of the column in a group.
def last(e: Column): Column
Aggregate function: returns the last value in a group.
def last(columnName: String, ignoreNulls: Boolean): Column
Aggregate function: returns the last value of the column in a group.
def last(e: Column, ignoreNulls: Boolean): Column
Aggregate function: returns the last value in a group.

Let's look at example of invoking last on the DataFrame to output the last row:

import org.apache.spark.sql.functions._
scala> statesPopulationDF.select(last("State")).show
+------------------+
|last(State, false)|
+------------------+
| Wyoming|
+------------------+
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset