DataFrame.DropDuplicates Method

Definition

Namespace:

Microsoft.Spark.Sql

Assembly:

Microsoft.Spark.dll

Package:

Microsoft.Spark v1.0.0

Overloads

DropDuplicates(String, String[])	Returns a new DataFrame with duplicate rows removed, considering only the subset of columns.
DropDuplicates()	Returns a new DataFrame that contains only the unique rows from this DataFrame. This is an alias for Distinct().

DropDuplicates(String, String[])

Returns a new DataFrame with duplicate rows removed, considering only the subset of columns.

public Microsoft.Spark.Sql.DataFrame DropDuplicates (string col, params string[] cols);

member this.DropDuplicates : string * string[] -> Microsoft.Spark.Sql.DataFrame

Public Function DropDuplicates (col As String, ParamArray cols As String()) As DataFrame

Parameters

col

String

Column name

cols

String[]

Additional column names

Returns

DataFrame

DataFrame object

DropDuplicates()

Returns a new DataFrame that contains only the unique rows from this DataFrame. This is an alias for Distinct().

public Microsoft.Spark.Sql.DataFrame DropDuplicates ();

member this.DropDuplicates : unit -> Microsoft.Spark.Sql.DataFrame

Public Function DropDuplicates () As DataFrame

Returns

DataFrame