Invoke-AzHDInsightHiveJob
Submits a Hive query to an HDInsight cluster and retrieves query results in one operation.
Syntax
Invoke-AzHDInsightHiveJob
[-Arguments <String[]>]
[-Files <String[]>]
[-StatusFolder <String>]
[-Defines <Hashtable>]
[-File <String>]
[-JobName <String>]
[-Query <String>]
[-RunAsFileJob]
[-DefaultContainer <String>]
[-DefaultStorageAccountName <String>]
[-DefaultStorageAccountKey <String>]
[-DefaultProfile <IAzureContextContainer>]
[<CommonParameters>] Description
The Invoke-AzHDInsightHiveJob cmdlet submits a Hive query to an Azure HDInsight cluster and retrieves query results in one operation. Use the Use-AzHDInsightCluster cmdlet before calling Invoke-AzHDInsightHiveJob to specify which cluster will be used for the query.
Examples
Example 1: Submit a Hive query to an Azure HDInsight cluster
# Primary storage account info
$storageAccountResourceGroupName = "Group"
$storageAccountName = "yourstorageacct001"
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value
$storageContainer = "container001"
# Cluster info
$clusterName = "your-hadoop-001"
$clusterCreds = Get-Credential
# Hive job details
$statusFolder = "tempStatusFolder/"
$query = "SHOW TABLES"
Use-AzHDInsightCluster `
-ClusterCredential $clusterCreds `
-ClusterName $clusterName
Invoke-AzHDInsightHiveJob -StatusFolder $statusFolder `
-Query $query `
-DefaultContainer $storageContainer `
-DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
-DefaultStorageAccountKey $storageAccountKeyThis command submits the query SHOW TABLES to the cluster named your-hadoop-001.
Parameters
-Arguments
Specifies an array of arguments for the job. The arguments are passed as command-line arguments to each task.
| Type: | String[] |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultContainer
Specifies the name of the default container in the default Azure Storage account that an HDInsight cluster uses.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultProfile
The credentials, account, tenant, and subscription used for communication with azure
| Type: | IAzureContextContainer |
| Aliases: | AzContext, AzureRmContext, AzureCredential |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultStorageAccountKey
Specifies the account key for the default storage account that the HDInsight cluster uses.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultStorageAccountName
Specifies the name of the default storage account that the HDInsight cluster uses.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Defines
Specifies Hadoop configuration values to set when a job runs.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-File
Specifies the path to a file in Azure Storage that contains the query to run. You can use this parameter instead of the Query parameter.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Files
Specifies a collection of files that are required for a Hive job.
| Type: | String[] |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-JobName
Specifies the name of a Hive job. If you do not specify this parameter, this cmdlet uses the default value: "Hive: <first 100 characters of Query>".
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Query
Specifies the Hive query.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-RunAsFileJob
Indicates that this cmdlet creates a file in the default Azure storage account in which to store a query. This cmdlet submits the job that references this file as a script to run. You can use this functionality to handle special characters such as percent sign (%) that would fail on a job submission through Templeton, because Templeton interprets a query with a percent sign as a URL parameter.
| Type: | SwitchParameter |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-StatusFolder
Specifies the location of the folder that contains standard outputs and error outputs for a job.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
Inputs
None
