Edit

Share via

Add-AzHDInsightConfigValue

  • Reference
Module:
Az.HDInsight

Adds a Hadoop configuration value customization and/or a Hive shared library customization to a cluster configuration object.

Syntax

Add-AzHDInsightConfigValue
   [-Config] <AzureHDInsightConfig>
   [-Core <Hashtable>]
   [-HiveSite <Hashtable>]
   [-HiveEnv <Hashtable>]
   [-OozieSite <Hashtable>]
   [-OozieEnv <Hashtable>]
   [-WebHCat <Hashtable>]
   [-HBaseSite <Hashtable>]
   [-HBaseEnv <Hashtable>]
   [-Storm <Hashtable>]
   [-Yarn <Hashtable>]
   [-MapRed <Hashtable>]
   [-Tez <Hashtable>]
   [-Hdfs <Hashtable>]
   [-RServer <Hashtable>]
   [-SparkDefaults <Hashtable>]
   [-SparkThriftConf <Hashtable>]
   [-Spark2Defaults <Hashtable>]
   [-Spark2ThriftConf <Hashtable>]
   [-DefaultProfile <IAzureContextContainer>]
   [<CommonParameters>]

Description

The Add-AzHDInsightConfigValue cmdlet adds a Hadoop configuration value customization, such as core-site.xml or hive-site.xml, and/or a Hive shared library customization to the HDInsight configuration object created by the New-AzHDInsightClusterConfig cmdlet.

Examples

Example 1: Add a custom configuration value to the cluster configuration object

# Primary storage account info
$storageAccountResourceGroupName = "Group"
$storageAccountResourceId = "yourstorageaccountresourceid"
$storageAccountName = "yourstorageacct001"
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value

$storageContainer = "container001"

# Cluster configuration info
$location = "East US 2"
$clusterResourceGroupName = "Group"
$clusterName = "your-hadoop-001"
$clusterCreds = Get-Credential

# If the cluster's resource group doesn't exist yet, run:
#   New-AzResourceGroup -Name $clusterResourceGroupName -Location $location

# Config values
$coreConfigs = @{"io.file.buffer.size"="300000"}
$mapRedConfigs = @{"mapred.map.max.attempts"="2"}

# Create the cluster
New-AzHDInsightClusterConfig `
            | Add-AzHDInsightConfigValue `
                -Core $coreConfigs `
                -MapRed $mapRedConfigs `
            | New-AzHDInsightCluster `
                -ClusterType Hadoop `
                -OSType Windows `
                -ClusterSizeInNodes 4 `
                -ResourceGroupName $clusterResourceGroupName `
                -ClusterName $clusterName `
                -HttpCredential $clusterCreds `
                -Location $location `
                -StorageAccountResourceId $storageAccountResourceId `
                -StorageAccountKey $storageAccountKey `
                -StorageContainer $storageAccountContainer

This command adds a Hadoop configuration value to the cluster named your-hadoop-001.

Parameters

-Config

Specifies the HDInsight cluster configuration object that this cmdlet modifies. This object is created by the New-AzHDInsightClusterConfig cmdlet.

Type:AzureHDInsightConfig
Position:0
Default value:None
Required:True
Accept pipeline input:True
Accept wildcard characters:False

-Core

Specifies the Core Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-DefaultProfile

The credentials, account, tenant, and subscription used for communication with azure

Type:IAzureContextContainer
Aliases:AzContext, AzureRmContext, AzureCredential
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-HBaseEnv

Specifies the HBase Env configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-HBaseSite

Specifies the HBase Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Hdfs

Specifies the HDFS configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-HiveEnv

Specifies the Hive Env configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-HiveSite

Specifies the Hive Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-MapRed

Specifies the MapRed Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-OozieEnv

Specifies the Oozie Env configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-OozieSite

Specifies the Oozie Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-RServer

Specifies the RServer configurations. Valid only for RServer clusters.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Spark2Defaults

Specifies the Spark2 Defaults configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Spark2ThriftConf

Specifies the Spark2 Thrift SparkConf configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-SparkDefaults

Specifies the Spark Defaults configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-SparkThriftConf

Specifies the Spark Thrift SparkConf configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Storm

Specifies the Storm Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Tez

Specifies the Tez Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-WebHCat

Specifies the WebHCat Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Yarn

Specifies the YARN Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

Inputs

AzureHDInsightConfig

Outputs

AzureHDInsightConfig