Creating ADX Environment Using CLI

The other approach to create the ADX cluster and the database is by using the command line interface (CLI) commands. The CLI commands can be run either from the cloud shell from the browser or locally using the shell interface. Azure hosts the cloud shell.

You can download the latest CLI for Windows from here, which will install all the required service modules.

The first step is to login to the Azure subscription. This is done using the “az login” command. Once the command is run, it will open a browser for you to provide your credentials. After login, you can close the browser. The shell will list the details of your subscription.

You can also use “az account list” command to list all the subscriptions.

The second step would be to set your context to the subscription, where you want to create and configure the Azure Data Explorer. You do this using the command “az account set –subscription” command, where you pass the subscription id as parameter.

Now that we have the subscription selected, we will need to create the resource group using the command “az group create”

Finally, as the stage is set, we will get started with configuring the Azure Data Explorer gateway using the Azure CLI

We will create the cluster using the “az kusto cluster create” command, where we pass the name of the cluster to be created, resource group name, and the SKU detail. As usual, it takes a couple of minutes to be provisioned.

Once that is completed, the final step will be to create the database for the cluster to enable data ingestion. We will do this using the command “az Kusto database create”.

To remove the cluster you can use the “az kusto cluster delete” command to remove the cluster.

As it was previously stated, before you can start working with the CLI commands on your local machine, you need to download and install the latest CLI version.

climsiinstall

climsiinstall1

climsiinstall2

Now that the CLI for Windows has been installed, it is time to start buiding our ADX environment. The step-wise commands are listed and explained as below:

  1. Check the version of the CLI version. Should be greater than 2.0.4
    az --version
  2. Login to your Azure subscription

    az login

    clilogin
    In the above screenshot, you can see that there are two different subscription. You can set one active subscription to create your ADX cluster and database.

  3. Use the “az account list” command to list all the subscriptions
    az account list
  4. In order to set the subscription for creating resource, use the “az account set” command
    az account set --subscription <subscription id>
  5. Now that we have the subscription set, we need to create a resource group. In case you already have an existing resource group, you can skip this step.
    az group create --name <name> --location <location>

    Once the command is successfully completed, you will see the below screen.
    clicreateresource

  6. After the resource group has been successfully created, it is now time to create the Azure Data Explorer Cluster
    az kusto cluster create --cluster-name <cluster name> --sku name=<sku name> tier=<sku tier> capacity=<capacity count> --resource-group <resourcegroup name> --location <location>

    –sku name: name of the sku chosen
    –sku tier: SKU tier for the ADX cluster
    –sku capacity: The number of VM instances configured

    After successful creation of the ADX cluster, you will see the output as the below screen.

    clicluster

  7. Once the ADX cluster has been created, it’s now time to create the cluster database. Use the below command.
    az kusto database create --cluster-name <cluster name> --resource-group <resourcegroup name> --database-name <database name> --read-write-database soft-delete-period=<soft delete period in days> hot-cache-period=<hot cache period in days> location=<location>

    Once the database creation is complete, you will see the screen below
    clicreatedatabase
    -Soft-delete-period: Period in days until the data can be retained
    -hot-cache-period: Number of days for which the data remains in the hot cache

Azure Data Explorer Cluster Actions Using CLI

Now that the cluster and database creation is complete, you can start using ADX for analytics and ETL jobs. There are other actions as well that can be performed on the Azure Data Explorer cluster and associated database. Some of the important one are listed below.

  • az kusto cluster start – Although the cluster automatically starts when you create it, but you can start the ADX Cluster using az kusto cluster start
    az kusto cluster start --cluster-name <cluster name> --resource-group <resourcegroup name>
  • az kusto cluster stop – In order to stop the Azure Data Explorer cluster use the command below
    az kusto cluster stop --cluster-name atcsladx --resource-group atcsl
    
  • az kusto cluster update – To update the existing ADX cluster use the az kusto cluster update CLI command.
    az kusto cluster update --cluster-name <cluster name> --resource-group <resourcegroup name> --sku name=Standard_D14_v2 capacity=5 tier=Standard
    
  • az kusto database update – For updating the cluster database, use the az kusto database update command
    az kusto database update --cluster-name <cluster name> --resource-group <resourcegroup name> --database-name <database name> --read-write-database soft-delete-period=P365D hot-cache-period=P20D location="Central US"

    While the update is running on the ADX cluster as a result of the actions taken, if you go to Azure portal, under ADX Cluster overview page, you will notice that the state shows to be as “updating” along with a warning sign that “Your Azure Data Explorer cluster is currently in state Updating”.

    adxupdate1

    After the updates are complete, you can go to the ADX cluster and verify your changes by clicking on the Database link under Data.

    clidatabaseupdate

For a more comprehensive list of actions using Azure CLI, you can visit Microsoft Docs

Previous>> Creating ADX Environment Using PowerShell


Part – 1: Data Science Overview

Part – 2: Understanding Azure Data Explorer

Part – 3: Azure Data Explorer Features

Part – 4: Azure Data Explorer Service Capabilities

Part – 6: The Kusto Query Language

Part – 7: Data Obfuscation in Kusto Query Language

Part – 8: Data Ingestion Preparation: Schema Mapping

Part – 9: Overview of data ingestion in Azure Data Explorer

Part – 10: Managing Azure Data Explorer Cluster

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: