Configuring Alluvio Edge and Data Sources

Alluvio IQ's data sources are network monitoring applications such as Riverbed Technology's Alluvio NetIM and Alluvio NetProfiler, which forward observability data to Alluvio IQ for correlation and analysis via Alluvio Edge, which handles data source connections for IQ. Each data source must be configured explicitly to communicate with Edge.

Note: This procedure uses cloud-init to configure Alluvio Edge on its first boot. cloud-init is an industry-standard instance initialization method; you can read about it in detail here. If you are using VMware ESXi as the virtual machine type for the Edge, you will need to use the NoCloud datasource.

Follow these steps to configure an Alluvio Edge instance and a data source:

  1. Create new user accounts (non-admin) on your Riverbed Technology data sources. For NetProfiler: Operator, for NetIM: Alluvio

  2. In the Alluvio IQ web UI, open the Alluvio Edge Configuration page and click Add an Alluvio Edge to display the Add an Alluvio Edge wizard.

  3. Specify the Alluvio Edge instance name and virtual machine type, and click Next.

  4. If necessary, specify the connection type for the Edge:

    • If the Edge does not have access to the Internet, specify a proxy to use.

    • If the Edge has only port 443 open to the Internet, enable Use Websockets.

    • If neither condition applies, simply click Next.

  5. Inspect the summary of the Edge definition and click Submit to make it manageable by IQ. IQ shows you a brief summary of the next steps you need to execute.

    1. Download the VM Image and cloud-init files now, or do it later by returning to the Alluvio Edge Configuration page and clicking those commands in the ... menu.

    2. Deploy the Edge VM, including the cloud-init configuration.

    3. Return to the Alluvio Edge Configuration page and add data sources after the Edge is up and reachable.

  6. Select the Edge in the Alluvio Edge Configuration page and click on the ... menu. Choose Download VM Image to obtain the OVA for the Edge.

  7. Select the Edge in the Alluvio Edge Configuration page and click on the ... menu. Choose Download Cloud Init Script to obtain the file, "cloud-init-<edge device id>.zip". The cloud-init file contains the connection string that is needed to create an ISO file, as well as the user-data and meta-data files. The cloud-init file contains the user-data and meta-data files. The user-data file contains the connection string that enables connectivity between the Edge VM and the cloud, and the meta-data file contains the hostname for the Edge VM.

  8. Unzip the cloud-init.zip file. This will unzip to a directory named cloud-init-<edge device id>, which contains two files, meta-data and user-data. This is a well known initialization practice; refer to the latest cloud-init documentation if you need clarification.

  9. For ESXi, create an ISO image with the downloaded cloud-init.zip file. Refer to the latest cloud-init documentation for the NoCloud process for clarification. Create a new directory named "cloud-init" and copy the contents of the cloud-init-<edge device id>.zip file to it (user-data & meta-data).

    1. To generate the ISO image on Linux, use the following command:

      genisoimage -output seed.iso -volid cidata -joliet -rock user-data meta-data

    2. To generate the ISO image on Mac OS, use the following command (make sure the `cloud-init` directory contains ONLY `user-data` and `meta-data`):

      hdiutil makehybrid -o seed.iso -hfs -joliet -iso -default-volume-name cidata cloud-init/

    3. To generate the ISO image on Windows: Download and execute the PowerShell script, iso_image_on_windows.ps1 . This displays the ISO Creator Tool; specify the inputs for the ISO as prompted, and click CreateISO to complete the process.

  10. For ESXi, deploy a VM using the provided OVA file and the ISO image you created.

    1. Recommended specification: (RAM: 16GB, vCPU: 4 cores, and Storage 300GB)

    2. Deploy OVA using provided OVA file - DO NOT power on automatically.

    3. Mount the ISO (seed.iso) created earlier

      On VMware, this can be added using CD/DVD drive > iso image

    4. Power ON the VM

      Upon first boot, cloud-init should run and set the initial configuration. (This may take a few minutes; wait for at least five minutes before attempting log in to the VM.)

    5. SSH to the VM and log in as admin/admin; you'll be prompted to change the password.

  11. For Azure and AWS:

    1. Recommended instance type is: “t2.xlarge” (RAM: 16GB, vCPU: 4 cores, and Storage 300GB)

    2. While configuring the instance, under “Advanced Details”, paste the content of the user-data file.

    3. After configuration, click “Launch”. AWS will prompt you about “Selecting an existing key pair or create a new key pair”. Here, select “Proceed without a key pair” to complete the deployment process

  12. If you use a firewall with a domain whitelist or a list of blocked ports, make sure to modify it to whitelist the following DNS names and to open ports 443 and 5671 for the deployed Edge:

    • mcr.microsoft.com

    • iotivmqorgqdldua.azurecr.io

    • \*.azure-devices.net

    • \*.blob.core.windows.net

    • \*.docker.io

    Note: Refer to Firewall Rules for complete firewall rules for Alluvio IQ.

  13. Check the Alluvio Edge instance's health:

    • Verify that the Status for the Edge is green on the Alluvio Edge Configuration page. This page does not refresh automatically, so refresh it explicitly if you are watching for a status change.

    • Contact Riverbed Technology in case of any errors or issues

  14. Select the Edge in the Alluvio Edge Configuration page and click ..., then click Connect a Data Source to display the Connect a Data Source wizard.

  15. Choose the data source type and specify the IP address or hostname of the data source, as well as the protocol (HTTPS is the default) and port by which to communicate, then click Submit.

  16. Add the credentials of the data source. SSH to the Edge VM and run alluvio-conf update-data-source-credentials. Use the credentials you created in step 1. For example:

  17. On the Alluvio Edge Configuration page, select the data source and choose Edit from the ... menu. Specify the Edge VM IP address and enable the data source. With enabling, the Edge is connected to the data source and ready to push data to Alluvio IQ.

  18. Select the Edge in the Alluvio Edge Configuration page and click the arrowhead to its left; this will show the status of connected data sources. The web UI should show green status for the Edge and connected data sources.