/
Jethro Management Package for Ambari

Jethro Management Package for Ambari

Feb 19, 2019

The purpose of this document is to describe the Jethro management package for Apach Ambari.

The overview of Ambari and Jethro's management package for Ambari will be followed by installation, management and monitoring, and uninstall instructions.

Overview

In a nutshell, Ambari is an open-source management platform for provisioning, managing, monitoring and securing Apache Hadoop clusters. Jethro's Ambari service goal is to supply a simple integration with Ambari, so that HDP users will be able to deploy and manage Jethro across their Hadoop clusters. Once integrated, users will have a ce

Service Configs

ntral location for monitoring their Jethro clusters, in terms of security, cluster health and resource utilization.

Jethro Components

The Jethro's components exposed in Ambari within the Jethro Ambari service are:

  1. Jethro Server - The query engine of Jethro. It can be deployed on any host (minimum 1 host). The more hosts this component will be on, the better availability the BI users will get. 

  2. Jethro Maint - The maintenance Linux service of Jethro. It can be deployed on any host (minimum 1 host). One component per Jethro instance, should be active at any time.

  3. Jethro Scheduler - The scheduler Linux service of Jethro. It can be deployed on any host (minimum 1 host). At least one component per Jethro instance, should be active at any time.

  4. Jethro Manager - A web interface served by a Linux service that provides wider management abilities to Jethro's administrator. The web is also accessible beyond Ambari's web. Only a single manager component can be deployed, and any host can be used for that although it is recommended to install it on the Name Node.

During the installation of Jethro service for Ambari, Ambari service advisor will automatically provide a recommendation for a basic deployment of all four components, on the available hosts .

Jethro Metrics

  1. Number of running instances - Will be presented as a gadget by default.

  2. Jethro instances storage size - Will be presented as a gadget by default.

  3. Running maint service (per instance) - Will be used as an alert, in case any Jethro Instance doesn't have one active maint component, on any of hosts this Instance is active on.

  4. Running load scheduler service (per instance) - Will be used as an alert, in case any Jethro Instance doesn't have at least one active scheduler component, on any of hosts this Instance is active on.

Jethro Parameters

The parameters exposed through Ambari, are Jethro's Instances global parameters.

  1. Enable cubes - The default value is '1' (enabled). It represents the Jethro configuration parameter: dynamic.aggregation.auto.generate.enable.

  2. Cube queries server  - The default value is 'auto' (localhost). When using this value, each host will function as a query engine for cubes creation and maintenance. To assign a specific host for it, replace this value with the desired host's details, according to the format: <IP>:<PORT> (without '<>'). The IP should be the IP of the host which will be used as a cube queries server (must have an active Jethro Server component), and the PORT should be the port of the Instance on that host.

Note

Please note:

  1. Editing Jethro's cubes parameters through Ambari, for more than one instance, is not recommended, since the query engine defined in the 'Cube queries server' belongs to a single Instance, and therefore cannot be used for a different Instance needs. The only value that can fit all Instances at the same time, is 'auto'.

  2. Only Instances which were deployed by Ambari, can updated their parameters through Jethro Ambari service.

Working with multiple Jethro instances under Ambari

Jethro's Ambari service, supports well the usage of a single Jethro Instance per host. However, having 2 or more Jethro instances on the same host, creates a limitation:

You cannot deploy more than one Jethro instance, to the same host.

The reason for that is that Ambari represents less levels of objects than Jethro. While Jethro uses a 4-level object model (Application ,Instance, Host, Service), Ambari uses a 3-level object model (Service, Host, Component). This creates a requirement to reduce one Jethro object model from being used in Ambari, which limits the ability to deploy more than one Jethro instance, per host.

Installation

Prerequisites

The integration is supported for:

  • Ambari 2.4 and above

  • Centos/Redhat 6 and above

Please note - If you are using a Centos/Redhat version earlier than 7.0, make sure to update the default links to the Jethro and Jethro Manager RPM files, during the process of adding the Jethro service, to RPM links that supports these OS versions. The default RPM links are for versions of Jethro that supports Centos/Redhat 7.0 and up.

Management Package Installation

  1. Log in your ambari-server with user root, or with a sudoer (requires to add sudo before every command).

  2. Copy/Download the Jethro management pack (mpack) to your ambari-server (jethro-mpack.tar.gz - don't ungz it).

  3. Install the Jethro management pack on your ambari-server:

    ambari-server install-mpack --mpack=<path to the mpack file>

  4. Restart the ambari-server:

    ambari-server restart

  5. Create an Extension Link, to link the installed extension to Amabri according to your HDP stack version:

    curl -u admin:admin -H 'X-Requested-By: ambari' -X POST -d '{"ExtensionLink": {"stack_name": "HDP", "stack_version":"<your stack version>", "extension_name": "JethroExt", "extension_version": "1.0.0"}}' http://localhost:8080/api/v1/links/

    For example, if you are running HDP 2.6.0:

    curl -u admin:admin -H 'X-Requested-By: ambari' -X POST -d '{"ExtensionLink": {"stack_name": "HDP", "stack_version":"2.6", "extension_name": "JethroExt", "extension_version": "1.0.0"}}' http://localhost:8080/api/v1/links/

     *To find the HDP stack version - Open Ambari, click on the Admin tab and then select the Versions tab.

Adding the Jethro Service to Ambari

After the installation of the package completes, you'll be able to add Jethro's service for Amabri's web interface:

  1. Navigate to Ambari's home page. Click 'Actions' (buttom-left on the services bar), and click 'Add Service':

  2. Select the Jethro checkbox from the available services, and click next:

  3. Amabari will display all the components contained in the selected service. You can select a target host for each component.
    It is also possible to add more components (for multiple hosts deployment):

Jethro's Ambari Service Configs

Before every deployment, Jethro's Ambari configuration properties are required to be set.

During the process of adding Jethro's Ambari service, you will be required to define for the first time, at least two mendatory parameters:

  1. A name for the Jethro Instance.

  2. A storage path on HDFS, to be used as the Instance's storage.
    It is assumed that the given HDFS path was already defined prior to every deployment of an instance, and owned by user 'jetrho'. The recomnded path is /user/jethro/instances.

If the instance name provided already exists on the provided storage path, the Instance will be 'attached' to the host. Otherwise, the Instance will be created.

The configs screen will be accessible afterwards also via the 'configs' tab of the Jethro Ambari service.