Using the InsightEdge Web Notebook


This section describes how to use the interactive Apache Zeppelin Web Notebook with InsightEdge.

Starting the Web Notebook

The InsightEdge Apache Zeppelin web notebook can be started in any of the following ways:

  • Run the insightedge demo command; the web notebook is started automatically at localhost:9090 .
  • Start the web notebook manually at any time by running zeppelin.sh/cmd from the <XAP HOME>/insightedge/zeppelin/bin directory.

When Apache Zeppelin is running, you can browse to localhost:9090 and start exploring the pre-built notebooks:

image

Connecting a New Apache Zeppelin Notebook to InsightEdge

If you want to create a new Apache Zeppelin web notebook instead of using the example notebooks that come packaged with InsightEdge, there are two ways to run a Zeppelin notebook against InsightEdge:

  • Write a Spark application that reads and writes data to the data grid using the Spark context.
  • Write an SQL query that is interpreted by the InsightEdge JDBC interpreter (which is run directly against the data grid).

Initializing the Spark Context

In order to establish the connection between Apache Zeppelin through Apache Spark and then to the data grid, each notebook should start with a paragraph that injects the InsightEdge settings to the Spark context. Important settings include the following properties:

  • spaceName
  • lookupGroups
  • lookupLocators

These properties are injected through the following in the notebook:

  • InsightEdge class:
    case class InsightEdgeConfig(
                                 spaceName: String,
                                 lookupGroups: Option[String] = None,
                                 lookupLocators: Option[String] = None)
  • Apache Zeppelin mandatory initialization paragraph:
    %spark
    import org.insightedge.spark.implicits.all._
    import org.insightedge.spark.context.InsightEdgeConfig
    
    //spaceName is required, other two parameters are optional
    val ieConfig = new InsightEdgeConfig(spaceName = "mySpace", lookupGroups = None, lookupLocators = None)
    
    //sc is the spark context initalized by zeppelin
    sc.initializeInsightEdgeContext(ieConfig)

InsightEdge JDBC Interpreter

Apache Zeppelin uses interpreters to compile and run paragraphs. The Apache Zeppelin instance that is packaged with InsightEdge comes with a custom JDBC interpreter that enables running SQL queries directly on the data grid using the web notebook. The queries are executed by the InsightEdge SQL Driver.

Configuring the JDBC Interpreter

The JDBC interpreter connects to the data grid via a JDBC URL. To configure the URL value to point to the data grid, do the following:

  1. In the Apache Zeppelin web interface, navigate to the Interpreters section.
  2. Select the insightedge_jdbc interpreter, and click Edit.
  3. Edit the default.url parameter as follows: jdbc:insightedge:spaceName=<space-name>
  4. Save the changes you made to the interpreter.

Querying the Data Grid in Notebooks

When the JDBC interpreter is properly configured, Zeppelin paragraphs that are bound to the %insightedge_jdbc interpreter can run SQL queries directly on the data grid.

Using the Web Notebook

The Apache Zeppelin web notebook comes with sample notes. We recommend that you review them, and then use them as a template for your own notes. There are several things you should take into account.

  • The data grid model can be declared in a notebook using the %define interpreter:

    %define
    package model.v1
    
    import org.insightedge.scala.annotation._
    import scala.beans.{BeanProperty, BooleanBeanProperty}
    
    case class Product(
        @BeanProperty @SpaceId var id: Long,
        @BeanProperty var description: String,
        @BeanProperty var quantity: Int,
        @BooleanBeanProperty var featuredProduct: Boolean
        ) {
        def this() = this(-1, null, -1, false)
    }
    
    %spark
    import model.v1._
    
  • You can load external .JARs from the Spark interpreter settings, or with the z.load("/path/to.jar") command:

    %dep
    z.load("./insightedge/examples/jars/insightedge-examples.jar")
    
See also:

For more details, refer to Zeppelin Dependency Management .

  • You must load your dependencies before you start using the SparkContext (sc) command. If you want to redefine the model or load another .JAR after SparkContext has already started, you must reload the Spark interpreter.