Glossary
Live Platform core concepts
Last updated
Live Platform core concepts
Last updated
Term
Definition / Comments
Platform
Live is extensible for rapid developing applications by reusing software components for data analysis (scheduled ETLs or ad hoc), visualization and monitoring
It is more than a software library and SDK and also different from infrastructure services. More like a framework but also with UI and complete system features.
Event
Basic atomic data unit of the platform.
Event Type: is used as a partitioning key in different parts of the platform.
Event time: at least one time domain is required, ingestion time is assumed
An event must be serializable as a JSON object. But no rigid schema is required.
Events can be pulled from data sources or pushed to Live. i.e. some people call "pushed events" as "real time events" (because they are arriving now to the platform)
Query
A Live Query provides a unified model for both querying and processing events. A query can run in historical mode only, "following real time" mode only or start in historical and then switch to continuous.
The two modes highlights the two different cases:
Unbounded data processing
Bounded data processing
When in the following mode (when time span includes the relative "now"), we are processing an unbounded data stream.
When in historical only mode, we are processing a bounded data collection.
An easy to understand difference between "table" and "stream" models is that stream is data at motion and table is data at rest.
A Live Query can output control events ortogonal to data events.
Time Span
Every query has a time interval filter, which can be "none". More information here.
This required time component poses difficulties for most people starting with Live, but "from ts 0" is a common way for saying "in all time", when one doesn't want slicing in time. The following query makes sense only for "time spans" which include the "relative now" (indirectly or explicitly). i.e. "last hour" is an ever moving time interval that needs to be reified
Storage Provider
Live abstraction for a persistent data system. External records can be read and translated to events and read by a Live Query in the uniform event model
Query Provider
Live abstraction for a runtime engine (Pipes, SQL, groovy, etc). Supports continuous computation where computation outputs results incrementally over time (independently on how data is read)
Real Time
Live's system clock. The default pipes provider supports "following real time" by processing events in system time.
Stream computing
When computation can be done by incrementally accepting new data points (hopefully with a bounded use of memory, but not always is)
Plugin
a file that contains JVM or JS code that runs embedded in the Live backend process or in the browser Live UI process. It can extend the platform by adding new features and modify existing behavior. In Live, 1 plugin is 1 .jar or .zip file, it has one name, one version and can depend on other plugins.
Pipes
A declarative language for expression transformations and aggregations over time used in Live as its default query provider. It can perform advanced filtering of input records. It is best suited for processing events in system time or to filter | join | aggregate data from multiple|heterogeneous databases.
Feature
Visible UI or observable system behavior created by code artifacts. A software update / release typically contains "new features" or modifies existing "features". Also typically, a feature can be turned or off individually without impacting others.
Dashboard
Dashboard contains widgets, one widget can have multiple Live Queries associated with it and a UI component for rendering it in the web browser. By default, Live only allows pipes-based expressions in widgets, but some Live branches allow using different query providers.
Rule
One rule runs one Live Query in continuous mode for triggering (creating) alert events
Extension element
Live has a backend entity called extension to represent integration instances (something than can be started/stopped)
Storage
Managed integration with a database, can be readonly (postgresql or mongodb or jdbc/sql). Some other sources of data are not necessarily storages, such as Kafka queues.