4Paradigm Sage Hyper-Engine
The Middleware that Supports the Development and Operation of AI Applications

Product Value

End-to-end

Comprehensive algorithm coverage supports the entire process: application research, development, running, operation and maintenance.

High Technology

High dimensional AI model, with average response time of less than 30 microseconds, self-iteration of model.

High Performance

Supported by various types of self-developed high-performance algorithms and techniques, the computational efficiency can be hundreds or even thousands of times higher than Spark.

Expandable

Standardized process supports the production of third-party models such as TensorFlow, PMML and H2O.

Enterprise-class

Supports data and resource isolation, interfacial operation and maintenance and customized alarm policies, and collection, retrieval, and persistence of logs.

Easy Integration

Perfect API and SDK, providing support for the entire process of machine learning.

High Availability

With stateless internal micro-service and built-in network load balancer, creating a system that has high availability and horizontal scaling.

Product Functions

  • Data Integration
    Unified Access for Multi-Source Heterogeneous Data

    Provides unified access engine for structured data and unstructured data, such as images and texts, in order to establish seamless connection with mainstream data warehouses and relational databases; supports composite data access in AI application scenarios, including tagging image datasets, samples and models.

    Time Series Data Grouping Management

    Through data sharding storage technology, data sharding can be performed at a certain time field, and time slicing can be used as a basis for obtaining data, thereby achieving a quick positioning of data shards required by the model in the machine learning scenario.

  • Data Governance
    Unified Governance of the Whole Domain Data

    4Paradigm Sage Data Platform provides a unified governance framework and meta-information management for heterogeneous data. The isomorphic data is integrated through the data group, isolated through the data domain, and could support sub-businesses and sub-scenario data management and comprehensively improve the data governance level in large-scale AI application process in horizontal and vertical dimensions.

    Data Lifecycle Positioning and Tracking

    The unified locator (prn) builds a global identity system for enterprise data, enabling quick locating and tracking of the changes in the lifecycle of the data.

  • Model Training
    High-dimensional Features

    4Paradigm’s self-developed ultra-high-dimensional algorithm can process data features from a level of tens of billions to trillions. In comparison with conventional AI algorithms, besides the macro/high-frequency features, it can also effectively include micro/long tail features, which can greatly improve the model prediction accuracy and enhance the success rates of businesses.

    Unified Online/Offline Feature Processing

    Through the self-developed unified feature computing engine, it supports the online/offline feature processing with unified feature scripts, and is compatible with Sage and third-party computing frameworks, achieving an all-at-once development and on-demand operation.

    High-Performance and Real-Time Feature Computing Engine

    With the support of the feature engine compiler, users can automatically convert feature scripts into highly optimized java code and finally into bytecode, allowing scientists to program high-performance feature projects. In anti-fraud scenarios, with the same feature set, the performance is 10 times better than PySpark.

    Comprehensive Algorithm Coverage

    It covers both machine learning and deep learning algorithms including logic regression that is 100 times more efficient, GBDT with strong interpretability, self-developed large-scale discrete sparse neural network DSN, He-Treenet which helped a top bank win a scientific and technological innovation award, and linear classifier. For large-scale discrete data, GDBT, the self-developed large-scale computing framework for machine learning can provide unique algorithmic values.

  • Model Verification
    Self-iteration of Model

    4Paradigm’s ‘Machine Learning Cycle’ theory supports the artificial intelligence application system and covers the entire process of AI application construction. The model can perform self-iteration according to the changes in business so as to provide support to enterprise business decisions in real time.

  • Model Launch
    Unified Deployment of Customized Model

    It supports the production of third-party models, such as Tensorflow, PMML and H2O, with a standardized process and the online multi-instance deployment mode, with balanced load and elastic scaling. It can also perform real-time monitoring on the model serving indicators such as QPS, request time, and request timeout rate.

    In-memory Time Series Database RTIDB

    RTIDB is 4Paradigm’s self-developed high-performance in-memory time series database, which ensures performance and the effectiveness of model online services in many hard real-time AI scenarios such as anti-fraud in finance.

    Real-time Online Computing

    With strong end-to-end real-time computing power, it can perfectly support the whole process of online machine learning applications while ensuring the effectiveness of high-dimensional models at the same time.

  • Monitoring Statistics
    Multi-Dimensional Monitoring, Operation and Maintenance

    For the large-scale real-time logs generated by 4Paradigm Sage systems and AI applications, it supports one-stop management services from collection to warehousing and from query to display. Users can establish customized monitoring indicators and rules for alerts, and flexibly display them on the monitoring dashboard in order to achieve personalized monitoring, operation and maintenance. It also supports seamless connection with the internal operation and maintenance of enterprises to help enterprises achieve innovative AI application.

  • Management Service
    Multi-tenant System

    The multi-tenant system designed for enterprise users can achieve the isolation of data resources and computing resources among tenants, so as to ensure asset security, optimize resource allocation and improve resource utilization efficiency.

    Complete API & SDK

    With multiple standard components such as storage center and online service engine, customers, partners and developers can carry out the one-stop building of the entire process of machine learning, based on their own business/IT system, which can shorten the development cycle and reduce the development cost of AI applications.