Lindorm CQL : How does Lindorm implement Cassandra CQL compatible

1. BACKGROUND

Aliyun Multi-model database Lindorm is a Semi-structured, structured standardized storage system for massive data scenarios, and is widely used inside and outside Alibaba group. Lindorm has been committed to “make enterprise data available and visiable” and continue to maintain rapid capability updates and technology upgrades. On the one side , Lindorm reduce unit storage costs by introducing low-cost storage pools, at the same time sense user data popularity through intelligent cold and hot separation technology, and then improve query performance. On the other side, Lindorm provides rich global secondary index and search engine capabilities, users can use this through native Lindorm API, HBase compatible API, Lindorm SQL query mode compatible with JDBC standards. Besides Lindorm is a Cloud Native Multi-model database, it can support wide-column、 time series、 search engine、 distribute file system engine. The data of each model is integrated and interoperable, and one write is readable everywhere to meet the needs of users in different scenarios, making application development more agile efficient.

Apache Cassandra is an open source NOSQL database, its design and implementation is based on Google’s bigtable and Amazon’s Dynamo. It has been widely loved in the open source community, the SQL-Like Language CQL(Cassandra Query Language) secondary index UDF/UDT/UDA and other capabilities have a strong mass base.

We decided to do Lindorm compatibility with Cassandra CQL after some research, analysis and planning. So when Alibaba Cloud’s self- developed product Lindorm encounters Apache’s top project Cassandra, what sparks will it collide?

2. The reason for doing Lindorm compatibility with Cassandra CQL

2.1. Benefits of compatibility

2.1.1. customer’s benefits

By studying the related functions and features of Cassandra, we found that some features can bring many benefits to users of the cloud-native multi-mode database Lindorm:

• CQL can improves development efficiency and optimizes user experience

CQL is short for Cassandra Query Language, is a like of SQL-Like Language that the Cassandra community developed for support better access to Cassandra. Users who are familiar with SQL operations can learn the use of CQL easily and quickly. Lindorm compactibility with Cassandra CQL can bring some benefits for lindorm users:

1. Richer features: CQL provide rich data type including 22 native data type、 collection type、 User-defined-type and so on, rich function entrances (DDL/DML/ACL) and very much index operation entrances;

2. Easy to Unstand: CQL is completely open source, with more learning materials, which is convenient for users to get started quickly and lower the threshold for learning to use Lindorm;

3.Ecological perfection: The Cassandra CQL client supports multiple languages such as Python, Go, C++, Java, NodeJs, C#, and can also support multiple frameworks such as spring and orm frameworks. Users can quickly get started and use them conveniently.

In addition, CQL can natively support NoSQL-specific capabilities such as TTL and JSON, and is very suitable as a unified query language for semi-structured data. For the specific introduction of CQL language, please refer to the specific introduction of CQL. After being compatible with Cassandra CQL, users can use Lindorm more efficiently, improve user experience, and lower the threshold for Lindorm use.

• Rich functions continue to meet the increasing needs of users

In addition to CQL, Cassandra also exposed a wealth of indexing capabilities: native secondary index, SASI index, Materialized View and the recently implemented SAI, and has a series of user-defined operations such as UDT/UDA/UDF/Trigger. With the continuous iteration of the community and commercial companies, the current features of Cassandra can represent the evolution direction of users’ demand for NOSQL databases to a certain extent. After Lindorm is compatible with Cassandra CQL, users’ needs for storage and database products can continue to be met, thereby simplifying the complexity of the user-side technical architecture.

As a cloud-native database that has evolved for many years, Lindorm has a unique and advanced technology system. Through compatibility with Cassandra CQL, it can meet the evolution needs of Cassandra users, allowing cloud Cassandra users and other Cassandra self-built users to enjoy the technical experience brought by Lindorm. For users of database Cassandra, they can enjoy the following technical advantages:

• Elasticity advantage

Lindorm provides flexible capabilities of any scale (1MB-100PB, 1tps- tens of millions of tps), and its core comes from Lindorm’s pooling design. The separate utilization of computing resources and storage resources is achieved through storage and computing separation technology, and then through the technology provided by Serverless, the on-demand charging is gradually improved, and multi-tenant isolation/current limit is realized, thereby achieving intelligent scheduling. However, Cassandra’s design based on local disks has a higher cost for expansion, and the use of instance charges may also be higher.

• Storage cost advantage

Lindorm builds the cloud native storage LindormStore, leverages low- cost storage such as OSS to reduce storage costs, and uses high- performance storage such as cloud disks for access acceleration. In addition, the Lindorm wide table engine layer will also be reduced through high compression encoding and cold and hot separation methods. User storage cost. The overall design of Cassandra is based on local disks, and it is difficult to leverage cloud infrastructure to reduce costs.

• Performance advantage

After years of polishing on the cloud and the group’s business, Lindorm is superior to the open source Cassandra system in terms of read/write performance, glitches, data compression rate and other indicators. Through Lindorm’s compatibility with Cassandra, more open source users and non-Cassandra users can enjoy To Lindorm’s low- cost, cost-effective technology dividend. The following figure shows the performance index comparison between Lindorm and open source Cassandra in a throughput scenario:

• Advantages of ecological tools

Lindorm provides Lindorm Tunnel Service (formerly BDS) and other components to open up data flow with the Alibaba Cloud ecosystem such as DB, ODPS, Kafka/MQ. At the same time, through in-depth cooperation with DataWorks, Blink, DLA and other teams, it ensures that users’ data can be written and read. Lindorm compatibility with Cassandra can avoid duplication of wheels for Cassandra, and it can also facilitate the users of cloud data Cassandra to quickly use Lindorm’s complete ecological components.

2.1.2. Benefit of the product

The Lindorm team maintains a number of database products on the cloud, and compatibility with Cassandra through Lindorm will bring great value gains to related database products.

• Cassandra’s huge base users

Under the operation of the commercial company Datastax and the open source community, Cassandra has accumulated heavyweight customers such as Apple and Huawei. Its ranking on DB-Engines is currently ranked first in the wide column database category. It is understood that DataStax’s revenue in 2019 exceeded 1 billion yuan. Using Lindorm’s performance, cost, and flexibility advantages, Lindorm can be compatible with Cassandra to easily absorb the market share of self-built Cassandra.

• Optimize the user experience and lower the threshold of use

By being compatible with Cassandra CQL, it is convenient for more users to use Lindorm conveniently. Cassandra’s rich multi-language client can also meet the needs of different language systems to access Lindorm platform.

• Technical resource coordination

Compatible with Cassandra CQL through Lindorm and open up the underlying technology, which can fully enhance the technical competitiveness of cloud Cassandra products, and by reusing the underlying technology of Lindorm, reduce the repetitive investment in core-level technology.

2.2. Feasibility Analysis

Through the realization of Lindorm and Cassandra engine and the comparison of their data model design, we found many common characteristics, and it is these common characteristics that replace the foundation of Lindorm compatible with Cassandra CQL.

• Data Model

Through in-depth analysis of the data models of Lindorm and Cassandra, we found that although there are subtle differences between the two models, they are essentially KV models, that is, a Key will be associated with its corresponding Value. In this way, when externally requesting the key, both Lindorm or Cassandra can return the correct Value required.

• LSM-Tree Based engine

In Lindorm’s multi-mode engine, the table engine and Cassandra are designed and implemented based on the LSM-Tree idea, so the read and write modules of the two will have a common WAL/Memtable/SSTable module to participate, and there will be similar compactions. The strategy is to merge asynchronous data. Theoretically, functional compatibility can be achieved on the read- write link of a stand-alone engine.

• Compatibility with Cassandra CQL

For CQL, Cassandra receives the Driver’s String query statement through the network protocol, and then parses and converts it into the required Statement. The entire process is relatively clear and independent. If the corresponding network protocol is implemented at the Lindorm layer for query statement analysis , Parse becomes the QP layer request required by Lindorm, then it is feasible to connect to the analysis of CQL language and the theoretical analysis of conversion.

• Compatibility with Cassandra’s other feature

The above illustrates the compatibility of CQL syntax. When a CQL is translated into a corresponding request, it will become an actual functional requirement, including index query, data type support, trigger, etc. Fortunately, Lindorm itself has the ability to implement the above functions and already has the above mentioned functions. The ability. Therefore, compatibility of the specific functions of CQL is also feasible.

3. The way Lindorm doing compatible with Cassandra CQL

3.1. design concept

• One data model with Multi-terminal:

Write in one place, available everywhere. Ensure the data compatibility of all CQL drivers, Phoenix drivers, HBase API, Lindorm API, Lindorm SQL and other access methods

• Protocol level compatibility:

Compatible with community driver, you can access Lindorm through Cassandra client without changing a line of driver code

• Cloud Native:

Design for cloud infrastructure to avoid duplication of investment

3.2. Technology Architecture

Cassanda CQL compatibility needs to be considered in two aspects: 1. Compatibility of CQL entry capabilities-Lindorm’s compatibility with CQL syntax; 2. Compatibility of functions exposed by Cassandra CQL.

Here are the implementation details:

• CQL protocol compatible

Because all Cassandra Drivers in the community must comply with the CQL binary protocol, the Lindorm server is fully compatible with the CQL binary protocol (V4 version) to ensure that the various drivers in the community can interact with the server normally. The initial interaction between the driver and the server layer will involve a series of protocol requests. A single CQL request is to directly send a String query statement to the server. Therefore, the compatibility of the binary protocol is mainly compatible with the initialization of the long connection and the single CQL request 2, and the Lindorm compatible Cassandra CQL mainly does the following work：

• Initial connection compatibility

• Driver’s interested event registration;

• Certified/non-certified/SSL operation registration;

• Get cluster topology status operation;

• Get the connection node schema information;

• DDL, DML, ACL and other operations

For other session operations other than initial connection initialization, the driver will not perform any processing before sending the request. The CQL will be sent to the server in the form of String. After the server receives the request, it will be converted to the corresponding one through the CQL Parser implemented by Lindorm Server. Structure the Statement and dispatch it to the engine for processing;

Through the above method, it is possible to achieve no need to modify any line of code of the community driver, and to be compatible with all drivers supported by the community. Users can directly use the community multi-language driver to access Lindorm. At the level of CQL syntax analysis, Lindorm is compatible with CQL and directly uses the Cassandra 3.x version of the Cql.g definition file for compilation processing. CQL parsing mainly uses antrl3 for basic parse, and performs lindorm on the existing CQL syntax functions. The specific technical details of grammar adaptation are not outlined in detail in this section.

• Query request metadata maintenance

Most of the drivers in the original CQL binary protocol will initialize the connection to obtain the entire topology status of the cluster, and then connection related nodes based on the topology status. For specific query, select the server with lower load according to the topology state to query. During the investigation, we found that this model has several main problems:

• The network connection is complicated, the client must communicate with all Server nodes, which is difficult to achieve in cross-VPC scenarios

• When there are too many clients, there will be a link storm on the server. In the serverless scenario, the client size of a single cluster may reach 10w+. The direct connection means that the number of tcp links for a single Lindorm Server will exceed 10w+.

• The original Cassandra implementation relies on Gossiper, and there is a delay in information transmission

For this reason, Lindorm has made two major transformations. The first is to integrate all service discovery and Table source information acquisition into Lindorm’s internal processing, instead of relying on a separate Gossiper module like Cassandra for acquisition. The second is the front-end hanging ALB method, through ALB for connection distribution and load management, a client only has one long link. This method can more easily support complex network environments, has better security, and is easier to integrate with Lindorm’s cloud management and control.

• Feature support

By being compatible with the CQL binary protocol and implementing CQL Parser, Lindorm can seamlessly connect to the user’s CQL requests, such as DDL/DML/ACL operations; for the functional level, Lindorm natively supports secondary indexes and has the ability to search, which can be released through CQL spit Related capabilities.

4. Competitive product comparison

4.1. Overall introduction

In addition to native Cassandra, AWS KeySpace, ScyllaDB, Huawei Cloud GaussDB (for Cassandra), etc. are all compatible with or translated Cassandra.

AWS Keyspace is a Cassandra hosting service in a serverless scenario based on the AWS public cloud platform. It can be seen from external documents that it is compatible with most of the functions of Cassandra and selectively implements DDL&&DML, but the original Cassandra ACL is completely abandoned. , Switch to AWS’ own IAM method.

ScyllaDB is the so-called C++ version of Cassandra, which focuses on performance advantages and relatively complete compatibility. As the old saying goes, those who learn from me live, and those who look like me die. Although ScyllaDB has a design that is very similar to Cassandra, good compatibility and high performance on paper, it has not been well recognized by the market. ScyllaDB still inherits the shortcomings of Cassandra’s lack of flexibility and weak consistency guarantee. The concept of Quorum Read looks good, and it also brings the resource overhead of row-level repair to compensate for the consistency of the copy.

In contrast, Lindorm is compatible with Cassandra, based on the CQL language implemented by Lindorm kernel and compatible Cassandra functions, it can achieve a high degree of compatibility with Cassandra and meet all common needs of users, mainly including DDL&&DML&&ACL&&Data Type. The lower level Lindorm is a new upgrade with a unified storage, unified query, and multi-mode engine architecture. With the help of cloud infrastructure dividends, it focuses on the advantages of cloud native flexibility, multi-mode integrated processing, extreme cost performance, and enterprise-level stability. Avoid the defects of Cassandra's native architecture design.

4.2. Comparison list

The following Cassandra function support comparisons are based on Lindorm2.2, Cassandra 3.11.4, AWS Keyspace (no version information exposed), ScyllaDB (open source version 4.1.5), including functions in internal testing. Lindorm is compatible with Cassandra CQL to CQL Compatibility can reach more than 90%.

• API Support

• DATA TYPE

• Product form

5. Conclusion

By being compatible with Cassandra CQL, Lindorm users can use various multi-language Drivers to access Lindorm in CQL mode, the experience will become more friendly and smooth, and reduce the user’s learning costs, and facilitate the low-cost migration of SQL users. For Cassandra users, Lindorm can be migrated seamlessly, using the cold and hot separation technology provided by Lindorm, and relying on low-cost cloud native LindormStore to reduce database costs. For other users, whether they are using Lindorm cluster version or Lindorm serverless version, compatibility with Cassandra will provide a new database option.

Lindorm is compatible with Cassandra CQL and uses CQL as the main entrance of Lindorm to handle read and write requests in the TP scenario of NOSQL users. In addition, CQL can also release various black technologies that Lindorm has accumulated in the past, such as intelligent cold and heat separation, and efficient retrieval capabilities.

.Downward, Lindorm will continue to absorb outstanding and extensive user-demand function points, and gather the best of hundreds of families.

6. Future work

In the future, Lindorm’s main work for compatibility with Cassandra CQL will focus on the purpose of “downward compatibility and upward expansion”. Upward, it must be able to support CQL and other excellent function points of query languages, such as aggregation operations, User Defined Function/Type, etc. Downward, Lindorm’s own capabilities should be better exposed. For example, core capabilities such as Serverless, LSearch, and cold and hot separation will be exposed through CQL.

Alibaba Tech

First hand and in-depth information about Alibaba’s latest technology → Facebook: “Alibaba Tech”. Twitter: “AlibabaTech”.