Project Circe June Update

Scylla Open Source 4.5 Coming Soon!

We’re on the verge of releasing Scylla Open Source 4.5 (following RC2, which went out in early June). Let’s have a look at the new features and capabilities you can look forward to in the coming release.

Load and Stream SSTables

This feature extends nodetool refresh to allow loading arbitrary SSTables. It will help make restorations and migrations much easier. You can take an SSTable from a cluster and place it on any node in the new cluster. When you trigger the load and stream process, it will distribute and stream the data across the nodes in the new cluster. Previously, one had to carefully place the SSTable within every node that owned key ranges that belong to it. Today, this feature does the job for you.

Project Alternator

We’re making improvements to our DynamoDB-compatible API in a number of ways:

  • The sstable loader utility will work with Alternator tables, beginning with 4.5.
  • Cross-Origin Resource Sharing (CORS) will allow client browsers to access the database via JavaScript, avoiding a middle tier.
  • You will be able to limit maximum concurrency, with queries exceeding that concurrency returning a RequestLimitExceeded error.
  • Nested attribute paths will allow the modification of just an object’s attributes instead of the entire object.
  • Slow query logging will allow you to find queries that exceed a threshold and log them to system_traces.node_slow_log.
  • Support for attribute paths in ConditionExpression, FilterExpression, and ProjectionExpression.

Raft

Raft implementation in Scylla is a core deliverable of Project Circe. While the changes made to the Scylla infrastructure will have no visible effect yet, we’re adding the major building blocks upon which a number of future capabilities will be delivered.

  • Schema Tables on Shard 0 — To date Scylla stored the database schema in a set of tables, sharded across all cores like ordinary user tables. With Scylla 4.5, this schema data will be maintained by shard 0 alone. This is the first step to letting Raft manage them.
  • Log Data — Raft will now be able to store its log data in a system table, implemented in a modular fashion.
  • Joint Consensus — Now merged, this provides the ability to change a Raft group from one set of nodes to another, which is a requisite for cluster topology changes and data migrations to different nodes.
  • Additional changes to the Raft implementation provide support for non-voting nodes, per-server timers, and leader step-down.

Change Data Capture (CDC)

We are thrilled that our users are eagerly looking for ways to leverage the new CDC capabilities in Scylla. (Have a look at our recent webinar with Confluent on how to build event streaming architectures using CDC with Kafka.) This month we optimized CDC enablement on large Scylla clusters with many partitions and streams: first was limiting the number of streams (though this does incur some loss of efficiency), and, secondly, we adopted a new format that uses partitions and clustering rows.

Other June Releases

Beyond Scylla Open Source, we also provided a new update to our Scylla Enterprise 2021 release, as well as updates to our supporting applications and utilities:

Velocity of Software Delivery

We often hear from Scylla users that the velocity of software delivery matters to them when deciding what infrastructure components to implement in their ecosystems. Already in the first half of this year we delivered Scylla Open Source 4.3 and 4.4, plus early in the second half we will deliver 4.5. This strong, steady release cadence allows us to add new capabilities while also allowing us to fix bugs at a rapid and regular clip.

  • 28 authors pushed
  • 383 commits for the month
  • 1,487 files were changed
  • 84 authors pushed
  • 393 commits for the month
  • 1,441 files were changed
  • 52 authors pushed
  • 122 commits for the month
  • 737 files were changed
  • 20 authors pushed
  • 49 commits for the month
  • 140 files were changed
  • June 06 — featuring a new process for making Docker images
  • June 13 — which enables off-strategy compaction for bootstrap and replace operations
  • June 20 — changes to how range tombstones are internally represented
  • June 27 — making the bootstrap process more robust

Sign Up for Scylla University LIVE!

We look forward to seeing you at Scylla University LIVE for our Summer Session. This is an event you won’t want to miss. Besides the tracks about Scylla operations and development, we’re also going to have sessions devoted to hooking up Scylla to the rest of your big data architecture, including integrating it with Apache Spark and Apache Kafka.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
ScyllaDB

ScyllaDB

The monstrously fast and scalable NoSQL database.