Migrate Cassandra to ScyllaDB with none downtime

Businesses need many things to run their businesses successfully. One in all the important thing things is the info warehouse used to store data utilized by applications and various analytics platforms. For good business growth, firms need reliable databases and in response to their technology budget and expertise, they select databases. Working and consulting with the enterprise, they launched Cassandra to support NoSQL data warehousing operations. Cassandra was working rather well, but now that the corporate had grown, they needed help with the NoSQL store. They experienced cascading delays from the Cassandra Hot partition as traffic increased with events and campaigns. Garbage collection was also becoming a bottleneck because it heavily impacted database performance, leading to poor application performance. One other essential reason was the will to avoid database management and search for an authority company that would manage it without significant changes to the applying.

Why ScyllaDB?

While exploring various database solutions, ScyllaDB caught our attention. We were interested by the answer and did a number of proof of concept in ScyllaDB. In the long run, we decided this may be the appropriate alternative for the environment and scale. Listed below are a number of the principal reasons for our decision:

• We’ve seen a major 4x increase in performance for a similar size of information we store in Cassandra.
• Our compute infrastructure was reduced so we didn’t must run as many nodes because the Cassandra cluster. With a smaller ScyllaDB cluster, we achieved high performance.
• We used the ScyllaDB manager component to assist us with database administration work resembling incremental backups and optimization of compute infrastructure.
• One essential reason was the scalability of ScyllaDB. We tested our application by scaling the ScyllaDB cluster horizontally and vertically.
• ScyllaDB is written within the C programming language, so we didn’t need to worry about garbage collection and performance impacts.
• Since we were already using Cassandra, we didn’t must make any changes at the driving force or schema level to migrate to a ScyllaDB cluster.
• Our overall RTO and RPO have improved due to data warehouse performance and reliability. For this purpose, we also performed several trial exercises.

(Good read: Migrating Cassandra to ScyllaDB)

Migration strategy and flow

There are numerous migration strategies to migrate data from a Cassandra cluster to a ScyllaDB cluster. There are also policy documents and flows created by the ScyllaDB team. Still, here we would really like to speak about a way that helped us move petabytes of information from Cassandra to ScyllaDB with none application downtime. It was a quite simple migration for us (that is after we selected the approach). We’ll start listing the steps we took to migrate the database:

1. Creating the identical table schema from Apache Cassandra to ScyllaDB (It is crucial to grasp that the migration shall be table-based. As an alternative of the entire database, list the applications and table mapping. Then migrate each application regarding its table).

 cqlsh "-e DESC SCHEMA" > cass_schema.cql cqlsh --file 'adjusted_cass_schema.cql'

cluster := gocql.NewCluster("")
cluster.Keyspace = ""
session, err := cluster.CreateSession()

if err != nil {
   panic(err)

}

defer session.Close()

query := session.Query(`INSERT INTO your_table (column1, column2) VALUES (?, ?)`, value1, value2)

scyllaSession, err := connectToScyllaDB()

if err != nil {

                log.Fatal(err)

}

defer scyllaSession.Close()

dataFromScylla, err := readFromScyllaDB(scyllaSession)

dataFromScylla, err := readFromScyllaDB(scyllaSession)

if err != nil {

                log.Println("Error reading from ScyllaDB:", err)
                // If not present in ScyllaDB, try reading from Cassandra
                cassandraSession, err := connectToCassandra()

                if err != nil {
                                log.Fatal(err)
                }

                defer cassandraSession.Close()

                dataFromCassandra, err := readFromCassandra(cassandraSession)

                if err != nil {
                                log.Fatal("Error reading from Cassandra:", err)
                } else {
                                fmt.Println("Data read from Cassandra:", dataFromCassandra)
                }

} else {
                fmt.Println("Data read from ScyllaDB:", dataFromScylla)
}

dsbulk unload -h  -k keyspace_name -t table_name -url table_name -cl LOCAL_QUORUM --timestamp --ttl

dsbulk load -h  -k keyspace_name -t table_name -url table_name -cl LOCAL_QUORUM --timestamp --ttl