-
Comparing MySQL to Vertica Replication under MemCloud, AWS and Bare Metal
Back in December, I did a detailed analysis for getting data into Vertica from MySQL using Tungsten Replicator, all within the Kodiak MemCloud. I got some good numbers towards the end – 1.9 million rows/minute into Vertica. I did this using a standard replicator deployment, plus some tweaks to the Vertica environment. In particular: Integer…
-
Analytical Replication Performance from MySQL to Vertica on MemCloud
I’ve recently been trying to improve the performance of the Vertica replicator, particularly in the form of the of the new single schema replication. We’ve done a lot in the new Tungsten Replicator 5.3.0 release to improve (and ultimately support) the new single schema model. As part of that, I’ve also been personally looking to…
-
Kafka Replication from MySQL and Oracle
Hello again everybody. Well, I promised it a couple of weeks ago, and I’m sorry it has been so long (I’ve been working on other fun stuff in addition to this). But I’m pleased to say that we now have a fully working applier that takes data from an incoming THL stream, whether that is Oracle…
-
Data Migration: Handling Data Types
Handling the structure, and deciding which fields, data types, or data points, relate to which fields or target elements within the destination database are one thing. Normally you can make an intelligent decision about the information that is being transferred and how that structural information should be handled. The actual data can be a completely…
-
Data Migration: Mapping the Data
When moving the data between different databases the primary considering is what that’s going to look like so that it can be used in the target environment. Later chapters are going to dig deeper into this topic, but let’s fly over some of the key considerations here. Mapping Columns to Tables If we were replicating…
-
Data Migration: Moving the Actual Data
There are two key elements to the exchange of any information between databases. One is the data structure used for the exchange, and the other is the transformation required to reach those structures. Some of these are driven by the source database, others by the target database. For example, when moving data from RDBMS to…
-
Data Migration: Methods
Throughout the series we will examine four distinct methods for moving and sharing information stored in databases, and each have their specific meanings, techniques and tricks to get the best out of the process. The four methods are: Moving – this refers to the process of moving data to another database for a simple export…
-
Extending the Tungsten Replicator Core JS Filter Functionality
Tungsten Replicator has a really cool feature in that we can filter data as it goes past on the wire. The replicator itself is written entirely in Java and writing filters for it is not as straightforward as it looks, which is why the much better feature is just to use the JavaScript mechanism and…
-
Data Migration: Database Terms and Structures
In the previous post we looked at a number of different database types and solutions, and it should be clear that there are a huge range of different terms for the different entities that make up the database structure. All the different entities fit into one of four categories, and they have significance because when moving…
-
How to Buffer posts+hashtags from your Blog using Zapier
I try to automate as much my life as possible, particularly when it comes to computers. I’ve been using the automated ‘Social Sharing’ on WordPress.com (and indeed, my blogs in general) for years. However, I’m also a keen Buffer user and WordPress.com does not offer a Buffer connection. Because I also use Buffer to handle…