Cloud-Driven Data Modernization

There are many reasons to look at modernizing your data. Ensuring that the data you hold is secure and complies with regulations, to enable a future design strategy or to make best use of the tools and technologies now available. As Deloitte found back in 2019, the second highest rated driver for data modernization is moving to the cloud.

There’s an inescapable logic to data modernization, given that the cloud offers you new ways to deploy, distribute, and access your data. If you want to exploit those capabilities, your applications and data storage will change. There couldn’t be a better time to modernize your data models to match their new environments.

But what does modernization actually mean in practice? At Studio 3T, we’ve spent our time focused on the transition of data from legacy SQL databases to MongoDB document databases. That in itself is a massive shift in how data is managed. Consider SQL and relational databases. The foundations of those technologies were laid down when memory was tight, disc access took extremely long times and CPU power was precious.

SEE ALSO: “Cloud moved decision-making responsibility out of finance and into engineering”

Those constraints led the designers down a path of slicing data into ever smaller tables and then relying on a network of references to bind the data together. To query it, SQL emerged as a language which could turn a question into a plan to look at those smaller tables and pull the results together by joining tables through those references.

Take away the constraints and there’s no essential need to normalize data like that. With modern RAM, disk and CPUs, holding all of the related data for a client is not only possible, it can be the optimal path to rapid data access.

With all relevant data in the document, a client’s records are always, literally, to hand when the client’s data is retrieved. There’s no secondary queries or joins to build out that information. It’s all in the document. That is what underlies a document database like MongoDB, the ability to bring all the data together in one structured document. Underneath, the database handles indexing and optimizing overall access, rather than expecting data to be arranged to be easily indexed.

And that’s also where the challenges lie when migrating to MongoDB from SQL databases. Working out how to create a complete document that encapsulates all the various tables and resolving your ideal data model can take a while. And then moving the data from SQL to MongoDB efficiently. Think of them as the two elements of modernizing data.

Studio 3T’s SQL migration tooling is built to make the latter, practical task of extracting and restructuring your data into MongoDB documents as easy as possible. It lets you select tables, map them into documents and take referenced tables and turn them into arrays or objects as appropriate. What is appropriate though?

That’s where tools like Hackolade come in, to help the design process of collapsing this referential model down into a focussed and clean document data model. Hackolade is a data model design bench where you can take your SQL databases DDL and turn that into a one-to-one model for MongoDB. Then, outside of MongoDB, you can use Hackolade’s design tools to suggest denormalizations for tables, folding them into the one document or keeping some data outside the document. Hackolade gives you a workbench to mould and document design changes.

The division between tools reflects the two components of data modernization that go to make up a successful modernization. First, the strategic element, encompassed by Hackolade, which comes down to planning how your new data models will look, how they will take advantage of data localisation, and how the information will be arranged within the document.

There is then the tactical element of modernization, which involves moving the actual data from SQL databases to MongoDB quickly and efficiently. Being a separate element has advantages in terms of repeatability too.

SEE ALSO: The Risks of Poor Enterprise Cloud Management and How to Avoid Them

The strategy can be refined as insights are gained into what works best for applications, models can be tuned and then the tactical migration repeated to update the migrated data to those tuned models.

The approach shifts what is usually seen as a big bang change to a more iterative pathway, allowing developers to explore the best way to cluster data into documents. This can, hand in hand with new development, lead to a more fertile, greener field to grow your cloud applications on.

The move to the cloud is still ongoing around the world. In our MongoDB Trends report for 2020, only 63% of EMEA companies surveyed had some or all of their data in the cloud, trailing the 76% of APAC companies who had moved data to the cloud. For all these organizations, the move to the cloud goes hand in hand with an opportunity to modernize their data, and that’s an opportunity that should not be missed.

The post Cloud-Driven Data Modernization appeared first on JAXenter.