Home Business Intelligence Introducing Final Mile ETL: One Device for Higher Information Transformation

Introducing Final Mile ETL: One Device for Higher Information Transformation

Introducing Final Mile ETL: One Device for Higher Information Transformation


Think about you join a database to analytics and the information is just not within the form you want to have it. For instance, coordinates ought to be separated by latitude and longitude, some values are in a distinct format or sort, and a few tables can also have a dangerous design. Usually, you would want to spend so much of time context-switching between the ELT/ETL pipeline and analytics, the place, within the transformation (T) section you would want to edit code that transforms information from the appliance form to the analytics form. This expertise is form of regular in our trade however does it need to be? For that reason, we’re introducing Final Mile ETL!  With it, you are able to do every part in a single software which considerably improves the power to iterate on and the pace of growth, customization, and safety. You could now ask how? Nicely, sufficient phrases, it’s time for an instance. Let’s deep dive into it!

What do I imply by “exploratory analytics”? Let’s say, now we have simply three tables with information, and we want to discover its worth or study some information based mostly on this information. It implies that as a substitute of 1 concrete aim to attain, we are going to primarily attempt to discover some worth on this information! With outlined exploratory analytics, listed below are three tables in a database (Airports, Nation checklist, and GDP — Gross Home Product):

Tables in a database
Tables in a database

You possibly can see that the coordinates are in a single column known as coordinates as a substitute of latitude and longitude, or worth in GDP the desk is textual content quite than numerics. We are able to tackle these points with the assistance of Final Mile ETL contained in the analytics.

Let’s join the database to the analytics (in case you are not conversant in GoodData, I encourage you to test the documentation). The result’s the next:

The result of connected database to analytics (GoodData)

What you may see within the picture above are datasets. We are able to convert a dataset to a so-called SQL dataset:

Converting dataset to SQL dataset

The SQL dataset offers us the chance to write down SQL queries which might be executed straight within the database. Let’s simply test what forms of airports now we have within the database:

SQL query from analytics

Plainly the Airports desk comprises a number of forms of airports comparable to heliports, and even closed airports. Let’s say that I’m simply within the medium and huge airports — it’s not an issue in any respect. I don’t have to go to the ELT/ETL pipeline, I merely create a brand new SQL dataset known as Airports remodeled straight within the analytics, and I can try this with the next SQL code:

New dataset Airports transformed

You possibly can see that I can do it with fairly a easy SQL question and the result’s the next: