Microsoft Azure’s data storage, management and analytics services have rolled up their sleeves during this first half of the year. With the announcement of Azure Data Explorer’s general availability, the updates to the data warehouse including the incorporation of Azure Data Lake Storage Gen2, and the preliminary version of Azure Data Factory Mapping Data Flow, Microsoft’s cloud-based big data service aims to increase its efficiency and bolster its security. Read on to learn all about the latest Azure Data Services improvements.
Updates to the Azure Data Lake data warehouse
Azure Data Lake Storage (ADLS) is Microsoft’s cloud data storage service for big data analytics. The latest version of this data warehouse, ADLS Gen2, includes several new features:
- Increased compatibility with Apache environments through the incorporation of a driver (Azure Blob File System, or ABFS) that is officially a part of Apache Hadoop and Spark.
- New hierarchical name space for increasing the performance of data warehouse analytics, considerably reducing database overload when processing big data.
- Incorporation of security features such as TLS 1.2 encryption, specific firewalls for storage accounts, and integrating virtual networks.
Just like the previous version, Azure Data Lake Storage Gen2 incorporates other services such as Databricks, HDInsight, Data Factory, SQL Data Warehouse and Power BI.
Improvements to Azure Data Factory
Since mid-February, the Microsoft Azure cloud data integration and automation service has been equipped with the preliminary version of a new tool: Azure Data Factory Mapping Data Flow. With it, Azure Data Factory turns data transformation into a visual experience that doesn’t require programming knowledge, allowing a larger number of users to access the business intelligence.
By integrating Mapping Data Flow into Azure Data Factory, users can visually design, create and manage data transformation processes without needing to know Spark. Furthermore, the new tool counts with an interactive debugger that executes, triggers and supervises ETL projects.
General availability of Azure Data Explorer
For the first quarter of the year, Azure Data Explorer (ADX), a real-time streaming quick data analytics service, was only available in certain regions. On February 7, 2019, ADX, a tool that can be used for browsing billions of records in under a second, became available in 41 Azure regions all over the world, including European countries.
Data Explorer counts with two different servers, Engine and Data Management, implemented in virtual Azure machines. Data Management intakes the unprocessed data, and also sees to data preparation, backpressure and errors. On the other hand, Engine processes inbound data that hasn’t been processed, combining automatic scalability and data partitioning to achieve higher speeds.
Images | Pixabay, Pexels/ Vitaly Vlasov
