Posts

How to parse XML response from REST API in Azure Data Factory

-
Image
Parse XML response from REST API in Azure Data Factory Well Azure ecosystem is evolving day by day but there are still many features that are missing or in the pipeline of getting them in the latest release, One such feature is parsing of XML response from API in Azure Data Factory which is yet not available in ADF as of today.  These days all latest REST API response type is JSON, but there are many legacy API which are still using XML as response type. As a Data/ETL Engineer you might have come across such scenarios in On Premises system where we could have used C#,SSIS, Informatica or similar ETL or programming methodologies to retrieve data a XML response type API and load them into target application/system. Now as we have moved PAAS & SAAS system where each service or application is hosted in cloud we would need to do similar activity of reading xml response from API and process it. Lately ADF accepts XML file as a source file type and using copy activity we can load the ...
Post a Comment
Read more

Power BI Usage Metrics with the help of Power Automate - PBI Analytics

-
Image
Power BI Usage Metrics Analytics   Metrics play vital role in organization as it gives real-time and forecasted analysis for business players to take decision. These decision help the organization to grow and prosper in the market. Most of these metrics are build using BI tools available in market such as Power BI, Tableau, Spotfire ..etc. Today I am going to take Power BI as an example and discuss further on it. Typically any  organization   will have 100's of reports created for various analytical analysis which business is looking for and they will be adding 100 more as and when new request comes. And there will many analyst and business stakeholders how will be using them proactively and some rarely.  What about user analytics of these users and report analytics of all the reports that are being used? These analytics will help the IT team to priorities the current user base, priorities the reports eco system and also will give insigh...
1 comment
Read more

Microsoft Power Automate -- Timeout Limitation of 120 Sec in SQL Actions -- Solution

-
Image
Problem : If you have used Microsoft Power Automate for any of your process automation task and if it includes any SQL operations you might have came across a limitation within any sql action for timeout of 120 seconds. So that means any sql operation that is part of your process automation has to complete within 120 seconds. Ah isn't this strange? there might be some sql operation which would take more than 120 seconds like creating a backup of existing tables on request or any big DML task which would run for hours together. So in existing sql task available in Power Automate this can't be achieved. Below is Microsoft link for current limitation  https://docs.microsoft.com/en-us/power-automate/limits-and-config Solution : Well we have got many other actions triggers available in power automate to trigger or automate a process, I am going to use DevOps action to create a release and in Azure DevOps will keep the SQL scripts which needs longer execution time. This will simu...
Post a Comment
Read more