(Glue-2105) Q: Do you support Azure Data Lake as a target?

Owned by SNP
May 13, 2021
1 min read

Yes, Datavard Glue does support ADLS.


Please note that data extraction to ADLS may fail with the following error:

HTTP Error. Status: '404'. Exception: ''. Reason: 'Not Found'.
File '....csv' creation failed.

The reason is that the transported package is too big for ADLS.

For performance reasons, the SDK is using a fixed Content-Length request body. If request body length exceeds a default limit, ADLS returns 404.13 status code. Please refer to Request Limits document for more info, 

The limit is 30000000 bytes or about 28.6MB. Please try to limit your stream sizes to < 28MB (ideally 4MB chunks are the best size in terms of performance and reliability).


The best practice is to use a smaller package size for the data transfer (e.g. 10.000 – 20.000 rows) to ADLS to match recommended 4MB chunks (link).