(SM-2408) Azure Data Lake Gen2 via JDBC

Owned by SNP
Aug 20, 2024
4 min read

This article describes a process required to establish a connection from SAP to Azure Data Lake Storage Gen2.

Azure storage configuration

This section describes the steps needed for the preparation of the landing area on the Azure side.

Storage Account and ADLS container

To create the MS Azure storage account, follow the official documentation Create a storage account to use with Azure Data Lake Storage Gen2.

Landing area (container/directory)

Create a container or a directory within the container where all files extracted from the SAP system will be stored.
In Microsoft Azure Storage Explorer, select the container/directory that will be used and set appropriate permissions depending on the authentication method chosen.

Authentication

Authentication to ADLS can be delivered by two methods:

  • SAS Token: Recommended for Proof of Concepts, faster setup, but provides fewer security management options.

  • OAuth 2.0: Recommended for production deployment, fully manageable, but requires application registration and detailed permission setup.

SAS Token

If you use the SAS token, please ignore the OAuth 2.0 Authentication steps. Similarly, you can skip steps related to the OAuth profile in the SAP configuration section.

To generate the SAS token, go to the Azure portal.

The SAS token should grant permission to Create, Read, Write, Delete and List.
Click Generate SAS token and URL.
Copy Blob SAS token string and store it for configuration on the SAP side.

OAuth 2.0 Authentication

OAuth 2.0 required application registration. To create a new registration, follow these steps:

  1. Go to Azure Active Directory > App registrations > New application registration.

  1. Fill in the required fields and click Create.

  1. Write down Application ID and Directory ID, as it will be required later during the Storage management configuration.

  1. Click Certificates & secrets and generate a New client secret. Write down the secret, which will be used later during the configuration.

  1. Allow access to the registered application (Client ID) to the landing area.

  1. The registered application also needs to execute permission on ALL parent directories and filesystem (enabling directory structure traversal).

SAP system configuration

After preparation is complete on the Azure side, fill in the required information on the SAP side to establish a connection.

Java Connector Setup

First Java connector needs to be setup. In detail guide can be found here (SM-2408) Java Connector Setup.

Storage Management

In this step, the actual connection to the landing area is created.

  1. Execute transaction /DVD/SM_SETUP.

  2. Switch to Edit Mode and click New Storage.

  3. Choose storage type ADLS_JAVA and specify Storage ID.

  4. Specify the following parameters:

    1. Java connector RFC: RFC that points to running JCo Instance.

    2. Target host: URL of the ADLS host.

    3. Container: Name of the container on ADLS

    4. Path prefix (optional): /<path to landing area> within ADLS container.

    5. HTTP Call Repeat: Specify the number of times the HTTP request will be retried in case of failure (0 = no repetition).

    6. Repeat Delay (seconds): Seconds between repetitions - if not filled, the default value will be set to 3 seconds (instead of 0).

The bottom part of the storage connection configuration depends on the authentication method chosen.

a) If a SAS token is used, either insert it directly to the respective field, or you can insert it into the field next to Hash button and by clicking Hash, convert it into a hashed string to prevent plain text visibility.

SAS token value alerts:

  • Remove a question mark ? character at the beginning of the generated SAS token.

b) If the OAuth authentication method is chosen, fill in the OAuth profile with information recorded during application registration:

OAuth 2.0 profile: The ID of the profile.
Client ID: Application ID created in the section Creating Application Registration.
Client secret: Key created in the section Creating Application Registration.
Resource: Static address https://storage.azure.com/.
Tenant: Part of the URL identifier is described in the section Tenant.

Confirm storage creation by saving the information (Ctrl+S).

To check the connection click Check button.

 

Use compute hash to calculate the hash of data.