This guide describes a process required to establish a connection from Datavard Storage Management to Azure Data Lake Storage Gen2. 

Prerequisites

SAP requirements

• SAP NetWeaver 7.01 SP-level 015
• HTTPS service enabled
• SSFLIB Version 1.850.40 ; CommonCryptoLib (SAPCRYPTOLIB) Version 8.5.20 (+MT)

Azure Data Lake Gen2 requirements

• Azure Data Lake Storage Gen2 storage account
• Details needed for connection to ADLS Gen2 based on selected authentification method:
  • 1581023257oAuth2.0 - "CLIENT_ID", "CLIENT_SECRET" and "TENANT".
  • SAS Token - only Shared Access Signature Token itself
• Existing filesystem in the storage account

Azure storage configuration

You should perform these steps before the implementation.

Create a storage account

...

1. Click Create a resource 
2. Search for Template deployment, then click create
   

Application registration

Anchor
Creating Application Registration Creating Application Registration

Application registration is used for authentication to ADLS. Application registration is only required if AAD (OAuth) authentication is used. An alternative is to use SAS token which is easier.

To create a new registration of an application, follow these steps:

1. Go to Azure Active Directory > App registrations > New application registration

Image Modified

2. Fill the required fields and Click Create.

3. Write down Application ID and Directory ID, as it will be required later during the Storage management configuration.

4. Click Certificates & secrets and generate a New client secret. Write down the secret, as it will be used later during the configuration.

Creating a landing folder 

Create a folder where all the new files extracted from the SAP system will be located and set the correct permissions for this folder.

1. Go to your Microsoft Azure Storage explorer and identify a folder that will be used with the storage.
2. Make sure that the App registration that was created previously has rwx access and default privileges checked for the directory 
   
3. The App registration also needs privileges to execute on ALL parent directories and filesystem (enabling directory structure traversal)
   
   

SAP system configuration

After preparation is complete on the Azure side, fill in the required information on the SAP side to establish a connection.

STRUST

The root certificate authority of Microsoft needs to be loaded via transaction STRUST to establish a secure SSL connection. 

1. With the help of your internet browser, copy the CA public certificate into the file, as shown in the figure below.

Image Modified

2. In STRUST, import this certificate into SSL Client (Anonymous) PSE.

Image Modified

3. Go to the transaction SMICM and restart the ICM services as shown on in the figure.

Image Modified

Storage RFC

Create RFC type G for Azure Data Lake Gen2 primary endpoint.

• Set Target Host to your ADLS address (e.g. dvdadls2.dfs.core.windows.net) and Path Prefix to /<filesystem>/<Path to landing folder>.
  

• Set SSL to "Active" and Certificate list to "ANONYM SSL Client (Anonymous)".
• Set HTTP Version to 1.1 and Compression to Active

...

oAuth 2.0

Anchor
oAuth 2.0 oAuth 2.0

Authentication RFC

Start with the creation of an RFC of the type G for Microsoft for the Microsoft Active directory. This RFC represents a connection to the authority server that grants an authentication token for ADLS. 

• Set Target Host to: login.microsoftonline.com. 
• Set SSL to "Active" and Certificate list to "ANONYM SSL Client (Anonymous)".

  
  Image Modified

Authentication profile

The authentication profile contains login information, which you should create in the table /DVD/OAUTH_CONF. 

OAUTH_PROFILE you may choose any value to identify a profile used for the authentication

CLIENT_ID is an Application ID created in the section 1581023257 Creating Application Registration.

CLIENT_SECRET is a key created in the section 1581023257 Creating Application Registration. Can be hashed by the report /DVD/XOR_GEN.

GRANT_TYPE is the fixed value "client_credentials"

RESOURCE is the fixed value "https://storage.azure.com/"

TENANT is an identifier described in the section Tenant

URL is left blank

Linking authentication profile

The next step is to link the authentication profile with RFCs created in the table /DVD/HDP_AUT_OA2.

Setting the authentication method

The authentication method needs to be set to OAUTH2.0 in the table /DVD/HDP_CUS_C.

Creating storage in Datavard Storage Management

After the configuration is complete, you need to define the storage that serves as a target for the extraction.

1. Go to the transaction /DVD/SM_SETUP.
2. Switch to Edit Mode and click New Storage.
3. Create new storage of the type ADLS_GEN2 and fill the RFC destination.

4. Specify the following parameters:

   • HTTP RFC Destination – Storage RFC destination created previously
   • Use OAuth token - Mark as checked to use OAuth
   • HTTP call repeat - Number of retries if request to ADLS Gen2 is not successful.
   • Repeat delay (seconds) - Seconds between repetitions - if not filled (0), default value 3 is used.

   

SAS Token

Anchor
SAS Token SAS Token

...

If you decide to use SAS Token, you can skip all steps in Azure Storage Configuration besides creating the directory and you can also skip steps in the OAuth section.

To generate the SAS token, go to the Azure portal (see the screenshot below for more information).

Creating

...

storage in Datavard Storage Management

After the configuration is complete, you need to define the storage that serves as a target for the extraction.

1. Go to the transaction /DVD/SM_SETUP.
2. Click on Create.
3. Create new storage of the type ADLS_GEN2.

4. Specify the following parameters:

   • HTTP RFC Destination – Storage RFC destination created previously
   • HTTP call repeat - Number of retries if request to ADLS Gen2 is not successful.
   • Repeat delay (seconds) - Seconds between repetitions - if not filled (0), default value 3 is used.
   • Authentication - select "Use SAS token"
     

   • SAS token – insert Shared Access Signature Token generated in Azure Portal, and click [Hash] to prevent SAS token visibility n.

     Note
     SAS token value alerts: Remove a question mark '?' character at the beginning of the generated SAS token. If a value ends with '%3D', replace it with the equals sign '='.

     
     

...

Failover (Read-only) storage enables to utilize Azure Storage Redundancy, e.g. read-access geo-redundant storage (RA-GRS) or read-access geo-zone-redundant storage (RA-GZRS), to improve high availability/disaster recovery. For more information about Azure replication strategies, please refer to official Microsoft documentation: https://docs.microsoft.com/en-us/azure/storage/common/storage-redundancy.

When the primary storage is unavailable, i.e. connection check fails, the application will automatically switch to Failover storage for data read from another data center in a secondary region. Follow the procedure below to enable it on the SAP side.

...

After that, additional parameters appear in Storage Management settings (Tcode /DVD/SM_SETUP), where you can specify HTTP Destination to your Failover storage and SAS token.

Failover storage RFC

Create RFC type G for Azure Data Lake Gen2 secondary endpoint.

• Set Target Host to your secondary ADLS address - it is the same as for primary storage, but appends the suffix –secondary (e.g. dvdadls2-secondary.dfs.core.windows.net) and Path Prefix to /<filesystem>/<Path to landing folder>.
  

• Set SSL to "Active" and Certificate list to "ANONYM".
• Set HTTP Version to 1.1 and Compression to Active

...