(SP32) Data Archiving Process

Create Data Archiving Process

For InfoProviders (InfoCubes and DSOs) a standard Data Archiving Process has to be created to archive the data first. 

As of SAP BW 7.30, Advanced DataStore Object archiving is supported. This feature does not apply to Mass archiving yet. Almost every ADSO type is supported. NLS reporting is available for the Cluster, File, MSSQL, Sybase IQ, MaxDB, DB2, and Oracle databases.


For PSAs and Change Logs the archiving is done only inside OutBoard. 

  1. Data Archiving Process (DAP) can be created by using transaction RSDAP or from the Administrator Workbench (transaction RSA1) by selecting an InfoProvider
    1. Select your InfoProvider from the context menu
    2. Select "Create Data Archiving Process" (if none exists) or go to "Data Archiving Process", where you can Display, Change and Delete Archive processes.
  2. When archiving a group of objects, above the standard SAP functionality, OutBoard provides automated DAP creation as mass processing. (See section on Mass archiving for more details)

    ΤA: RSDAP / RSA1

    DAP creation in RSA1

  3. When creating/ changing the DAP there are five tabs

  • General Settings
  • Selection Profile
  • Semantic Groups
  • ADK (automatically switched off if OutBoard archiving is used)
  • Nearline Storage

What needs to be set up for Outboard is explained in more detail below.


Creating the data archiving process

General Settings

Identifies the type of archive storage used on a system, i.e. "ADK-Based Archiving", NLS, or both. Datavard's OutBoard works without ADK-Based Archiving.
So to enable storage by OutBoard:

  • Uncheck "ADK-Based Archiving"

Enter "OUTBOARD" as "Nearline Connection".

To view the connection status of Outboard, click on the scales icon on the right of the Near Line Connection input field.

Connection status of Outboard

Selection Profile

Determines, which InfoObjects archiving requests are created. In figure 9, in the general settings tab under 'selection profile' Time Slice Archiving was selected therefore time characteristics can be used as primary partitioning characteristic, but when time characteristic is not in key also any non-time key field can be used as a primary partitioning characteristic.
Typical Time Slice Characteristics are:

  • Calendar day
  • Calendar year/ month
  • Calendar year
  • Fiscal year
  • Fiscal year/period
  • Request Date Created (for WODS)

By choosing 'Allow Arch. of non-compressed data', the user is also allowed to archive the data from F-table = non-compressed data. If you decide to allow this, the performance of archiving is lower, because needed SELECTS are more complicated. This option is available as of SAP BW 7.30. 
To further partition data use the InfoObjects under 'Additional Partitioning Characteristics', by selecting from the table of InfoObjects on the right and move them to the left table (using arrows).

Selecting Additional partitioning characteristics, tab Selection Profile


If a non-key characteristic is used for archiving warning messages and a log with further information is displayed, it is highly recommended to select the time characteristic that is a part of the DSO key. The InfoProvider key fields that are available for selection are highlighted as shown in the figure below.

Characteristic selection


When there is no time characteristic in the key of the DSO, then the “characteristic for time slice” refers to a time characteristic in the data field of the DSO. The primary partitioning characteristic refers to the selection defined in the Data Archiving Process. The way archiving works is that the system will select all primary partitioning characteristics that are older than the date of the “characteristic for time slice. More information can be found in the document SAP guide.

Semantic Group

Due to the size of the requests that are going to be archived, these must be split into smaller parts, called "data objects".
Data objects can be defined in the Semantic Group tab of a DAP. If undefined, by default a data object is created for every 50 000 records
In the "Semantic Group" tab you specify the structure of your archived data in a sorted way and distribute into data objects. The system reads from the database after sorting according to grouping characteristics.


Archived data structure


For example, if 0CALYEAR and ZMANAGER are chosen, data objects could look like this:

Data Object

0CALYEAR

ZMANAGER

1

2000

XYZ

2

2000

WZY

3

2001

XYZ

The use of Semantic Groups is optional and can enhance the performance when reporting and archiving data when created to an optimal structure.

Creating a Data Archiving Process, the tab Semantic Group


The use of Semantic Groups has beneficial effects on reporting and loading OutBoard performance and is strongly recommended to use at any time. Contact Datavard Consultancy for support, to design the optimal Semantic structure for your InfoProviders, or refer to the Semantic Group Best Practices document.


InfoObjects in the Semantic group should be chosen the way that the average package size will not have more than '100 000 rows.

Nearline Storage

This tab gives an overview of the Nearline connection and size of the data package (data packet up to SAP BW 7.02). Data objects are grouped into data packages. When a request is loaded via DTPs, it can be parallelized on the data package level.Nearline Storage

Creating the Data Archiving Process, tab Nearline Storage

We recommend keeping the size of the Data Package low:

  • InfoCubes: 50MB
  • DSO: 100MB
  • For objects over 500 million records, should be higher (200 - 500 MB)

To determine how a new Data Package is created can be done in two ways :
1. By defining the "Maximum Size in MB":

  • For InfoCubes 50MB
  • For DSO's  100MB

2. By defining the "Maximum Number of Data Objects":

  • InfoCubes: 50
  • DSO: 100

→ We don't recommend using this option (number 2), because the size of a DataObject can be unpredictable


If none of these fields is filled, there is the danger that memory limits on the application server will be exceeded causing the archiving to fail

Activate DAP

Once the Outboard settings have been completed,  activate the new data archiving process.

When using OutBoard with Storage Management you will be asked to select the OutBoard archiving profile (for more information please see the section: OutBoard Cockpit -> Settings). 

Aging profile selection

Once activated a Log Display will appear:

Data Archive Process activation Log

When a DSO's primary key does not include its DAP time characteristic it is possible to encounter some problematic situations during the activation and reload of data. If a new record arrives at the DSO, with the same primary as an archived record but a different time characteristic, this record will be placed in the active table. If the archive is reloaded the system might dump it because it will find a record with the same key in the active table.

The following solution reloads while activating a request, all the duplicate records from the archive in the active table, and overwrites them with the new data. 

Set an infoprovider to support reload of duplicates
  • Open the /DVD/NLS_SET_RELOAD_DUPLICATES report.
  • Insert names of your InfoProviders.
  • Check the 'Support reload duplicates' to true to activate the functionality, ('false' to deactivate it).
  • Execute.

Editing the DAP

"Edit Data Archiving Process" via transaction:
RSA1
RSDAP :
a/ In the context menu of the chosen InfoProvider
b/ Choose the object type you desire by clicking on the drop-down menu in "Object Type". The Object Type can be either an InfoCube or a DataStore Object
c/ Fill out the "Obj. Name" with the name of the InfoCube or DataStore Object you wish to work with
d/ Process to Display / Edit.  Here you can view the status of whether the DAP is active or not.

ΤA: RSDAP / RSA1

Transaction RSDAP - Display/Edit/Delete existing DAP´s

Displaying the Data Archive Process

Delete DAP

To delete a DAP, open the transaction RSDAP and press "Delete" for the chosen InfoProvider (DSO) or in RSA1 select the InfoProvider from the context menu and use Delete in the Data Archiving Process.

When you delete a DAP data archived in Outboard for this corresponding InfoProvider will also be deleted.