Table of Contents:

1 General Prerequisites
- 1.1 Reuse Library release
- 1.2 SAP NetWeaver release
- 1.3 Open Ports
- 1.4 Kyano Java Connector
- 1.5 Kafka Client Config File
  - 1.5.1 Confluent cloud
  - 1.5.2 Apache Kafka with Kerberos
2 Create Kafka storage in Kyano Storage Management

Storage management supports writing data to Kafka. The integration provides the following function:

Automatic topic creation is derived from the source object name.
Sending JSON or AVRO serialized messages.
Integration with Schema Registry

The connectivity is tested with Confluent Kafka and Apache Kafka, but should be compatible with any Kafka distribution or Kafka-compatible API.

General Prerequisites

Reuse Library release

Kafka integration is available in Reuse Library 2311 or higher.

SAP NetWeaver release

Kyano Storage management requires SAP ABAP stack NW 7.01 SP15 or higher.

Open Ports

To enable communication between SAP systems and Azure, outbound communication from the SAP system to the following services on the Kafka cluster needs to be allowed:

Port	Protocol	Target

Port	Protocol	Target
9092	TCP	Kafka host FQDN
2181	TCP	Zookeeper host FQDN

Kyano Java Connector

The Kyano Java Connector is a critical middleware component used for communication between SAP and Kafka. Follow the steps in the chapter (SM-2508) SNP Kyano Java Connector Setup before you continue. The minimum required Kyano Java Connector version for Kafka integration is 238.

Kafka Client Config File

Authentication to the Kafka cluster is configured using the standard client.config file. The file needs to be accessible to all SAP application servers. The recommended location is /sapmnt/<SID>/global/security/dvd_conn/client.config

Below you can see several example configuration files.

Confluent cloud

# Kafka cluster configuration
bootstrap.servers=<bootstrap-server>
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<api-key>" password="<api-secret>";

# Schema Registry configuration
schema.registry.url=<schema-registry-url>
basic.auth.credentials.source=USER_INFO
basic.auth.user.info=<schema-registry-api-key>:<schema-registry-api-secret>

Apache Kafka with Kerberos

# Kafka producer configuration
bootstrap.servers=<bootstrap-server>
acks=all

# Kerberos authentication configuration
security.protocol=SASL_SSL
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka

# JAAS configuration using keytab file
sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/path/to/bob.keytab" principal="bob@REALM";

# SSL truststore configuration
ssl.truststore.location=/opt/cloudera/security/jks/truststore.jks
ssl.truststore.password=<truststore-password>

If you need to specify a non-default krb5.conf file, you can do so by adding string -Djava.security.krb5.conf=/path/to/krb5.conf to parameter “Advanced → Additional java starting arguments” in t-code /DVD/JCO_MNG

In case the AVRO file type is used, the Schema Registry configuration is mandatory.

Create Kafka storage in Kyano Storage Management

Proceed as follows to create a Kafka storage:

Start transaction /DVD/SM_SETUP.

Click Create to create a new storage.

Enter a Storage ID.

In the Storage type field, enter KAFKA.

Click Confirm.

Fill out the desired entries as described in the table below.

Storage ID	Logical identifier of the storage (maximum 10 characters)
Java connector RFC	RFC destination used for communication with the Kyano Java connector
Configuration file	When the button is pressed, a pop-up window appears with a text input field where you can enter one of two options: Physical path to the `client.config` file. It must be accessible from all application servers or the JCO host in standalone JCO deployments. You can paste the contents of a config file, where passwords can be encrypted using the pattern `<pass><pass>` (e.g., `password="<pass>password123<pass>"`). When storage is saved, the password will be automatically encrypted.
File Type	Format of messages to send to Kafka Topics. Possible values: `JSON` or `AVRO`
Conversion type	In case Avro is used, a conversion type can be chosen. `CSV` or `JCo table`, the default and recommended is `CSV`.
Write mode	Possible values: `Transactional`: Guarantees exactly-once delivery but provides lower throughput due to the overhead of transaction management. `Non-transactional`: Provides higher throughput but only guarantees at-least-once delivery, which means that duplicate messages are possible.
Use topic	Flag to create a topic during table activation. In case it is turned off topic must already exist on the target.
Replication factor	A replication factor of the created topic.
Topic Partitions	Several topic partitions are created.

Click Save.

The new storage has been created.

(SM-2508) Kafka