Extractor type Remote File to HDFS is used to transfer a file from a remote path to Hadoop HDFS.
To do so, you must follow a similar procedure as described in (Glue-2011) Local File to HDFS.
Fill in following fields:
- Remote file path – path to the remote directory
- Remote file name – remote file name
- HDFS path – path in Hadoop HDFS filesystem where the file will be copied to
- HDFS filename – a name of the file after it is transferred
- RFC destination – RFC (Remote Function Call) which is used for the transfer
Remote File to HDFS Extractor
After the definition of the extractor, create a variant.
Remote File to HDFS Variant
There are 2 ways how to trigger the data transfer:
- Execute (in dialog) - execute the job directly
- Schedule (in background). In the case of larger files, we recommend using this option.