Destination S3 #

The extracted replicant-cli will be referred to as the $REPLICANT_HOME directory in the proceeding steps.

I. Setup Connection Configuration #

From $REPLICANT_HOME navigate to the sample S3 connection configuration file
```
vi conf/conn/s3.yaml
```

Make the necessary changes as follows:

type: S3

access-key: "REPLICANT" #Replace REPLICANT with the access key of AWS user created from IAM management
secret-key: "Replicant#123" #Replace Replicant#123 with the secret Key of the AWS User (note: make sure the specified user has  AmazonS3FullAccess)

bucket: "bucket_name" #Replace bucket_name with the name of your S3 bucket
root: "root_dir" #Replace root_dir with the name of the directory inside the s3 bucket where the CSV files will be stored

stage:
  type: SHARED_FS
  root-dir: /home/user/stage #Enter the path of the directory where CSV files will be staged before uploading to S3

#file-format: CSV #Possible file formats are CSV or JSON
max-connections: 50 #Maximum number of connections Replicant can open in the target s3

Setup Applier Configuration #

From $REPLICANT_HOME navigate to the sample S3 applier configuration file
```
vi conf/dst/s3.yaml        
```

If necessary, make the necessary changes as follows:

snapshot:
  #threads: 16
  #max-file-size: 33_554_432 #32MB
  #delimiter: `,` #CSV files created will have provided delimiter
  #quote: `”`
  #escape:   `\`
  #include-header: false #enable or disable toggle column names as header in CSV file

#realtime:
  #threads: 16

III. S3 File Format #

When data is being loaded into S3, the data is first converted to either a CSV or JSON file. Follow the link below to understand the data format for the CSV and JSON converted files.

S3 File Format

Note: It is highly recommended that you read the S3 file format page when using S3 as the target system.