Retrieving Data from an S3 Bucket

You must use SFTP (SSH File Transfer Protocol) and a valid public key in order to retrieve data from your Amazon S3 bucket. Currently Amazon Web Services supports the following SFTP clients:

Note that SFTP is the only way to access the S3 bucket.

Note. We should also mention that Amazon’s SFTP Transfer service is not yet available in either the Sao Paulo or the China AWS regions (although the service is expected soon). This doesn’t mean organizations in these regions aren’t eligible for SIEM Event Delivery; it just means that some extra configuration steps might be required to get the service up and running. See your Identity Cloud representative for more information.

If your organization needs to set up an allow list for data retrieval -- that is, if you need to limit data retrieval to a specific set of IP addresses -- Akamai has configured a pair of static IP addresses for each Identity Cloud region:

RegionDNS EntryIP Address 1IP Address 2
North America (Virginia)184.73.183.1383.209.201.47
Europe (Ireland)52.209.21.652.208.55.204
North America (Canada)http://eventdelivery.multi.prod.cc.janrain.com3.96.12.13915.223.99.188
Asia (Singapore)http://eventdelivery.multi.prod.sg.janrain.com54.151.248.22454.151.248.240
South America (Sao Paulo)http://eventdelivery.multi.prod.sp.janrain.com54.232.113.5754.94.105.120
Australia (Syndey)http://eventdelivery.multi.prod.sy.janrain.com13.238.72.5552.64.26.211
Asia (Tokyo)http://eventdelivery.multi.prod.tk.janrain.com54.248.31.8254.199.66.89

For example, in the US your allow list for SIEM data retrieval would include these two IP addresses:

  • 3.209.201.47
  • 52.208.55.204

When SIEM Event Delivery is activated, you’ll get back an API response similar to the following:

{
    "message": "Your request has been accepted and is being
    processed.",
    "sftp":
        {
         "uri": "sftp://user_htb8fuhxnf8e38jrzub3c7pfrr",
         "user": "user_htb8fuhxnf8e38jrzub3c7pfrr",
         "host": "v1.api.us.janrain.com"
       }
}

The uri and the user fields are especially important: that’s the information needed to access your S3 bucket. As noted elsewhere, each organization is given a single user account (the user field), with the username composed of user_ followed by your application ID (for example, user_htb8fuhxnf8e38jrzub3c7pfrr). All users who access the S3 bucket must log on using this same username (as well as an SSH key associated with the S3 bucket).

Meanwhile, the uri field specifies the URL for your S3 bucket. In the preceding example, that URL is sftp://user_htb8fuhxnf8e38jrzub3c7pfrr@eventdelivery.v1.api.us.janrain.com.

The exact steps required to access your S3 bucket depend on which SFTP client you use. For example, if you use Cyberduck you’ll need to follow a procedure similar to this:

  1. Start Cyberduck and then click Open Connection:
  2. In the dropdown dialog, set the protocol to SFTP (SSH File Transfer Protocol):
  3. Type the URL to your Amazon S3 bucket (for example, sftp://user_htb8fuhxnf8e38jrzub3c7pfrr@eventdelivery.v1.api.us.janrain.com) in the Server field and the port number for the S3 bucket in the Port field:
  4. Enter your S3 bucket username (e.g., user_htb8fuhxnf8e38jrzub3c7pfrr) in the Username field. Leave thePassword field blank:
  5. Click SSH Private Key and then select the private key you are using for S3 access. Keep in mind that the corresponding public key must have already been associated with the S3 bucket:
  6. When you are finished, click Connect:

After the connection is made, your SIEM event files will appear in the Cyberduck window.

To download a file, right click the file name and then click either:

  • Download (to download the file directly to your Downloads folder).
  • Download As (which enables you to specify a different file name and/or download location).
  • Download To (which lets you change the download location but not the file name).

To remove a file from the S3 bucket, right-click the file name and then click Delete.