Configure details

Next, we can configure the Data Export expression file to tailor your integration.

An expression file in SnapLogic is a configuration file that can be used to house custom variables and values which are referenced by settings in the pipelines. In this case, we provide a template file that is designed to work specifically with the Identity Cloud Data Export.

We’ll complete this configuration in three main steps:

  • Download the expression file template
  • Configure the expression file
  • Upload the configured expression file to your project folder

Download expression file template

In the SnapLogic Manager tab:

  1. In the left navigation under the Identity Cloud Data Export project space, click on the AIC_DataExportPattern folder to view its contents
  2. In the main panel, click directly on the DataExport.expr file in the Name column to download it to your computer Download expression file in SnapLogic

Configure expression file

  1. Open the DataExport.expr file you downloaded in a text editor
  2. Revise the values in the file to tailor it for your data export (see example and table below for details)

Example:

{
	"identityCloudAccount": "My Akamai account",
	"targetSaveLocationAccount": "My receiver account (SFTP or S3)",
	"notificationEmails": "email1@example.com, email2@example.com",
	"entitiesToExport": [
		{
			"entityType": "user",
			"attributes": "[\"uuid\", \"givenName\", \"familyName\", \"email\"]",
			"filter": "status='Active'",
			"exportFilePrefix": "optional_prefix",
			"exportFormat": "csv",
			"targetSaveLocation": "s3:///bucket@s3.us-west-2.amazonaws.com/dir/"
		},
		{ ANOTHER ENTITY TYPE }
	]
}
Key Value Instructions
identityCloudAccount The Label/Name for the Akamai account you created in SnapLogic.
targetSaveLocationAccount The Label/Name for the sFTP or S3 account you created in SnapLogic.
notificationEmails A comma-separated list of email addresses to receive error notifications.

An error notification email will be sent if there is a failure in the pipeline when the integration is executed. The email contains error details including what needs to be fixed.
entitiesToExport An array of objects, one for each Identity Cloud entity type to export from. If only exporting from one entity type, this will be an array containing one object.
entitiesToExport:
entityType		 The name of the Identity Cloud entity type to export from (e.g. user).
entitiesToExport:
attributes		 An array of Identity Cloud schema attributes to export. This defines the data that will be exported for each user record.

NOTES:

  • You must use the format seen in the template, including using backslashes to escape quotes (e.g. \"givenName\"). This format is required by SnapLogic.
    • Exporting object or plural attributes? Skip to More info for format and options.
  • It's a good idea to include the Identity Cloud uuid in the export. A user could change other identifying attributes such as email address, whereas the uuid is a unique identifier that stays constant for the life of the user record.
  • For best security practice, please do not export the password attribute. Even though the passwords are encrypted, exposing them outside of the Identity Cloud database adds unnecessary security risk.
entitiesToExport:
filter		 OPTIONAL filter which narrows down the exported user records to only those matching the given expression.

This filter is based on values stored in the Identity Cloud user record. The example code above has the following filter: "filter": "status='Active'". This means the integration will only export records where a schema attribute called status has been set to Active. Records with any other status value, including null, will not be exported.

If no filter is needed, leave this value blank ("").

This filter uses the same syntax as the /entity.find filter. For more information, see /entity.find.
entitiesToExport:
exportFilePrefix		 OPTIONAL file name prefix.

The naming convention of the export file is entityType_dateTimeStamp.ext (e.g. user_20200918T071048.csv or user_20200918T071048.json). If you need an additional identifier in the file name, the prefix you set here will change the file name to prefix_entityType_dateTimeStamp.ext.

The example code above has the following prefix: "exportFilePrefix": "optional_prefix". This will result in the following file name, for example: optional_prefix_user_20200918T071048.csv.

To keep the standard file name, leave this value blank ("").

You do not need to add an underscore after the prefix value - it is added automatically.
entitiesToExport:
exportFormat		 The format of the export file. Options are:
  • csv
  • json

If this is left blank or misconfigured, the file format will default to csv.
entitiesToExport:
targetSaveLocation		 Location within receiver account to send the export files. Note that this allows for a different directory per entity type, if desired.

sFTP Format: sftp://ipserver-address-or-ip/dir/
  • Example: sftp://ftp.snaplogic.com:22/incoming/
S3 Format: s3:///S3_bucket@s3.region.amazonaws.com/dir/
  • Example: s3:///mybucket@s3.eu-west-1.amazonaws.com/

Make sure to include the slash (/) at the end of this value, as in the examples above.

If you specify a directory in the path that does not exist in the receiver location, it will be created there the first time the data export is executed.
  1. Save the expression file to your computer in a place you’ll remember should you decide to revise the configuration in the future.

DO NOT RENAME THE FILE! The expression file you upload to your project folder must be named DataExport.expr - Any other file name will not be recognized by the Data Export pattern. If you will be running multiple instances of the Data Export integration, plan to save each different DataExport.expr file to a different folder on your computer to keep them organized and prevent overwriting.

Upload expression file

Once you’ve configured and saved your expression file locally, you can upload it to your project folder. In the Manager tab:

  1. In the left navigation under the Identity Cloud Data Export project space, click on your project folder to view its contents
  2. In the main panel, click on the Files tab
  3. In the upper right corner, click the Create icon ()
  4. Choose your configured DataExport.expr file from your computer and click Upload Upload expression file in SnapLogic

You can view the content of the file you uploaded in SnapLogic by hovering over the file name, clicking the little arrow next to it, and selecting View. View file in SnapLogic

Good to know: It is not possible to edit the expression file directly in the SnapLogic interface. In the future when you need to update your expression file, you must:

  1. Make your updates locally.
  2. Upload the updated DataExport.expr file to your project folder in SnapLogic.
  3. You should see an “Asset conflict error” pop up. Select Overwrite.

Your project folder should now contain:

  • Three (3) pipelines
  • Two (2) accounts
  • One (1) expression file