JUMP TO: |
---|
A single CSV file will be exported for each entityType
configured in your expression file. The format of the export file name is entityType_dateTimeStamp.csv (e.g. user_20200918T071048.csv
).
You can optionally add a custom prefix to each file name; see the exportFilePrefix
key under Configure expression file.
The exported CSV file follows this format:
entitiesToExport
: attributes
). The column header is the attribute name.Example:
uuid | givenName | familyName | birthday | |
---|---|---|---|---|
23291df7-5796-4c6f-8afd-a684b2ba7576 | John.Moore_908@test.com | John | Moore | 2/14/1991 |
157e6ef1-1513-403e-a86c-47849dab7478 | Tiffany.Henson_608@test.com | Tiffany | Henson | 4/18/1981 |
fbc382fb-3703-4bb6-8244-0d3794d41fd3 | Sonia.Hopkins_492@test.com | Sonia | Hopkins | 11/26/1956 |
You can configure object and plural attributes in your expression file a couple of different ways, depending on the information you want in the export file.
Configure the name of the parent object to receive all attributes within the object. Each attribute within the object will be placed in its own column in the CSV and the column headers will identify each attribute in dot notation.
\"primaryAddress\"
primaryAddress.address | primaryAddress.city | primaryAddress.state | primaryAddress.zip | primaryAddress.country |
---|---|---|---|---|
123 Street Ave | Cityville | OH | 12345 | USA |
Configure a child attribute in dot notation to receive only that attribute, rather than every attribute contained in the object.
\"primaryAddress.zip\"
primaryAddress.zip |
---|
12345 |
Configure the name of the parent plural to receive the entire contents of the plural. Plural data is exported as JSON within a single column.
\"legalAcceptances\"
legalAcceptances |
---|
[{“clientId”:“a7nnd3wp5qaemt2wu4aqxe6xrh9jq3jd”,“dateAccepted”:“2018-01-14 03:12:04 +0000”,“legalAcceptanceId”:“privacy_policy_v1”,“id”:4738},{“clientId”:“a7nnd3wp5qaemt2wu4aqxe6xrh9jq3jd”,“dateAccepted”:“2018-01-14 03:12:04 +0000”,“legalAcceptanceId”:“terms_v1”,“id”:4739}] |
Configure a child attribute in dot notation to receive only that attribute within the plural JSON, rather than every attribute contained in the plural. Note that the plural entry id is always returned along with the specified child attribute.
\"legalAcceptances.legalAcceptanceId\"
legalAcceptances |
---|
[{“legalAcceptanceId”:“privacy_policy_v1”,“id”:4738},{“legalAcceptanceId”:“terms_v1”,“id”:4739}] |
The first time the integration runs, it will export ALL records in the specified entity type(s). At the end of the first run, a file called DataExportLastRun.json is created in your project folder (you can find it in the Manager tab). This is the file which the integration will refer to and write to each time it runs thereafter, in order to grab only the new and updated records since the previous run.
If you ever need to reset and export ALL records again, simply delete this file. The absence of the file lets the integration know that there is no LastRunDate
, therefore it will act as if it is the very first run and export all user records.
If you accidentally delete this file but do NOT want to reset and export all records, you can create it with the same file name on your computer. The content of the file should look like this:
[{"EntityType":"user","LastRunDate":"'2020-10-07T17:57:22.713Z'"},{"EntityType":"user_other","LastRunDate":"'2020-10-07T17:57:24.045Z'"}]
You’ll need to replace the EntityType
and LastRunDate
values (above in orange) with the real entity type name(s) and the real last run date/time(s), then upload the file to your project folder before the next scheduled run begins.
If you run into API rate limit errors, these are likely coming from the Export Data snap.
The Export Data snap is the first snap in the DataExport_ProcessAndSave pipeline. Open this pipeline in the Designer tab, click on this snap to open it, and increase the value of the API interval (ms) setting. This will increase the time, in milliseconds, between calls being made to the Identity Cloud. The default is 1000 which should suffice in most cases. However if you’re running into API limit errors, try increasing this to solve the error.
If you’re getting a timeout error, you may also want to the check the Timeout setting in your scheduled task configuration. Make sure this setting is blank.
If your pipelines are deleted at any point (perhaps you deleted your pipeline to get a fresh copy from the pattern), this will disable your related scheduled task. SnapLogic will send you an email alert if this happens. In order to get the scheduled task working again, you must do two things: