JUMP TO: |
---|
A single file will be exported for each entityType
configured in your expression file. The naming convention of the export file is entityType_dateTimeStamp.ext (e.g. user_20200918T071048.csv
or user_20200918T071048.json
).
You can optionally add a custom prefix to each file name; see the exportFilePrefix
key under Configure expression file.
The exported csv
file follows this format:
entitiesToExport
: attributes
). The column header is the attribute name.Example:
uuid | givenName | familyName | birthday | |
---|---|---|---|---|
23291df7-5796-4c6f-8afd-a684b2ba7576 | John.Moore_908@test.com | John | Moore | 2/14/1991 |
157e6ef1-1513-403e-a86c-47849dab7478 | Tiffany.Henson_608@test.com | Tiffany | Henson | 4/18/1981 |
fbc382fb-3703-4bb6-8244-0d3794d41fd3 | Sonia.Hopkins_492@test.com | Sonia | Hopkins | 11/26/1956 |
The exported json
file contains an array with an object for each user record.
Example:
[
{
"givenName" : "John",
"email" : "John.Moore_908@test.com",
"uuid" : "23291df7-5796-4c6f-8afd-a684b2ba7576",
"familyName" : "Moore",
"birthday" : "1991-02-14"
},
{
"givenName" : "Tiffany",
"email" : "Tiffany.Henson_608@test.com",
"uuid" : "157e6ef1-1513-403e-a86c-47849dab7478",
"familyName" : "Henson",
"birthday" : "1981-04-18"
},
{
"givenName" : "Sonia",
"email" : "Sonia.Hopkins_492@test.com",
"uuid" : "fbc382fb-3703-4bb6-8244-0d3794d41fd3",
"familyName" : "Hopkins",
"birthday" : "1956-11-26"
}
]
You can configure object and plural attributes in your expression file a couple of different ways, depending on the information you want in the export file.
Configure the name of the parent object to receive all attributes within the object. In a csv
file, each attribute within the object will be placed in its own column and the column headers will identify each attribute in dot notation.
\"primaryAddress\"
primaryAddress.address | primaryAddress.city | primaryAddress.state | primaryAddress.zip | primaryAddress.country |
---|---|---|---|---|
23 Street Ave | | ityville | | H | | 2345 | | SA | |
"primaryAddress" : {
"address" : "123 Street Ave",
"city" : "Cityville",
"state" : "OH",
"zip" : "12345",
"country" : "USA"
},
Configure a child attribute in dot notation to receive only that attribute, rather than every attribute contained in the object.
\"primaryAddress.zip\"
primaryAddress.zip |
---|
2345 | |
"primaryAddress" : {
"zip" : "12345"
},
Configure the name of the parent plural to receive the entire contents of the plural. In a csv
file, plural data is exported as JSON within a single column.
\"legalAcceptances\"
legalAcceptances |
---|
{“clientId”:“a7nnd3wp5qaemt2wu4aqxe6xrh9jq3jd”,“dateAccepted”:“2018-01-14 03:12:04 +0000”,“legalAcceptanceId”:“privacy_policy_v1”,“id”:4738},{“clientId”:“a7nnd3wp5qaemt2wu4aqxe6xrh9jq3jd”,“dateAccepted”:“2018-01-14 03:12:04 +0000”,“legalAcceptanceId”:“terms_v1”,“id”:4739}] | |
"legalAcceptances" : [
{
"clientId" : "a7nnd3wp5qaemt2wu4aqxe6xrh9jq3jd",
"dateAccepted" : "2018-01-14 03:12:04 +0000",
"legalAcceptanceId" : "privacy_policy_v1",
"id" : 4738
},
{
"clientId" : "a7nnd3wp5qaemt2wu4aqxe6xrh9jq3jd",
"dateAccepted" : "2018-01-14 03:12:04 +0000",
"legalAcceptanceId" : "terms_v1",
"id" : 4739
}
],
Configure a child attribute in dot notation to receive only that attribute within the plural JSON, rather than every attribute contained in the plural. Note that the plural entry id is always returned along with the specified child attribute.
\"legalAcceptances.legalAcceptanceId\"
legalAcceptances |
---|
{“legalAcceptanceId”:“privacy_policy_v1”,“id”:4738},{“legalAcceptanceId”:“terms_v1”,“id”:4739}] | |
"legalAcceptances" : [
{
"legalAcceptanceId" : "privacy_policy_v1",
"id" : 4738
},
{
"legalAcceptanceId" : "terms_v1",
"id" : 4739
}
],
The first time the integration runs, it will export ALL records in the specified entity type(s). At the end of the first run, a file called DataExportLastRun.json is created in your project folder (you can find it in the Manager tab). This is the file which the integration will refer to and write to each time it runs thereafter, in order to grab only the new and updated records since the previous run.
If you ever need to reset and export ALL records again, simply delete this file. The absence of the file lets the integration know that there is no LastRunDate
, therefore it will act as if it is the very first run and export all user records.
If you accidentally delete this file but do NOT want to reset and export all records, you can create it with the same file name on your computer. The content of the file should look like this:
[
{
"EntityType" : "user",
"ExportFilePrefix" : "",
"LastRunDate" : "'2022-08-25T19:13:08.682Z'"
},
{
"EntityType" : "user_other",
"ExportFilePrefix" : "",
"LastRunDate" : "'2022-08-25T19:13:13.817Z'"
}
]
You would need to:
If you run into API rate limit errors, these are likely coming from the Export Data snap.
The Export Data snap is the first snap in the DataExport_ProcessAndSave pipeline. Open this pipeline in the Designer tab, click on this snap to open it, and increase the value of the API interval (ms) setting. This will increase the time, in milliseconds, between calls being made to the Identity Cloud. The default is 1000 which should suffice in most cases. However if you’re running into API limit errors, try increasing this to solve the error.
If you’re getting a timeout error, you may also want to the check the Timeout setting in your scheduled task configuration. Make sure this setting is blank.
If your pipelines are deleted at any point (perhaps you deleted your pipeline to get a fresh copy from the pattern), this will disable your related scheduled task. SnapLogic will send you an email alert if this happens. In order to get the scheduled task working again, you must do two things: