Understand and learn how to import data in Insight Cloud.
While you can manually create objects for an object type one after the other in Insight, doing that for a complex schema with a large set of data can be exhausting and tedious.
To solve this, Insight Cloud offers a feature to import data from a CSV data source.
You can import Insight data from a CSV file to an object type of your object schema. Your CSV file should adhere to a structure and abide by certain rules so that Insight can process and import data well. All of these rules have been described next in this section.
CSV file specifications
Your CSV file must comply to the following structure:
- Your file must have one Header record and can have multiple Data records. Each comma separated value in the Header/Data is a field.
- Your file must have a header record on the first line. This consists of comma separated string of names of the attributes of an object type in your schema. This is the object type that you want to import data for.
- Every record (line) of data including the header is on a new line, separated by a line break.
- The last record may not have a line break.
- The last record must not be followed by a comma.
- Headers are not case sensitive.
- You can decide the sequence of attributes in the header record. The data records on the subsequent lines must then follow the same order.
- Any spaces in the fields on both header and data records are considered a valid value. However, if you insert leading/trailing spaces in header or data records, the import process will ignore them.
- Apart from the commonly used comma( , ) as the separator between fields, you can also use a semicolon( ; ) to separate data.
- If any field in your data record itself contains a comma for e.g: Apple, Orange then you need to surround that field with double quotes. So the field in your CSV file should look like this:
Your file additionally should follow the rules below for Insight import to work well:
- You must mention the Key attribute of your object type in the header record. If not, the file will not be processed and data will not be imported.
- In the header record, if you mention an attribute which is marked as the label for your object type, you must provide a value for it in the data record. If not, the file will not be processed and data will not be imported.
- Apart for the Key and Name attributes, other default attributes on your object type like Created, Updated are ignored and not considered for importing data.
- A field in the data record must be compatible with the type of the corresponding attribute in the object type so that it can be parsed correctly. E.g. if an attribute type is boolean, the file can have a data record field which can be either true or false.
Error/information messages for few of these scenarios have been described below.
Below is an example of a CSV file for an object type Department in an object schema People Schema.
Example of a CSV file
E.g, Lets say that you have the Department object type of People Schema and it looks like this:
For the above, the CSV file could be as shown below. This CSV file was created using a text editor. You can use any text editor like Notepad, Atom, Sublime etc to create your CSV files.
We have issues with non-english characters in Microsoft Excel and hence we do not support Microsoft Excel for imports.
However, if you use Google Sheets with UTF-8 characters, that should work well.
Rules for data records in CSV file
The following table describes how to mention data record fields for the corresponding header record field (your object type attribute) in your CSV file so that you avoid errors later in the import process.
|Header record||Data record|
You may/may not provide a value.
If you DO provide a value, then the object corresponding to the key in your schema will be updated.
If you DO NOT provide a value, a new object will be created for the corresponding object type in your schema.
|Referenced Object field|
You must provide the Key for an existing object. If you provide Name or any other attribute value, it will not be processed.
If you have to specify multiple values, use the pipe(||) symbol to separate them.
E.g. In your People schema, suppose you have Department objects HR and IT with keys OR-19 and OR-11 respectively.
Department is referenced in the Employee object. You want to mention both these departments in the data record for Employee. In that case, Employee CSV must look like this:
Key, Name, Designation, Department
OR-1, John Doe, Senior Engineer, OR-19||OR-11
|Date type field|
Supported date format is ISO-8601. Date format must be one of the above:
YYYY: The year specified with four digits
UTC Offset - the difference in hours and minutes from Coordinated Universal Time (UTC) for a particular place and date.
The offset is indicated in bold in this value : 2001-09-12T13:15:17.309+2:00.
|Status field on an object schema|
You can mention the name or the Id of the values.
E.g. on your object schema, suppose you have a Status field with the following possible values:
If you want one of these values to be mentioned in the data record, you simply have to provide the respective name or Id for these.
If you want to mention multiple records, you must separate them by the double pipe (||) symbol like this: Preboarding||Onboarded OR like this: 9||13
Note that if you misspell the name or if the name is non-existent, it will not be considered.
|Any multi-valued field||The rule is same as that mentioned for a referenced object type.|
To initiate a CSV import, on your cloud instance:
- Click on the CSV Import option in the menu on the left panel. Note that if you use the alternate navigation, this option will be visible in the top menu bar.
- Select the object schema and the object type for which you want to import the data.
- Specify the delimiter (could be either a comma or a semicolon) from the dropdown.
- Upload your CSV file.
- Click on Start CSV Import button.
A reference screenshot of the import screen is shown below.
Once the import process gets completed, you will see a message informing you about it as shown below.
If the import process produces any problems, you will see a corresponding error message as shown below which states how many objects were imported and how many were not. Additionally, it will display an error message stating the problem in the data record due to which the import failed.
If you have a huge data set to import, the progress of the import is shown too as shown below.
Only one import per object type is allowed. It is not possible to begin importing while another import is running. If you try to initiate another import while the current one is still ongoing, you will get an error as shown below.
Large imports may take longer to complete. Few metrics for large imports are shown below. The time may also vary based on the complexity of the object tree.
|Number of objects||Import time|