The Data Import Utility provided on the List screen of the Sites page in Flowlink Cipher is used to import comma-separated value (CSV) and data dump (DDP) files.
• Supported CSV files come from the Flowlink 5.1 functions File > Export and Database > Archive/Transfer, however, files generated by third‑party applications are also supported if they use the header and data record formats described below.
• DDP files contain site data that was physically collected from a device in the field and needs to be uploaded to Cipher.
To import one or more files:
1. Click the Import ISCO Data button on the sites screen to launch the Data Import Utility.
2. Select the file(s) to be imported by either:
• Clicking the Browse Files button and navigating to the relevant files. Select the source file(s) and click Open.
• Dragging each source file into the utility window from its file location.
3. Define the Delimiter used in the source data; comma is used by default.
4. Select the Date format to use for CSV files.
5. Click the Upload Files button in the utility.
Each time an import job is started, it is added to a job queue. This queue organizes the data‑import jobs submitted through the Data Import Utility, avoiding conflicts and delays during import. The job queue maintains the following information for each job:
• a unique identifier (ID)
• the user who submitted the job
• the site and tenant to which data is being imported
• a timestamp of when the import was performed
For users with the Site Manager role, a Jobs button 
is available in the toolbar at the top of the interface. This button can be used to view a list of the jobs the current user has in the queue, and the status of each job. The possible states include:
• Waiting
• Running
• Complete
Once is a job is complete, a notification will be added to the Notifications list indicating whether the import was successful or not.
A Cancel Job button 
is also provided to cancel a job that has a status of Waiting or Running. Once a job is cancelled, options are displayed to retry the import 
, download the data 
or delete the job 
.
If an import requires the creation of a new site, the system will check the site count limit for the license as well as the tenant, if the site belongs to a tenant. If either limit has been reached, an error message will be displayed and the data will not be ingested. The site will still be created, but it will be inactive. Future data ingestions for the site will not be stored in Cipher while the site is inactive.
CSV File Headers
For the header, two different formats are supported, a short header and a long header, each of which are explained below.
A short header consists of 7 rows of attributes followed by an empty row, with the first data record being in row 9. The first 7 rows of the header must be formatted as follows:
Row | Row Label | Purpose |
|---|---|---|
1 | Site Name | The name of the site from which each quantity was ingested. Trailing spaces in site names will be removed when the data is imported. |
2 | ISCO Quantity | The quantity type for each value in the data records. The supported quantity types are provided below. |
3 | Label | The label that will be displayed for the dataset when viewed in Flowlink Cipher. |
4 | Units | The units for the values in the data records, which are recorded as abbreviations. The supported abbreviations are provided below. |
5 | Resolution | The number of digits to the right of the decimal point in quantity values. This setting is not used during import. All significant digits are imported from the CSV file. |
6 | Significant Digits | The number of significant digits to the right and the left of the decimal point in timestamps. This setting is not used during import. All significant digits are imported from the CSV file. |
7 | Device Name | The name of the device that collected the data for a quantity value. This will be included in the information found on the Devices tab of the Site Information page when viewing the site associated with the dataset. If this setting is not populated, it will default to "Imported" in the resulting dataset. |
8 | null | |
9 | The first data record in the CSV file. | |
The image below shows an example of a CSV file with a short header.
A long header consists of the same rows as a short header as well as an additional 4 rows, with the first data record being in row 13:
Row | Row Label | Purpose |
|---|---|---|
1-7 | Short header rows | |
8 | Device Serial Number | The serial number of the device that collected the data for a quantity value. This will be included in the information found on the Devices tab of the Site Information page when viewing the site associated with the dataset. |
9 | FlowCon | This setting is not used. |
10 | FlowConDetails | This setting is not used. |
11 | ISCO Data Type ID | This setting is not used. The quantity types for each value in the data records are identified by row 2. |
12 | null | |
13 | The first data record in the CSV file. | |
The image below shows an example of a CSV file with a long header.
CSV Data Records
The data records in a CSV file must follow specific formatting in order for Flowlink Cipher to successfully import the data. The formatting rules are:
• Date Time must be the first value in each data record, followed by a comma.
• Each data record must contain the same number of commas as there are quantities defined in row 2 of the header.
• Consecutive commas must be used to represent a quantity without a value.
• Each quantity value must follow a comma.
• Date Time values must use one of the following formats:
Format | Sample | |
|---|---|---|
24 Hour Clock | MM/DD/YYYY HH:MM:SS | 01/01/2020 23:59:59 |
1/1/2020 23:59:59 | ||
MM/DD/YYYY HH:MM | 01/01/2020 23:59 | |
1/1/2020 23:59 | ||
12 Hour Clock | MM/DD/YYYY HH:MM:SS AM/PM | 01/01/2020 12:00:00 AM |
1/1/2020 11:59:59 PM | ||
MM/DD/YYYY HH:MM AM/PM | 01/01/2020 12:00 AM | |
1/1/2020 11:59 PM | ||
Supported Quantities and Units
The following table lists the quantity types supported for row 2 in a CSV file.
Alkalinity | Flag | pH | TDS |
Ammonia | Flow Rate | Phosphate | Temperature |
Ammonium | Fluorescence | Potassium | TOC |
Analog Percent | Frequency | PR | Total Organic Carbon |
Angle | Generic | Pressure | Transmissivity |
Barometric Pressure | Input Voltage | Rainfall | TSS |
Battery Voltage | Lead | Rainfall Intensity | Turbidity |
BOD | Level | Relative Humidity | Velocity |
Calcium | Load | Resistivity | Volume |
Carbon Monoxide | Magnesium | SAC | Wind Direction |
Chloride | Manganese | Salinity | Wind Speed |
Chlorine | Mass | Sludge Index | Wireless Signal |
Chlorophyll | Mercury | Sludge Volume | Percent |
COD | N0x | Sodium | Flag Event |
Color | Nitrate | Solar Radiation | Program Report Event |
Conductivity | Nitric Oxide | Specific Conductance | History Event |
Current | Nitrogen Dioxide | Sulfur Dioxide | Summary Report Event |
Dissolved Oxygen | ORP | Suspended Particulates | Diagnostic Event |
Elapsed Time | Ozone | TDG |
The following table lists the unit abbreviations supported for row 4 in a CSV file.
Unit | Abbreviation | Unit | Abbreviation | Unit | Abbreviation |
|---|---|---|---|---|---|
AcreFeet | af | Generic | Gen | MillimetersPerHour | mm/hr |
AcreFeetPerDay | afd | GramsPerLiter | g/l | MillionGallons | mgal |
Amps | A | Hazen | HU | MillionGallonsPerDay | mgd |
Bar | bar | HU | HU | MilliSiemensPerCentimeter | mS/cm |
Celsius | °C | Hertz | Hz | Millivolts | mV |
CubicFeet | cf | Inches: | in | NTU | NTU |
CubicFeetPerDay | cfd | InchesPerHour | in/hr | OhmMeter | Ohm-m |
CubicFeetPerHour | cfh | Kilograms | Kg | OneOverMeter | m^1 |
CubicFeetPerMinute | cfm | KilogramsPerHour | Kg/h | Pa | Pa |
CubicFeetPerSecond | cfs | Liter | l | PartsPerMillion | ppm |
CubicMeters | m3 | LitersPerHour | l/h | PartsPerThousand | ppt |
CubicMetersPerDay | m3d | LitersPerMinute | l/m | Percent | % |
CubicMetersPerHour | m3h | LitersPerSecond | l/s | pH | pH |
CubicMetersPerMinute | m3m | Meters | m | PoundsPerHour | lbs/h |
CubicMetersPerSecond | m3s | MetersPerSecond | m/s | PRU | PRU |
Degrees | ° | MetersPerHour | m/h | PSIAbsolute | psia |
Fahrenheit | °F | MicrogramsPerCubicMeter | ug/m3 | PSIGuage | psig |
Feet | ft | MicrogramsPerLiter | ug/l | PSIRelative | psir |
FeetPerSecond | ft/s | MicroSiemensPerCentimeter | uS/cm | RFU | RFU |
Flag | Flag | MilliAmps | mA | SRU | SRU |
FlagEvent | FlagEvent | MilliEquivalantPerLiter | mEq/L | SampleUnits | SU |
Gallons | gal | MilligramsPerLiter | mg/l | Seconds | s |
GallonsPerHour | gph | MillilitersPerGram | ml/g | SquareMetersPerDay | m2/day |
GallonsPerMinute | gpm | MillilitersPerLiters | ml/L | Volts | volts |
GallonsPerSecond | gps | Millimeters | mm | WattsPerSquareMeter | W/m2 |