Importing Data Using the Data Import Tool (DIT)

This article explains the steps involved in using the Data Import Tool.

For an overview of the Data Import Tool (DIT), see Accessing the Data Import Tool (DIT).

File Format

The Data Import Tool (DIT) accepts data in HMIS CSV or XML format.

For CSV file specifications, please see the Format Specifications .

For XML files, the Clarity XML Schema is used for files not containing custom fields. 

If you're importing HMIS XML files, the DIT allows the XML schema to be expanded to include fields outside of the standard HMIS XML schema. For more information see the "Custom Fields" section of Accessing the Data Import Tool (DIT). Users can import single files, a ZIP file containing one or more files, or an entire folder containing one or more files. When uploading a compressed ZIP file, the importer expects the ZIP only to contain CSV or XML files.  Any extraneous files, folders, or subfolders will cause an error suggesting to review the file structure within the compressed file before attempting another upload.

Minimal Set of HMIS CSV Files Within a Zip File for DIT Import

The following CSV files are required in order to import client records (and no other data) using the DIT:

• Client.csv: include the client record(s), as usual, in this file.

• Enrollment.csv, Exit.csv, and Project.csv: these files must exist at a minimum with the header row, but record data is optional. If enrollment records are also sent, then there is a caveat:

  • If there are records within Enrollment.csv, then the Project.csv file has to have corresponding project records for each project that the enrollment records reference, so the DIT can map the enrollments to Clarity projects.

• Export.csv: include the standard Export.csv record in this file. 

A sample file ("Minimal HMIS CSV File (one enrollment record)") is available in our Data Import Tool (DIT) Sample Files article.

Maximal Set of HMIS CSV Files Within a Zip File For DIT Import

The DIT imports everything in the current FY24 HMIS CSV Spec (currently at v1.4 at the time this was published), except the following files which are completely ignored if transmitted:

• AssessmentResults.csv

• CEParticipation.csv

• EnrollmentCoC.csv

• Funder.csv

• HMISParticipation.csv

• Inventory.csv

• Organization.csv

• User.csv

Minimal Set of Data Elements Within a Clarity XML File

Clarity XML (which extends HMIS XML), generally has more stringent DIT validation than HMIS CSV (except CSV has stronger date validations presently). Clarity XML won't validate unless it also contains these top-level elements, with any required sub-elements with the IDs referenced elsewhere in the XML. These are:

• Organization

• Project

• User

A sample file ("Clarity XML Single Client") is available in our Data Import Tool (DIT) Sample Files article.

Maximal Set of Data Elements Within a Clarity XML File

For the subset of FY24 HMIS Data Standard elements in Clarity XML, data element processing in the DIT is the same as for HMIS CSV. That is, maximal for Clarity XML contains everything in the current FY24 HMIS XML Spec, except the following data elements are not imported if transmitted:

• AssessmentResults

• CEParticipation

• EnrollmentCoC

• Funder

• HMISParticipation

• Inventory

• Organization

• User

Uploading a File

Once a file has been added to the DIT queue, the file name, and file size will be displayed. Click START UPLOAD to begin the upload process or click CANCEL to remove the file from the queue.

File Validation

Once the file is uploaded from the queue, the next step is validation. During this step, the system ensures the file(s) adhere to data formatting specifications. To begin the validation process, click START VALIDATION.  

Once validation is complete, the page displays information about the file(s) and a section to map Program, Service, and Assessment data.

The Data Analysis section displays a total count of each type of record within the file(s). Unduplicated Programs, Services, and Assessments appear below. 

Mapping Programs, Services, and Assessments

The DIT allows users to map source records from the imported file(s) to existing destination records within Clarity Human Services. Establishing mapping relationships is key to ensuring data quality. You can filter results on the source Organization or Program, if applicable. The available options for mapping are based on your agency (or the agency you're switched into). Please note that if Programs, Services, and Assessments are not mapped here, the associated records will not be included in the import.

For each data type, choose the Clarity Human Services Program, Service, or Assessment to map the data to.

Click the SAVE for each section once you've associated the source data with a Clarity Human Services record.

Mapping Coordinated Entry Events 

Coordinated Entry Events can be imported with the DIT within the Program Event Mapping interface. The program mapping for the imported event(s) should be selected first. After program mapping, select the Coordinated Entry Event within the applicable program where program_event_category will be imported.

A few caveats to note prior to importing and mapping the file(s) are listed below. 

• Manual CE Event imports are supported.
• CE Events must be set up at the program level prior to mapping/importing data with the DIT for the mapping options to display as intended. 
• Imported data will appear as event transactions as opposed to services within Clarity Human Services. 

Importing the File(s)

Once the mapping is complete, the data can be imported into Clarity Human Services. Click IMPORT to start the import.

 

The number of processed records is displayed during the import. Click CANCEL to abort the import at any time.

 

When the import completes, the import will display as Imported.

Viewing the Import Results

Click SHOW RESULTS at the bottom of the page to see how the data was imported into Clarity Human Services.

The Message column informs you how the DIT handled each record. If an error occurs, details on the error will be displayed, allowing you to resolve the error on a subsequent import attempt.

 

Updated: 07/23/2024