Bulk Data Management API Integration

Overview

This article describes how to integrate with the Bulk Data Management API to import a file and track its processing.

The Bulk Data Management API offers the ability to query file information, retrieve original files (as well as error files that were generated as a result of an import error), and upload files to import into the system. This article will describe how to integrate with the API to upload and import a file, as well as query the file’s import processing status.

Upload A New Import File

To upload a file, issue an HTTP POST to the api/file endpoint (https://api.idibilling.com/bulkdatamanagement/2x/{environment}/api/file ). The request should have a Content-Type of multipart/form-data, which allows transmission of not only the actual file contents but other metadata that is required to import the file.

Request Parameters

The request body must specify the following parameters, all of which are required.

Note: If the fileFormatID specified is not valid, the file will receive a status of “Unrecognized File Format” and will fail to process.

Response

On a successful call to upload a file, the API will return a status code of 200 OK, and the response body will contain information about the file that you should save for later. Specifically, it will return a fileID, which is a unique identifier for the import file in the Bulk Data system.

Here is a sample response from the API (in JSON):

{
    "fileID": 328,
    "createDate": "2019-08-13T16:38:12.7659151-04:00",
    "createUser": "myuser",
    "fileFormatID": 5,
    "fileSize": 255,
    "fileStatusID": 1,
    "name": "my-import-file.csv"
}

Sample Code

The following is sample code of one way to make a request to the Bulk Data Management API to upload a file.

private async Task<WebResponse> UploadBulkDataImportFile(string environment, string simpleWebToken, string fileName, int fileFormatID, Stream fileStream, string fileContentType)
{
    Uri uri = new Uri(string.Format("https://api.idibilling.com/bulkdatamanagement/2x/{0}/api/file", environment));
    var request = HttpWebRequest.Create(uri) as HttpWebRequest;

    request.Headers.Add("Authorization", "WRAP access_token=\"" + simpleWebToken + "\"");

    request.Method = "POST";
    request.Accept = "application/json";
    var boundary = "---------------------------" + DateTime.Now.Ticks.ToString("x");
    request.ContentType = "multipart/form-data; boundary=" + boundary;

    using (var requestStream = request.GetRequestStream())
    {
        boundary = "--" + boundary;

        // file name
        var buffer = Encoding.ASCII.GetBytes(boundary + System.Environment.NewLine);
        requestStream.Write(buffer, 0, buffer.Length);
        buffer = Encoding.ASCII.GetBytes(string.Format("Content-Disposition: form-data; name=\"{0}\"{1}{1}", "fileName", System.Environment.NewLine));
        requestStream.Write(buffer, 0, buffer.Length);
        buffer = Encoding.UTF8.GetBytes(fileName + System.Environment.NewLine);
        requestStream.Write(buffer, 0, buffer.Length);

        // file format id
        buffer = Encoding.ASCII.GetBytes(boundary + System.Environment.NewLine);
        requestStream.Write(buffer, 0, buffer.Length);
        buffer = Encoding.ASCII.GetBytes(string.Format("Content-Disposition: form-data; name=\"{0}\"{1}{1}", "fileFormatID", System.Environment.NewLine));
        requestStream.Write(buffer, 0, buffer.Length);
        buffer = Encoding.UTF8.GetBytes(fileFormatID + System.Environment.NewLine);
        requestStream.Write(buffer, 0, buffer.Length);

        // file
        buffer = Encoding.ASCII.GetBytes(boundary + System.Environment.NewLine);
        requestStream.Write(buffer, 0, buffer.Length);
        buffer = Encoding.UTF8.GetBytes(string.Format("Content-Disposition: form-data; name=\"{0}\"; filename=\"{1}\"{2}", "File", fileName, System.Environment.NewLine));
        requestStream.Write(buffer, 0, buffer.Length);
        buffer = Encoding.ASCII.GetBytes(string.Format("Content-Type: {0}{1}{1}", fileContentType, System.Environment.NewLine));
        requestStream.Write(buffer, 0, buffer.Length);
        fileStream.CopyTo(requestStream);
        buffer = Encoding.ASCII.GetBytes(System.Environment.NewLine);
        requestStream.Write(buffer, 0, buffer.Length);

        var boundaryBuffer = Encoding.ASCII.GetBytes(boundary + "--");
        requestStream.Write(boundaryBuffer, 0, boundaryBuffer.Length);
    }

    var response = await request.GetResponseAsync();

    return response;
}

Query Import File Status

To get information about the file just uploaded, issue an HTTP GET to the api/file/{id} endpoint (https://api.idibilling.com/bulkdatamanagement/2x/{environment}/api/file/{id} ). The {id} parameter is the fileID value that was returned in the response from uploading the file. The response of this call will contain information about the file, including details about its processing status.

The fileStatusID (and its human-readable partner, fileStatus) will indicate the status of the processing of the file. A full list of file statuses can be obtained from the the api/fileStatus endpoint (https://api.idibilling.com/bulkdatamanagement/2x/{environment}/api/fileStatus ).

If a fatal error is encountered during processing, the statusMessage property will contain a description of the error that occurred.

Retrieving Error Information

Individual file records can error for any number of reasons, dependent upon the type of file being imported. (For example, a customer being imported could fail if it is missing a piece of information required to create and/or modify a customer.) When one ore more individual file records results in an error status, an error file will be generated.

Note: Querying the api/file/{id} endpoint will return the number of records that resulted in in an error status in the errorRecords property.

The error file can be downloaded by issuing an HTTP GET to the api/file/{id}/errorFile endpoint (https://api.idibilling.com/bulkdatamanagement/2x/{environment}/api/file/{id}/errorFile ). This endpoint will return a file that contains the following:

• The first row of the file will be the unique code identifying the file format. This will be the same as the original file imported.
• The second row of the file will be the column header row, but will contain a column for all fields defined by the file format. In addition, the first column will be a special column that will contain the error message for this row. This may differ from the original file imported, since the original file is not required to contain every single field defined by the file format.
• The rest of the file will contain one row for each record that resulted in an error, and the value of the first column will be a message indicating the error that occurred.

Because of the way the error file is created, it is guaranteed to be a file that is valid for the file format specification. This means that the error file can be downloaded, the error(s) fixed within that file itself, and then directly re-uploaded.