Soracom Product Updates logo

Product Updates

Soracom is IoT connectivity with powerful APIs that enable you to build world class applications

Subscribe to Updates
  • 日本語

Labels

  • All Posts
  • Feature
  • Improvements
  • Devices
  • Announcement
  • Beta
  • Deprecation
  • End of Support

Jump to Month

  • May 2025
  • April 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • March 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021
  • May 2021
Feature
3 years ago

SORACOM Harvest Data Download format change notice

When using SORACOM Harvest to capture and store data from your devices, you can easily view and download the data directly from the User Console. This update introduces several changes when downloading data in order to improve precision and support additional features.

This update will take effect on September 21.

Note that these changes apply only to the following options in the Soracom User Console:

  • Downloading data using the "Download content as JSON" option
  • Downloading data using the "Download chart data as JSON" option
  • Downloading data using the "Download chart data as CSV" option

This update does not affect the response format returned when retrieving data using the Harvest Data APIs.

Date and time format

Currently, the date and time that corresponds to each data entry is output using the format Sep, 16, 2021 13:45, with minute precision.

With this update, the date and time format will be changed to the ISO 8601 format, and will include both millisecond precision and time zone information, in order to provide more precision and to be easier for various applications to process. The new output will use the format 2021-09-16T13:45:59.123-08:00.

An exception to this is the "Download chart data as CSV" option, where the original date and time format will also be included in order to maintain compatibility with spreadsheet applications:

  • The __time column will follow the original date and time format, and will be updated to include second precision using the format Sep, 16, 2021 13:45:59.
  • A __iso8601Time column will be added to the CSV and will use the ISO 8601 format of 2021-09-16T13:45:59.123-08:00.

Simple Location Tracking data

Simple Location Tracking (in Japanese) is an optional service for plan-KM1 SIMs that determines the approximate geographic location of a device based on its cellular connection, without the device needing its own GPS or Wi-Fi positioning functionality.

While the longitude and latitude data provided by Simple Location Tracking is automatically combined with other data uploaded by a device to Harvest Data (as well as other Soracom services), this location data was not included when using the "Downloading content as JSON" option.

With this update, the location data follow the same format as the Harvest Data API response format (in Japanese), and will be included as a metadata property for each data entry.

Remove contentType property

With this update, the contentType property previously included when downloading data using the "Download chart data as JSON" and "Download chart data as CSV" options will no longer be included.

Data format samples

When downloading data using the "Download content as JSON" option, the new format will look like this:

[
  {
    "resourceType":"Subscriber",
    "resourceId":"295050012345678",
    "time":"2021-09-16T13:45:59.123-08:00",
    "contentType":"application/json",
    "content":{
      "clickType":2,
      "clickTypeName":"DOUBLE",
      "batteryLevel":1,
      "binaryParserEnabled":true
      //...other data from device
    },
    "metadata":{
      "locationQueryResult":"success",
      "location":{
        "lat": "47.6062",
        "lon": "-122.3321"
      }
    }
  },
  // ...other data entries
]

When downloading data using the "Download chart data as JSON" option, the new format will look like this:

[
  {
    "resourceType":"Subscriber",
    "resourceId":"295050012345678",
    "time":"2021-09-16T13:45:59.123-08:00",
    "chartData":{
      "clickType":2,
      "batteryLevel":1,
      "$metadata.location.lat":"47.6062",
      "$metadata.location.lon":"-122.3321"
      //...other data from device
    }
  },
  // ...other data entries
]

When downloading data using the "Download chart data as CSV" option, the new format will look like this:

__resourceType,__resourceId,__time,__iso8601Time,$metadata.location.lat,$metadata.location.lon,batteryLevel,clickType
"Subscriber","295050012345678","Sep, 16, 2021 13:45:59","2021-09-16T13:45:59.123-08:00","47.6062","-122.3321","1","2"
...