Harvest Data maximum data entry retrieval limit updated
When sending device data to Harvest Data, data can be easily retrieved from the User Console or using the Soracom API/CLI. With this update, the amount of data that can be retrieved at a time has been updated as follows.
Using the User Console
When viewing Harvest Data from the User Console, device data is retrieved automatically according to the selected resource and time period. You can download the data by clicking the Download button. Device data will be downloaded in 250MB increments. If the amount of data exceeds 250MB, click the Resume button to continue downloading subsequent increments.
For additional information on viewing and downloading data from the User Console, please refer to the Harvest Data documentation.
Using the Soracom API or CLI
Harvest Data can also be downloaded using the DataEntry:getDataEntries API or the data get-entries
CLI command.
With this update, the maximum number of data entries that can be returned per request has been updated to 1,000. You can use the limit
parameter to specify how many data entries to return, however please note that setting a limit
higher than 1,000 will have no effect, as the number of data entries that will be returned by the API will still be limited to 1,000 entries per request.
If there are more than 1,000 data entries, the API response will include a link
header that specifies the URL to use for retrieving the next increment of data entries. You can then repeat each API call until the response no longer includes the link
header in order to retrieve all of the data entries. For more details on how to use this "pagination," please refer to the Handling Pagination documentation.
When using the Soracom CLI, you can use the --fetch-all
option in order to perform the above pagination automatically.