I have an IoT device that logs data every 3 minutes and uploads that data to Azure Table Storage every 10 minutes. This means the table entities have the built in Timestamp property and a date property. Sadly, date is stored as a string and not as an actual datetime. Timestamp is automatically generated as the time Azure got the data, but date is the timestamp for when the device recorded the data. This means that several entities can have the same timestamp, but different dates (actual data):
Timestamp date tagName tagValue
2025-04-03T18:20:19.2764867Z 2025-04-03T18:19:55Z LEVEL 52.2
2025-04-03T18:20:19.2754824Z 2025-04-03T18:11:14Z LEVEL 52.2
2025-04-03T18:20:19.2754824Z 2025-04-03T18:15:35Z LEVEL 52.2
2025-04-03T18:10:20.8985209Z 2025-04-03T18:02:54Z LEVEL 52.2
2025-04-03T18:00:19.3550008Z 2025-04-03T17:58:33Z LEVEL 52.2
2025-04-03T18:00:19.3550008Z 2025-04-03T17:54:13Z LEVEL 52.2
2025-04-03T17:50:20.126223Z 2025-04-03T17:45:52Z LEVEL 52.2
2025-04-03T17:50:20.126223Z 2025-04-03T17:49:53Z LEVEL 52.2
My goal is to query the table for the most recent level reading using the REST API. My plan to accomplish this is to get all entities uploaded in the last hour then sort through them to find the most recent date property. I've tried two approaches:
This URL returns data 90% of the time:
https://mystorage.table.core.windows.net/HistoryValues?sv=2019-02-02&st=2025-04-02T15%3A33%3A02Z&se=2030-04-03T15%3A33%3A00Z&sp=r&sig=<mysig>&tn=HistoryValues&$filter=Timestamp%20ge%20datetime%272025-04-02T20:33:02Z%27%20and%20tagName%20eq%20%27LEVEL%27&$select=tagValue,date
This one never returns data:
https://mystorage.table.core.windows.net/HistoryValues?sv=2019-02-02&st=2025-04-02T15%3A33%3A02Z&se=2030-04-03T15%3A33%3A00Z&sp=r&sig=<mysig>&tn=HistoryValues&$filter=date%20ge%20%272025-04-02T20:33:02Z%27%20and%20tagName%20eq%20%27TOWER_LEVEL%27&$select=tagValue,date
Both return HTTP status 200, but only the first query returns entities. The second does not return any entities. If put the filter date ge '2025-04-02T20:33:02Z' and tagName eq 'TOWER_LEVEL' in Azure Storage Explorer, it currently returns 312 entities.
The simple solution seems to be to filter one the Timestamp. I ran the function that queries by timestamp every 3 minutes overnight, and every 30 minutes no entities are returned. At 5:03 -> data, 5:06 -> data, 5:09 -> data, etc. until 5:30 when 0 entities were returned. At 5:33 I got data. This continued: 5:36 -> data, 5:39 -> data, ..., 5:57 -> data, but at 6:00 zero entities were returned again. But 6:03, 6:06, 6:09, etc all work as expected until 6:30 when I got no data. This pattern held all night long. No data was ever returned at the top and bottom of the hour. This is why I thought I would try to filter by date, but this is not working at all.
Why would the Timestamp query fail every 30 minutes even though there is clearly data that matches the query? Why does my date filter work in Storage Explorer but not in the REST query? Any advice on how to achieve the goal of getting the most recent level reading when date is stored as a string?



$filter=date%20ge%20%272025-04-03T11:57:24Z%27returned the same data in the browser as it did in Storage Explorer. Two things I find interesting::does not have to be url encoded (both ways work) and the table does not need()(both mystorage.table.core.windows.net/HistoryValues and mystorage.table.core.windows.net/HistoryValues() work) despite the documentation saying otherwise.Timestampfield to fetch entities from the past 65 minutes to avoid issues during write times like :00 and :30. Filter by tagName = 'LEVEL' in the REST query. Filter the results by the 'date' field is a string type on the client side by converting it to Date objects.