Converting byte array to json in Compojure

Question

I'm working on a Compojure application that serves json files. I'm using clojure.java.jdbc library to retrieve the files from sqlite database:

(defn grid-result [z x y]
  (j/with-connection db-specs
    (j/with-query-results results ["SELECT grid FROM grids WHERE zoom_level = ? AND tile_column = ? AND tile_row = ?" z x y]
      (doall results))))

The result of the query is this:

({:grid #<byte[] [B@7e88dd33>})

In order to display it as json, I update the headers, and strip the query result to byte array and use ByteArrayInputStream:

(defn grid-response [grid]
  {:status 200
   :headers {"Content-Type" "application/json"}
   :body (new java.io.ByteArrayInputStream (:grid (first grid)))})

This is how the request is handled:

(defroutes app-routes
  (GET "/api/:z/:x/:y.grid.json" [z x y] (grid-response (grid-result z x y))
  (route/not-found "Page not found"))

All above results in:

"xœ«VJ/ÊLQ²ŠVR (éŒ1jÄ¨£FŒ1jÄ¨£FŒ1jÄ¨ÄEÊPP¨¨ªQl„
Å®€ƒQ#F5b¤«£”ZYÈ�gh`jndaXƒf†æ0†Œa¨[�7ð+k"

Json that I'm trying to retrieve has a format of:

grid({"keys": ["""], "data": {"105803": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UFGH0011"}, "106178": {"predicti_9": 0.0265, "prediction": "6B1", "OA01CDOLD": "15UHFE0001"}, "106171": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UHFC0001"}, "105721": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UFGC0013"}, "106170": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UHFB0001"}}, "grid": ["  ", "  "]});

What I've also tried:

Changed grid-response function to convert byte array to String:

:body (String. (:grid (first grid)) "UTF-8")

Results in ((with UTF-8 or without):

x??VJ/?LQ??VR?(?1j??F?1j??F?1j??E?PP????Ql?
???Q#F?5b?????ZY?�gh`jndaX?f??0??a?[�7?

Mapped each char to str:

(apply str (map #(char (bit-and % 255)) (:grid (first grid))))

Results in:

xœ«VJ/ÊLQ²ŠVR (éŒ1jÄ¨£FŒ1jÄ¨£FŒ1jÄ¨ÄEÊPP¨¨ªQl„
Å®€ƒQ#F5b¤«£”ZYÈ�gh`jndaXƒf†æ0†Œa¨[�7ð+k

(Same as with ava.io.ByteArrayInputStream).

Any advice on how to convert that byte stream to json will be greatly appreciated.

Blob is a utfgrid encoded as UTF-8. On mbtiles-spec it reads that the UTFGrid data is gzip deflated. — Anna Pawlicka
– Anna Pawlicka, Commented Oct 1, 2013 at 19:37

Anna Pawlicka · Accepted Answer · 2013-10-03 19:17:35Z

4

The grids are compressed with zlib. So this is the solution I used:

(defn zlib-decompress
  [input]
  (with-open [input (-> input io/input-stream InflaterInputStream.)]
    (slurp input)))

This function returns a string representation of the json, so when I set the response's content type to "application/json", everything works.

answered Oct 3, 2013 at 19:17

Anna Pawlicka

7577 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jared314 Over a year ago

Since, in your example, you are just passing the data to the client, did you try setting the Content-Encoding http header to match the data (like gzip)?

Anna Pawlicka Over a year ago

I tried both gzip and deflated. It didn't work. But using the InflaterInputStream as above worked.

astine · Accepted Answer · 2013-10-02 14:06:57Z

2

You're trying to read GZIP compressed data. You can try using something like:

(java.io.BufferedInputStream.
 (java.util.zip.GZIPInputStream.
  (new java.io.ByteArrayInputStream (:grid (first grid)))))

to decompress it before displaying it.

answered Oct 2, 2013 at 14:06

astine

2863 silver badges9 bronze badges

Collectives™ on Stack Overflow

Converting byte array to json in Compojure

Changed grid-response function to convert byte array to String:

Mapped each char to str:

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Changed grid-response function to convert byte array to String:

Mapped each char to str:

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related