Returning MarkLogic EVAL REST service output as JSON

Question

I am working on a demo using MarkLogic to store emails exported from Outlook as XML, so that they stay searchable and accessible when I move away from Outlook.

I am using an AngularJS front-end calling either the native MarkLogic REST services of own REST services written in JAVA using Jersey.

MarkLogic SEARCH REST service works very well to get back a list of references to documents based on various search criteria, but I also want to display information stored inside the found documents.

I would like to avoid multiple REST calls and to get back only the needed information, so I am trying to use the EVAL REST service to run an xQuery.

It works well to get XML back (inside a multipart/mixed message) but I don't seem to be able to get JSON instead which would be much more convenient and is very easy with most other MarkLogic REST services.

I could use "json:transform-to-json()" in my xQuery or transform the XML to JSON in my JAVA code, but that does not look very elegant to me.

Is there a more efficient method to get where I am trying to go ?

Sam Mefford · Accepted Answer · 2016-03-21 14:58:09Z

5

First, json:transform-to-json seems plenty elegant to me. But of course it's not always the right answer.

I see three options you haven't mentioned.

server-side transforms - REST search supports server-side transforms which transform each document when you perform a bulk read by query. Those server-side transforms could generate any json you need.
search extract-document-data - this the simplest way to extract portions of documents. But it seems best if your documents are json to match your json response. Otherwise you get xml in your json response . . . unless you're ok with that.
custom search snippets - another very powerful way to customize what search returns

All of these options don't require the privileges that eval requires, which is a very good thing. Since eval allows execution of arbitrary code on your server, it requires special privileges and should be used with great care. Two other options before you use eval are (1) custom xquery installed in an http server, and (2) REST extensions.

answered Mar 21, 2016 at 14:58

Sam Mefford

2,47512 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Sébastien VINCHON Over a year ago

First of all, many thanks for responding so quickly and with plenty of good information. Sorry for the term "not elegant" that was maybe not the most appropriate. I meant that there had to be a better way than using the EVAL service that is not ideal in terms of security. I have tested using "extract-document-data" but unfortunately,as you explained, it returns a mix of JSON and XML, which is not ideal. I thought that XML and JSON were just two different ways of presenting the same information from the MarkLogic point of view, but apparently it is not that simple.

Sébastien VINCHON Over a year ago

Maybe i should just store data as JSON like in the Samplestack sample. Regard the two other options you suggest: It seems to me a bulk read by query returns each document as an individual part of a multipart/mixed message. Is it the same when applying a transform? Is there a way to get back an single JSON? Would you have a sample? I need to look at custom search snippets too I guess.

Sébastien VINCHON Over a year ago

What would you recommend as the best approach?

Sébastien VINCHON Over a year ago

I have finally chosen to store my data as JSON inside MarkLogic and to use "extract-document-data" since that was the most simple to feed an Angular JS front-end. Many thanks for your responses.

David Ennis -CleverLlamas.com · Accepted Answer · 2016-03-21 21:29:19Z

The answers from Sam are what I would suggest. Specifically I would set a search option for search-extract-document-data (This is a search API option. If you are posting the request, then you can add the option in the XML you post back. If you are using GET, then you need to register the option ahead of time and call it. Relevant URLs to assist:

As for json.. ML8 will transform content. Use the accept-header or just add format=json to your results...

Example - xml which is what my content is stored as:
http://localhost:8000/v1/search?q=watermellon ...

<search:result index="1" uri="/sample/collections/1.xml" path="fn:doc("/sample/collections/1.xml")" score="34816" confidence="0.5982239" fitness="0.6966695" href="/v1/documents?uri=%2Fsample%2Fcollections%2F1.xml" mimetype="application/xml" format="xml">
<search:snippet>
<search:match path="fn:doc("/sample/collections/1.xml")/x">
<search:highlight>watermellon</search:highlight>
</search:match>
</search:snippet>
</search:result>

...
Example - json which is what my content is stored as:
http://localhost:8000/v1/search?q=watermellon&format=json ...

        "index":1,
         "uri":"/sample/collections/1.xml",
         "path":"fn:doc(\"/sample/collections/1.xml\")",
         "score":34816,
         "confidence":0.5982239,
         "fitness":0.6966695,
         "href":"/v1/documents?uri=%2Fsample%2Fcollections%2F1.xml",
         "mimetype":"application/xml",
         "format":"xml",
         "matches":[  
            {  
               "path":"fn:doc(\"/sample/collections/1.xml\")/x",
               "match-text":[  
                  {  
                     "highlight":"watermellon"
                  }
               ]
            }
         ]
      }

...

For real heavy-lifting, you can use server-side transforms as in Sam's description. One note about this. Server-side transformations are not part of the search API, but part of the REST API. Just mentioning it so you have some idea of which tool you are using in each case..

Collectives™ on Stack Overflow

Returning MarkLogic EVAL REST service output as JSON

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related