Using Nlog logger with ECS layout, in kibana the json object appear as a string instead of multiple properties

Question

I'm working on some .Net framework application and i've been asked to send the logs to elasticsearch using kibana as the UI. To have something that is standardized i have to implement ECS (Elastic Common Schema).

Looking at the example we have on ECS github we only have to implement it on the following way:

Instead of sending to console, like we have on the example i send it to elastic search

The output from it, would be a nice Json object...

Maybe it is expected that on kibana we would see something like the following (Kibana - Discover):

Looking at that, probably the Json Object is supposed to be treated as a string and everything goes inside the message property, but that is not what i'm looking for, i want that json to be divided in many properties.

Since i'm new to Elastic stack world, i've tried to create a template inside the Index Management page and the performing there manual mappings like message._metadata.url to not treat some properties as part of the string but without success.

I'm having trouble finding useful information to solve this problem, can anyone give an hint?

UPDATE:

I found the property enableJsonLayout="true"that we can put on the target of Nlog that indeed turns whats on the Json layout as properties on ElasticSearch which is good.

Is this the right way to use ECS? How can i add aditional properties?

Rolf Kristensen · Accepted Answer · 2020-11-19 17:56:33Z

2

When you enable this enableJsonLayout="true" then it means that the configured Layout has to handle everything. For EcsLayout then you can find the documentation here:

https://github.com/elastic/ecs-dotnet/tree/master/src/Elastic.CommonSchema.NLog

EcsLayout will by default include all LogEvent Properties as metadata. See also https://github.com/NLog/NLog/wiki/How-to-use-structured-logging

But you can explicit add extra metadata-items:

<layout xsi:type="EcsLayout">
   <metadata name="MyProperty" layout="MyPropertyValue" />
</layout>

answered Nov 19, 2020 at 17:56

Rolf Kristensen

20.2k2 gold badges59 silver badges87 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Nmaster88 Over a year ago

Cool, you're a lifesaver that metadata property data works! Where did you found that we can add that extra metadata items? I looked at some of the EcsLayout and Nlog code and documentation and did not found that.

Rolf Kristensen Over a year ago

@Nmaster88 Created pull-request to update documentation. Right now you can get a preview here: github.com/snakefoot/ecs-dotnet/tree/master/src/…

Nmaster88 Over a year ago

Thanks for the PR, i see now that probably this attribute [ArrayParameter(typeof(TargetPropertyWithContext), "metadata")] is what allows this props to be added on the xml. Btw is it possible to know what are the objective of each one? metadata i already see that is suposed to add new props inside the metadata object, but the others labels and tags i can only guess.

Nmaster88 Over a year ago

I noted that these metadata props, that i add through the web.config always appear even when they have no value in elasticsearch, which means that on kibana i see empty fields, is it possible to change this behaviour? not send the prop when it is not needed.

Rolf Kristensen Over a year ago

@Nmaster88 "Labels" and "Tags" are part of the ECS-schema: elastic.co/guide/en/ecs/current/ecs-base.html

Collectives™ on Stack Overflow

Using Nlog logger with ECS layout, in kibana the json object appear as a string instead of multiple properties

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related