Skip to content

[azure] Handle DNAT log with policy/rgc/rc/rule attributes #6824

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 5, 2023
Merged

[azure] Handle DNAT log with policy/rgc/rc/rule attributes #6824

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0e77018
handle DNAT log with policy/rgc/rc/rule attributes
dmathieu Jul 5, 2023
b4558e5
add changelog entry
dmathieu Jul 5, 2023
dfecddf
test new dnat rule
dmathieu Jul 5, 2023
0754e7c
bump manifest version
dmathieu Jul 5, 2023
0233e7f
add missing attributes to expected file
dmathieu Jul 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/azure/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
- version: "1.5.24"
changes:
- description: Handle firewall events for DNAT'ed requests with attributes
type: bugfix
link: https://github.com/elastic/integrations/pull/6824
- version: "1.5.23"
changes:
- description: Update Azure Logs screenshot
Expand Down
3 changes: 2 additions & 1 deletion packages/azure/data_stream/firewall_logs/_dev/test/pipeline/test-networkrules-raw.log
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
{"category":"AzureFirewallNetworkRule","operationName":"AzureFirewallNetworkRuleLog","properties":{"msg":"UDP request from 192.168.0.2:51106 to 89.160.20.156:53. Action: Allow. "},"resourceId":"/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01","time":"2022-06-08T13:27:16.9515590Z"}
{"category":"AzureFirewallNetworkRule","operationName":"AzureFirewallNetworkRuleLog","properties":{"msg":"TCP request from 192.168.0.2:49680 to 89.160.20.156:1688. Action: Deny. "},"resourceId":"/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01","time":"2022-06-08T16:51:27.8692050Z"}
{"category":"AzureFirewallNetworkRule","operationName":"AzureFirewallNatRuleLog","properties":{"msg":"TCP request from 192.168.0.2:50306 to 89.160.20.156:3389 was DNAT'ed to 10.0.0.2:3389"},"resourceId":"/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01","time":"2022-06-08T20:40:56.4525380Z"}
{"category":"AzureFirewallNetworkRule","operationName":"AzureFirewallNatRuleLog","properties":{"msg":"TCP request from 192.168.0.2:50306 to 89.160.20.156:3389 was DNAT'ed to 10.0.0.2:3389. Policy: policy-01. Rule Collection Group: DefaultDnatRuleCollectionGroup. Rule Collection: DNAT. Rule: rule"},"resourceId":"/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01","time":"2022-06-08T20:40:56.4525380Z"}
{"category":"AzureFirewallNetworkRule","operationName":"AzureFirewallNetworkRuleLog","properties":{"msg":"TCP request from 192.168.0.2:51890 to 175.16.199.1:443. Action: drop. Signature: 2028816. IDS: JA3 Hash - [Abuse.ch] Possible Tofsee. Priority: 3. Classification: Unknown Traffic"},"resourceId":"/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01","time":"2022-06-08T20:40:56.4525380Z"}
{"category":"AzureFirewallNetworkRule","operationName":"AzureFirewallNetworkRuleLog","properties":{"msg":"TCP request from 192.168.0.2:54854 to 175.16.199.1:1521. Action: alert. Signature: 2102649. IDS: SQL service_name buffer overflow attempt. Priority: 1. Classification: Attempted User Privilege Gain"},"resourceId":"/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01","time":"2022-06-08T20:40:56.4525380Z"}
{"category":"AzureFirewallNetworkRule","operationName":"AzureFirewallNetworkRuleLog","properties":{"msg":"HTTP request from 192.168.0.2:54314 to ocsp.sca1b.amazontrust.com:80. Url: ocsp.sca1b.amazontrust.com. Action: Deny. ThreatIntel: Bot Networks"},"resourceId":"/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01","time":"2022-06-08T20:40:56.4525380Z"}
{"category":"AzureFirewallNetworkRule","operationName":"AzureFirewallNetworkRuleLog","properties":{"msg":"ICMP request from 192.168.0.2: to 175.16.199.1:. Action: alert. Signature: 2100366. IDS: ICMP_INFO PING *NIX. Priority: 3. Classification: Misc activity"},"resourceId":"/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01","time":"2022-06-08T20:40:56.4525380Z"}
{"category":"AzureFirewallNetworkRule","operationName":"AzureFirewallNetworkRuleLog","properties":{"msg":"ICMP request from 192.168.0.2: to 175.16.199.1:. Action: alert. Signature: 2100366. IDS: ICMP_INFO PING *NIX. Priority: 3. Classification: Misc activity"},"resourceId":"/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01","time":"2022-06-08T20:40:56.4525380Z"}
95 changes: 94 additions & 1 deletion ...zure/data_stream/firewall_logs/_dev/test/pipeline/test-networkrules-raw.log-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -341,6 +341,99 @@
"preserve_original_event"
]
},
{
"@timestamp": "2022-06-08T20:40:56.452Z",
"azure": {
"firewall": {
"category": "AzureFirewallNetworkRule",
"operation_name": "AzureFirewallNatRuleLog",
"policy": "policy-01",
"rule_collection_group": "DefaultDnatRuleCollectionGroup"
},
"resource": {
"group": "TEST-FW-RG",
"id": "/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01",
"name": "TEST-FW01",
"provider": "MICROSOFT.NETWORK/AZUREFIREWALLS"
},
"subscription_id": "23103928-B2CF-472A-8CDB-0146E2849129"
},
"cloud": {
"account": {
"id": "23103928-B2CF-472A-8CDB-0146E2849129"
},
"provider": "azure"
},
"destination": {
"address": "89.160.20.156",
"as": {
"number": 29518,
"organization": {
"name": "Bredband2 AB"
}
},
"geo": {
"city_name": "Linköping",
"continent_name": "Europe",
"country_iso_code": "SE",
"country_name": "Sweden",
"location": {
"lat": 58.4167,
"lon": 15.6167
},
"region_iso_code": "SE-E",
"region_name": "Östergötland County"
},
"ip": "89.160.20.156",
"nat": {
"ip": "10.0.0.2",
"port": 3389
},
"port": 3389
},
"ecs": {
"version": "8.3.0"
},
"event": {
"category": [
"network"
],
"kind": "event",
"original": "{\"category\":\"AzureFirewallNetworkRule\",\"operationName\":\"AzureFirewallNatRuleLog\",\"properties\":{\"msg\":\"TCP request from 192.168.0.2:50306 to 89.160.20.156:3389 was DNAT'ed to 10.0.0.2:3389. Policy: policy-01. Rule Collection Group: DefaultDnatRuleCollectionGroup. Rule Collection: DNAT. Rule: rule\"},\"resourceId\":\"/SUBSCRIPTIONS/23103928-B2CF-472A-8CDB-0146E2849129/RESOURCEGROUPS/TEST-FW-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/TEST-FW01\",\"time\":\"2022-06-08T20:40:56.4525380Z\"}",
"type": [
"connection"
]
},
"network": {
"iana_number": "6",
"transport": "tcp"
},
"observer": {
"name": "TEST-FW01",
"product": "Network Firewall",
"type": "firewall",
"vendor": "Azure"
},
"related": {
"ip": [
"192.168.0.2",
"89.160.20.156",
"10.0.0.2"
]
},
"rule": {
"name": "rule",
"ruleset": "DNAT"
},
"source": {
"address": "192.168.0.2",
"ip": "192.168.0.2",
"port": 50306
},
"tags": [
"preserve_original_event"
]
},
{
"@timestamp": "2022-06-08T20:40:56.452Z",
"azure": {
Expand Down Expand Up @@ -661,4 +754,4 @@
]
}
]
}
}
1 change: 1 addition & 0 deletions packages/azure/data_stream/firewall_logs/elasticsearch/ingest_pipeline/default.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ processors:
- "^%{DATA:azure.firewall.proto} request from %{IPORHOST:source.address}:%{NUMBER:source.port:long} to %{IPORHOST:destination.address}:%{NUMBER:destination.port:long}. Action: %{DATA:azure.firewall.action}. Policy: %{DATA:azure.firewall.policy}. Rule Collection Group: %{DATA:azure.firewall.rule_collection_group}. Rule Collection: %{DATA:rule.ruleset}. Rule: %{DATA:rule.name}$"
- "^%{DATA:azure.firewall.proto} Type=%{DATA:azure.firewall.icmp.request.code} request from %{IPORHOST:source.address} to %{IPORHOST:destination.address}. Action: %{DATA:azure.firewall.action}. $"
- "^%{DATA:azure.firewall.proto} request from %{IPORHOST:source.address}:%{NUMBER:source.port:long} to %{IPORHOST:destination.address}:%{NUMBER:destination.port:long} was DNAT'ed to %{IP:destination.nat.ip}:%{NUMBER:destination.nat.port:long}$"
- "^%{DATA:azure.firewall.proto} request from %{IPORHOST:source.address}:%{NUMBER:source.port:long} to %{IPORHOST:destination.address}:%{NUMBER:destination.port:long} was DNAT'ed to %{IP:destination.nat.ip}:%{NUMBER:destination.nat.port:long}. Policy: %{DATA:azure.firewall.policy}. Rule Collection Group: %{DATA:azure.firewall.rule_collection_group}. Rule Collection: %{DATA:rule.ruleset}. Rule: %{DATA:rule.name}$"
- "^%{DATA:azure.firewall.proto} request from %{IPORHOST:source.address}:(%{NUMBER:source.port:long})? to %{IPORHOST:destination.address}:(%{NUMBER:destination.port:long})?. Action: %{DATA:azure.firewall.action}. Signature: %{DATA:rule.id}. IDS: %{DATA:rule.name}. Priority: %{NUMBER:event.risk_score:long}. Classification: %{DATA:rule.category}$"
- "^%{DATA:azure.firewall.proto} request from %{IPORHOST:source.address}:%{NUMBER:source.port:long} to %{IPORHOST:destination.address}:%{NUMBER:destination.port:long}. Url: %{HOSTNAME:url.original}. Action: %{DATA:azure.firewall.action}. ThreatIntel: %{DATA:rule.name}$"
if: ctx?.json?.operationName == 'AzureFirewallNetworkRuleLog' || ctx?.json?.operationName == 'AzureFirewallNatRuleLog'
Expand Down
2 changes: 1 addition & 1 deletion packages/azure/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: azure
title: Azure Logs
version: 1.5.23
version: 1.5.24
release: ga
description: This Elastic integration collects logs from Azure
type: integration
Expand Down