I am defining a Stream Analytics Job via an Azure Resource Manager (ARM) template.
I need to persist some messages from an Event Hub input into a SQL Server database output.
when running this command:
az group deployment create \
--name "deployStreamAnalyticsJobs" \
--resource-group "CiTestRG" \
--template-file ./templates/stream-analytics-jobs-definition.json
then I see this output:
Deployment failed. Correlation ID: <ONE_GUID>. {
"code": "422",
"message": "The required property 'type' is missing from the request.",
"details": {
"code": "422",
"message": "The required property 'type' is missing from the request.",
"correlationId": "<ANOTHER_GUID>",
"requestId": "<YET_ANOTHER_GUID>"
}
}
but according to this documentation: https://learn.microsoft.com/en-us/azure/templates/microsoft.streamanalytics/streamingjobs#MicrosoftSqlServerDatabaseOutput
I should only be providing the database output details as:
"datasource": {
"type": "Microsoft.Sql/Server/Database",
"properties": {
"server": "string",
"database": "string",
"user": "string",
"password": "string",
"table": "string"
}
}
I think what the error message of the az command is trying to say is that those properties are not defined as string, but then why the docs don't specify the type but just pass those properties as string values?
This is the JSON definition of the ARM template:
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"location": {
"type": "string",
"defaultValue": "westeurope"
},
"hubName": {
"type": "string",
"defaultValue": "myIoTHub"
},
"eventhubs_messages_name": {
"defaultValue": "myEhName",
"type": "String"
},
"namespaces_oecollector_name": {
"defaultValue": "myEventHub",
"type": "String"
},
"streamAnalyticsJobName": {
"type": "string",
"defaultValue": "myStreamAnalyticsJob"
}
},
"resources": [{
"type": "Microsoft.StreamAnalytics/StreamingJobs",
"apiVersion": "2016-03-01",
"name": "[parameters('streamAnalyticsJobName')]",
"location": "[resourceGroup().location]",
"properties": {
"sku": {
"name": "standard"
},
"outputErrorPolicy": "Drop",
"eventsOutOfOrderPolicy": "adjust",
"eventsOutOfOrderMaxDelayInSeconds": 0,
"eventsLateArrivalMaxDelayInSeconds": 86400,
"inputs": [{
"Name": "EventHubOutputLable",
"Properties": {
"DataSource": {
"Type": "Microsoft.ServiceBus/EventHub",
"Properties": {
"eventHubName": "[parameters('eventhubs_messages_name')]",
"serviceBusNamespace": "[parameters('namespaces_oecollector_name')]",
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', parameters('namespaces_oecollector_name'), parameters('eventhubs_messages_name'), 'super-user'),'2017-04-01').primaryKey]",
"sharedAccessPolicyName": "RootManageSharedAccessKey"
}
},
"Serialization": {
"Properties": {
"Encoding": "UTF8"
},
"Type": "Json"
}
}
}],
"outputs": [{
"Name": "myOutputDb",
"Properties": {
"DataSource": {
"Type": "Microsoft.Sql/Server/Database",
"Properties": {
"Server": "my-sql-server-name",
"Database": "my-database-name",
"User": "my-sa-user",
"Password": "my-password",
"Table": "MySchema.MyTable"
}
}
}
}],
"transformation": {
"name": "Transformation",
"properties": {
"streamingUnits": 1,
"query": "WITH data AS (\r\n SELECT\r\n <THE_FIELDS_OF_THE_JSON_INPUT_AS_COLUMNS>\r\n FROM\r\n input\r\n WHERE\r\n topic = 'FOO'\r\n )\r\n \r\n SELECT * INTO myOutputDb FROM data"
}
}
}
}]
}
How can I make this Stream Analytics Job picking messages from an input Event Hub and storing them into an output SQL Server database?