I want to add a connector to my local Kafka cluster using apache Kafka Connect Rest API . Do you know any example to do this, please. Thanks
-
Have you tried searching for the connect rest api?OneCricketeer– OneCricketeer2020-01-20 13:11:50 +00:00Commented Jan 20, 2020 at 13:11
Add a comment
|
1 Answer
Confluent's documentation includes a clear example on how to add a Connector through Kafka REST API:
POST /connectorsCreate a new connector, returning the current connector info if successful.
Return 409 (Conflict) if rebalance is in process.
Example Request:
POST /connectors HTTP/1.1
Host: connect.example.com
Content-Type: application/json
Accept: application/json
{
"name": "hdfs-sink-connector",
"config": {
"connector.class": "io.confluent.connect.hdfs.HdfsSinkConnector",
"tasks.max": "10",
"topics": "test-topic",
"hdfs.url": "hdfs://fakehost:9000",
"hadoop.conf.dir": "/opt/hadoop/conf",
"hadoop.home": "/opt/hadoop",
"flush.size": "100",
"rotate.interval.ms": "1000"
}
}
Example Response:
HTTP/1.1 201 Created
Content-Type: application/json
{
"name": "hdfs-sink-connector",
"config": {
"connector.class": "io.confluent.connect.hdfs.HdfsSinkConnector",
"tasks.max": "10",
"topics": "test-topic",
"hdfs.url": "hdfs://fakehost:9000",
"hadoop.conf.dir": "/opt/hadoop/conf",
"hadoop.home": "/opt/hadoop",
"flush.size": "100",
"rotate.interval.ms": "1000"
},
"tasks": [
{ "connector": "hdfs-sink-connector", "task": 1 },
{ "connector": "hdfs-sink-connector", "task": 2 },
{ "connector": "hdfs-sink-connector", "task": 3 }
]
}
GET /
answered Jan 20, 2020 at 11:26
Giorgos Myrianthous
40.4k21 gold badges156 silver badges175 bronze badges
1 Comment
OneCricketeer
Connect plugins cannot be added to the cluster via the API, though