How do I connect to the local SQL server on Ubuntu from a python script in Docker container

Question

I have an SQL Server installed in Ubuntu 20.04 (which is installed in VirtualBox) and a Docker container with a python script in it. What I want to do is to connect from this python script to the SQL server running on the host.

Here is the Dockerfile I used to create an image:

# syntax=docker/dockerfile:1

FROM python:3.8-slim-buster
WORKDIR /app
COPY requirements.txt requirements.txt

RUN apt-get update && apt-get install -y gnupg2 curl
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/20.04/prod.list > /etc/apt/sources.list.d/mssql-release.list

RUN apt-get update
RUN ACCEPT_EULA=Y apt-get install -y msodbcsql17
RUN ACCEPT_EULA=Y apt-get install -y mssql-tools
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc

RUN apt-get update \
  && apt-get -y install gcc \
  && apt-get -y install g++ \
  && apt-get -y install unixodbc unixodbc-dev \
  && apt-get clean 

RUN pip3 install -r requirements.txt

COPY . .

CMD [ "python3", "-m" , "test", "run", "--host=0.0.0.0"]

I know it looks messy, but I encountered several issues while installing pyodbc, and it looks like all this code has solved the problem.

I can now successfully create an image, but when launching a container, it produces the following output:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/app/test.py", line 10, in <module>
    connection = pyodbc.connect(connection_string, autocommit=True)
pyodbc.OperationalError: ('HYT00', '[HYT00] [Microsoft][ODBC Driver 17 for SQL Server]Login timeout expired (0) (SQLDriverConnect)')
['ODBC Driver 17 for SQL Server']

It looks like there is something wrong with the connection string, which looks like this:

connection_string = 'Driver={ODBC Driver 17 for SQL Server};Server=tcp:host.docker.internal,1433;UID=SA;PWD=<my_password>'

I've been struggling with this for some time now but still cannot make it work. Would appreciate any help or advice!

UPD

As Pato recommended, I tried changing the Server option in the connection string with the IP address of the machine.

ip addr show returned me the following output:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:29:e8:b8 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic noprefixroute enp0s3
       valid_lft 71149sec preferred_lft 71149sec
    inet6 fe80::2517:652e:ac69:8ec9/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:35:26:ba:86 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:35ff:fe26:ba86/64 scope link 
       valid_lft forever preferred_lft forever

So, I tried substituting host.docker.internal with 127.0.0.1, 10.0.2.15 and 172.17.0.1. All of them gave me the following error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/app/test.py", line 10, in <module>
    connection = pyodbc.connect(connection_string, autocommit=True)
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 17 for SQL Server;Server=tcp:172.17.0.1,1433;UID=SA;PWD=<my_password>' : file not found (0) (SQLDriverConnect)")

ifconfig returned me this:

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        inet6 fe80::42:35ff:fe26:ba86  prefixlen 64  scopeid 0x20<link>
        ether 02:42:35:26:ba:86  txqueuelen 0  (Ethernet)
        RX packets 25340  bytes 1240556 (1.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 39308  bytes 691001962 (691.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.2.15  netmask 255.255.255.0  broadcast 10.0.2.255
        inet6 fe80::2517:652e:ac69:8ec9  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:29:e8:b8  txqueuelen 1000  (Ethernet)
        RX packets 649701  bytes 946934508 (946.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 55603  bytes 7183567 (7.1 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 27675  bytes 49732911 (49.7 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 27675  bytes 49732911 (49.7 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

@Pato Forgot to mention: the Ubuntu is installed in the VirtualBox. I'm not sure if I can connect to the SQL Server remotely. How can I check? — Oleg Ivanytskyi
– Oleg Ivanytskyi, Commented Nov 26, 2021 at 16:48
What address(es) and port(s) is the SQL Server instance configured to listen on? It's possible that it's configured to accept connections only on 127.0.0.1:1433 which would be unreachable from the Docker container - because 127.0.0.0/24 is the non-routable loopback network for every machine/guest/container. In the Ubuntu guest run less /var/opt/mssql/mssql.conf and look for the network.* settings. — AlwaysLearning
– AlwaysLearning, Commented Nov 27, 2021 at 0:12
Thanks, could you install and run your python script in the same virtual machine where SQL Server is? Use this ip 127.0.0.1,1433 in the connection_string. If it doesn't work, change the SQL Server drivers to pymssql. — Patricio Contreras Pardo
– Patricio Contreras Pardo, Commented Nov 28, 2021 at 22:56

Patricio Contreras Pardo · Accepted Answer · 2021-11-29 14:20:54Z

1

I was researching about this connection problem, and the error file not found is in this post. I tried to recreate your settings but I get the same error.

A fast solution is to change the driver to pymssql==2.2.2 (tested in my docker container).

pip3 install pymssql==2.2.2

Here's the example:

import pymssql

conn = pymssql.connect('host.docker.internal', 'sa', 'yourPassword', "database")

cursor = conn.cursor()

cursor.execute("""
IF OBJECT_ID('persons', 'U') IS NOT NULL
    DROP TABLE persons
CREATE TABLE persons (
    id INT NOT NULL,
    name VARCHAR(100),
    salesrep VARCHAR(100),
    PRIMARY KEY(id)
)
""")

cursor.executemany(
    "INSERT INTO persons VALUES (%d, %s, %s)",
    [(1, 'John Smith', 'John Doe'),
     (2, 'Jane Doe', 'Joe Dog'),
     (3, 'Mike T.', 'Sarah H.')])

# you must call commit() to persist your data if you don't set autocommit to True
conn.commit()

cursor.execute('SELECT * FROM persons WHERE salesrep=%s', 'John Doe')
row = cursor.fetchone()
while row:
    print("ID=%d, Name=%s" % (row[0], row[1]))
    row = cursor.fetchone()

conn.close()

# Result
'''
ID=1, Name=John Smith
'''

Update

Troubleshooting SQL Server Linux versions

Check if mssql-server.service is running.

patricio@server2:~$ sudo systemctl status mssql-server.service
[sudo] password for patricio:
● mssql-server.service - Microsoft SQL Server Database Engine
   Loaded: loaded (/lib/systemd/system/mssql-server.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2021-11-10 09:56:38 -03; 2 weeks 4 days ago
     Docs: https://learn.microsoft.com/en-us/sql/linux
 Main PID: 14885 (sqlservr)
    Tasks: 164
   CGroup: /system.slice/mssql-server.service
           ├─14885 /opt/mssql/bin/sqlservr
           └─14913 /opt/mssql/bin/sqlservr

Check if the port 1433 or your default port is open.

patricio@server2:~$ telnet 127.0.0.1 1433
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

patricio@server2:~$ nmap 127.0.0.1

Starting Nmap 7.60 ( https://nmap.org ) at 2021-11-29 08:41 -03
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000085s latency).
Not shown: 994 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
80/tcp   open  http
443/tcp  open  https
1433/tcp open  ms-sql-s
// ...

Install sqlcmd if it wasn't installed

patricio@server2:~$ sqlcmd -S 127.0.0.1 -U sa -p
Password:
1> SELECT CONVERT(varchar, SERVERPROPERTY('collation'))
2> GO

------------------------------
SQL_Latin1_General_CP1_CI_AS

(1 rows affected)

References:

From inside of a Docker container, how do I connect to the localhost of the machine?

Trying to access host.docker.internal results in Connection refused

edited Nov 29, 2021 at 14:20

answered Nov 29, 2021 at 0:29

Patricio Contreras Pardo

5072 gold badges4 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Oleg Ivanytskyi Over a year ago

thanks a lot for your help! Unfortunately, now I get the following error:

File "src/pymssql/_pymssql.pyx", line 652, in pymssql._pymssql.connect pymssql._pymssql.OperationalError: (20009, b'DB-Lib error message 20009, severity 9:\nUnable to connect: Adaptive Server is unavailable or does not exist (host.docker.internal)\n')

Patricio Contreras Pardo Over a year ago

@OlegIvanytskyi check the update. If your docker container can't ping or reach your SQL Server maybe this post can help you stackoverflow.com/q/20430371/9925593

Patricio Contreras Pardo Over a year ago

@Oleg check the references too

Oleg Ivanytskyi Over a year ago

OMG, thank you so much! Turns out, --network="host" is a necessary option when running a container. I can then use 127.0.0.1 instead of host.docker.internal and everything works fine. You saved my day!

Patricio Contreras Pardo Over a year ago

I'm glad it worked

Collectives™ on Stack Overflow

How do I connect to the local SQL server on Ubuntu from a python script in Docker container

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related