converting python 2 code to python 3

Question

I am trying to convert the following code written in python 2 to python 3. This python code does TCP port forwarding. It is from this page: http://code.activestate.com/recipes/483730-port-forwarding/

import socket
import sys
import thread

def main(setup, error):
    sys.stderr = file(error, 'a')
    for settings in parse(setup):
        thread.start_new_thread(server, settings)
    lock = thread.allocate_lock()
    lock.acquire()
    lock.acquire()

def parse(setup):
    settings = list()
    for line in file(setup):
        parts = line.split()
        settings.append((parts[0], int(parts[1]), int(parts[2])))
    return settings

def server(*settings):
    try:
        dock_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        dock_socket.bind(('', settings[2]))
        dock_socket.listen(5)
        while True:
            client_socket = dock_socket.accept()[0]
            server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            server_socket.connect((settings[0], settings[1]))
            thread.start_new_thread(forward, (client_socket, server_socket))
            thread.start_new_thread(forward, (server_socket, client_socket))
    finally:
        thread.start_new_thread(server, settings)

def forward(source, destination):
    string = ' '
    while string:
        string = source.recv(1024)
        if string:
            destination.sendall(string)
        else:
            source.shutdown(socket.SHUT_RD)
            destination.shutdown(socket.SHUT_WR)

if __name__ == '__main__':
    main('proxy.ini', 'error.log')

Here is what I have:

import socket
import sys
import threading
import time

def main(setup, error):
    # open file for error messages
    sys.stderr = open(error, 'a')
    # read settings for port forwarding

    threads = []

    for settings in parse(setup):
        #thread.start_new_thread(server, settings)
        t = threading.Thread(target=server, args=(settings))
        t.start()
        threads.append(t)

    for t in threads:
        t.join()

    # wait for <ctrl-c>
    while True:
        time.sleep(60)

def parse(setup):
    settings = list()
    file = open(setup)

    for line in iter(file):
        parts = line.split()
        settings.append((int(parts[0]), parts[1], int(parts[2])))

    return settings

def server(*settings):

    try:
        dock_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        dock_socket.bind(('', settings[0]))
        dock_socket.listen(5)
        while True:
            client_socket = dock_socket.accept()[0]
            server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            server_socket.connect((settings[1], settings[2]))

            Thread1 = threading.Thread(target=forward, args=(client_socket, server_socket))
            Thread1.start()

            Thread2 = threading.Thread(target=forward, args=(server_socket, client_socket))
            Thread2.start()

    finally:
        thread = threading.Thread(targer=server, args=settings)
        thread.start()

def forward(source, destination):
    string = ' '
    while string:
        string = source.recv(1024)
        if string:
            destination.sendall(string)
        else:
            source.shutdown(socket.SHUT_RD)
            destination.shutdown(socket.SHUT_WR)

if __name__ == '__main__':
    main('port-forward.config', 'error.log')

The python 3 version does seem to work. But I am not fully certain if it is written correctly. I am unclear about the threads portion of the code. The python 2 version is using the thread module while the python 3 version is using the threading module. Looking at the python 2 version, it uses locks in the main function. Do I need to use locks in the python 3 version? Another question is, should I be joining the two threads (thread 1 and thread 2) in the server function in the python 3 code?

Another question I have regardless of python version is the argument to the server function. I understand that the "settings" variable refers to a list. Why does there need to be a asterisk preceding the "settings" argument? I did take a look at the following page in python documentation https://docs.python.org/3/tutorial/controlflow.html#arbitrary-argument-lists

But I don't understand why passing the settings list without the asterisk doesn't work.

Another question I have is regarding the thread locks in the python 2 code. Why is the lock being acquired twice? I tried removing one of the locks and the program ends immediately after being started.

I have edited the post to add another question I had.

user695752
– user695752

2017-03-19 04:55:42 +00:00
Commented Mar 19, 2017 at 4:55 — user695752
– user695752, Commented Mar 19, 2017 at 4:55

alexis · Accepted Answer · 2017-03-17 09:32:13Z

6

You could have saved yourself some work by using the commandline tool 2to3 to convert the code to Python 3, like this (from the bash or Windows command line):

2to3 -w myscript.py

But let's answer your actual question: Why the star in the definition

def server(*settings):
    ...

The function thread.start_new_thread will in effect launch your server() function like this:

server(arg1, arg2, arg3)

where arg1, arg2 etc. come from the second argument of start_new_thread. The declaration def server(*settings) collects these arguments back into a single list, settings, which your function goes on to use. If you write def server(settings) (no star), your function is declared to take one argument but will be called with several.

answered Mar 17, 2017 at 9:32

alexis

50.4k18 gold badges108 silver badges173 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user695752 Over a year ago

Thanks Alexis for explaining the starred arguments.

Selcuk · Accepted Answer · 2019-11-14 06:24:16Z

3

You can use the 2to3 tool that come with python 3 as a standard module. It does most of the conversion for you.

$2to3 youpythonfile.py

You can use the _thread module in python 3 which is identical to thread in python 2.

edited Nov 14, 2019 at 6:24

Selcuk

60.1k12 gold badges114 silver badges119 bronze badges

answered Mar 17, 2017 at 9:24

Taku

34.1k12 gold badges79 silver badges88 bronze badges

Comments

Max Steely · Accepted Answer · 2017-03-17 09:21:30Z

1

You can use the futurize script which convert your py2 code into py3

See : http://python-future.org/quickstart.html#quick-start-guide for installation and http://python-future.org/quickstart.html#to-convert-existing-python-2-code for example

answered Mar 17, 2017 at 9:21

Max Steely

365 bronze badges

Comments

Noctis Skytower · Accepted Answer · 2017-03-20 14:58:51Z

As the author of the code that you referenced from the Python cookbook, it is my pleasure to rewrite the program for Python 3 and introduce you to another useful recipe. Port Forwarding is the older of the two recipes but was a complete program used as a college student at my first college. Module For Running Simple Proxies is a improved version of the original code and was designed for use in other programs rather than being a complete program by itself. The code that follows takes the idea of Port Forwarding and combines it with lessons learned in Module For Running Simple Proxies and lessons learned since then to provide a complete and hopefully easy-to-read program for your use and study:

import argparse
import re
import select
import socket
import sys
import threading


SETUP_PATTERN = r'^(?P<server_name>\S+)' \
                r'\s+(?P<server_port>\d+)' \
                r'\s+(?P<proxy_port>\d+)'
SETUP_TYPES = dict(server_name=str, server_port=int, proxy_port=int)


def main():
    arguments = parse_arguments()
    sys.stderr = open(arguments.error, 'a')
    for settings in parse_configuration(arguments.setup):
        threading.Thread(target=handle_connections, kwargs=settings).start()


def parse_arguments():
    parser = argparse.ArgumentParser(description='Forward TCP traffic.')
    parser.add_argument('setup', type=str, help='file with forwarding rules')
    parser.add_argument('error', type=str, help='location for error reports')
    arguments = parser.parse_args()
    return arguments


def parse_configuration(setup):
    with open(setup) as file:
        for line in file:
            match = re.search(SETUP_PATTERN, line)
            if match:
                yield {key: SETUP_TYPES[key](value)
                       for key, value in match.groupdict().items()}


def handle_connections(server_name, server_port, proxy_port):
    proxy = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    proxy.bind((socket.gethostname(), proxy_port))
    proxy.listen(5)
    while True:
        client, address = proxy.accept()
        server = socket.create_connection((server_name, server_port))
        threading.Thread(target=handle_traffic, args=(client, server)).start()


def handle_traffic(client, server):
    pairs, empty = {client: server, server: client}, ()
    while pairs:
        read, write, error = select.select(pairs.keys(), empty, empty)
        for connection in read:
            try:
                data = connection.recv(1 << 12)
            except ConnectionResetError:
                data = None
            if data:
                pairs[connection].sendall(data)
            else:
                connection.shutdown(socket.SHUT_RD)
                pairs.pop(connection).shutdown(socket.SHUT_WR)
    client.close()
    server.close()


if __name__ == '__main__':
    main()

Collectives™ on Stack Overflow

converting python 2 code to python 3

4 Answers 4

1 Comment

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related