0

Background: I'm converting python script which takes values from django-db to be multi-threaded using Process. I might missed something (quite new in django) but got "DatabaseError: SSL error: decryption failed or bad record mac" So I've closed the connection (according to this)... but than got "DatabaseError: SSL connection has been closed unexpectedly".

I decided to try psycopg2 connected & closed the connections in the locations of the django version. All worked well.

Issue: when I've tested the solution I saw that the same query return different values!

My query is very simple: "SELECT DISTINCT "someId" FROM aTable WHERE "date" < 'date_in_the_past';"

Please advice..

UPDATE: I can't show the complete code.. I've changed names etc' but flow is 100% the same. The lanchAll method launches process for every t_id, when I search with django.db connection I get 3 distinct values, when make it with psycopg2 I get 1 value. In runtime this is called with in_sim which is a date in the past. In lanchAll the django.db connection is marked as comment right next to the psycopg2...

from multiprocessing import Process
import sys
from numpy import *
from itertools import izip_longest
from datetime import * 
import psycopg2
from appName import settings
from django.core.management import setup_environ
from django.core.exceptions import ObjectDoesNotExist

import argparse
import pdb
setup_environ(settings)

from appName.models import *
from appName.aClass import *
from django.db import connection

def lanchAll(in_sim):
    p = 0
    try:
        con = psycopg2.connect(database=DB_NAME, user=DB_USER, password=DB_PASS,
                        host=DB_IP)
        cursor = con.cursor()
        # cursor = connection.cursor()
        if in_simolationDate is None:
            query = 'SELECT DISTINCT "t_id" FROM appName_tableNAme '\
                'WHERE "date" > now() - interval \'%s seconds\';'        
            cursor.execute(query, [TIME_WINDOW])    
        else:
            query = 'SELECT DISTINCT "t_id" FROM appName_tableNAme '\
                'WHERE "date" < %s ;'        
            cursor.execute(query, [in_sim])    

    except ObjectDoesNotExist as e:
        con.close()
        # connection.close()
        print((str('DB Error: %s\n' % e)))
        return -3
    if cursor.rowcount < 1:
        con.close()
        # connection.close()
        print ("offline!")
        return -4

    for row in onlineTagsCursor:
        t_id = row[0]   
        print "currently lunching tagId:" + str(t_id)
        processInstance = Process(target=launchThread, args=(t_id, p,in_sim))
        processInstance.start()

    con.close()
    # connection.close()


def launcher(in_id, in_p,in_sim):
    #I'll add it if you fell that it is needed.. 


#the model for the table     
class tableNAme(models.Model):
    m_id    = models.IntegerField()
    sc_id   = models.IntegerField()
    t_id    = models.IntegerField()
    r       = models.IntegerField()
    l       = models.IntegerField()
    g       = models.IntegerField(0)
    p       = models.IntegerField(0)
    date    = models.DateTimeField(auto_now=True)
3
  • Add the code you are using Commented Aug 6, 2014 at 15:48
  • Perhaps a transaction problem, with each process being atomic? Code is needed as @kroolik said. Commented Aug 6, 2014 at 16:25
  • Thank you both, I've added the problematic code. Commented Aug 7, 2014 at 7:55

1 Answer 1

0

I suspect you are forking after the connection has been created. Libpq connections (the ones used by psycopg) don't support fork.

Make sure the new processes create a connection of their own, they cannot use one created by the parent process.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.