1

I wrote a parent class:

class Parent():
    def __init__(self, spark_session=None):
        try:
            # Instantiate Spark Session
            self.spark = spark_session
            if not self.spark:
                self.spark = SparkSession.builder.config("spark.sql.debug.maxToStringFields", 1000).appName("SparkTest") \
                    .getOrCreate()
        except Exception as e:
            print("Initialization of spark & logging couldn't be performed: {}".format(e))

Then the child class has an extra argument in its constructor which is path:

class Child(Parent):
    def __init__(self, path, spark=None):
        self.spark = super().__init__(spark)
        self.path = path

Then when I write this:

a = Child("path", None)
print(a.spak)

For:

self.spark = spark_session

I get:

AttributeError: 'NoneType' object has no attribute 'spark' None

Any idea how to fix this and make spark object instantiate as expected.

Regards

1
  • Could you please post the full traceback (maybe remove your name)? Commented Jan 6, 2020 at 11:34

1 Answer 1

1

self.spark = super().__init__(spark)

__init__ is not a constructor. It initializes the object's attributes and returns None, not the object.

self.spark will be automatically initialized once you call super().__init__.

Note I also changed Parent.__init__ a bit. In your code self.spark is not defined in case an exception is raised which will lead to other errors down the road.

class Parent:
    def __init__(self, spark_session=None):
        self.spark = spark_session            
        try:  
            if not self.spark:
                self.spark = SparkSession.builder .config("spark.sql.debug.maxToStringFields", 1000).appName("SparkTest") \
                .getOrCreate()
        except Exception as e:
            print("Initialization of spark & logging couldn't be performed: {}".format(e))


class Child(Parent):
    def __init__(self, path, spark=None):
        super().__init__(spark)
        self.path = path
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.