0

As stated in this question, I should avoid calling self inside a map function. Extending this, I have two questions: Let's use the same code stated there:

class C0(object):

  def func0(self, arg): # added self
    ...

  def func1(self, rdd): # added self
    func = self.func0
    result = rdd.map(lambda x: func(x))
  1. Is result = rdd.map(lambda x:func(x)) the same thing as

    result = rdd.map(func)? Specially in this situation where I use func = self.func0 before?

  2. Let's say func0 calls another method from the class:

class C0(object):

  def func2(self, arg):
    ...

  def func0(self, arg): # added self
    self.func2(arg)
    ...

  def func1(self, rdd): # added self
    func = self.func0
    result = rdd.map(lambda x: func(x))

How does Spark handle this? Should I do func2 = self.func2 inside func0?

2

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.