As stated in this question, I should avoid calling self inside a map function. Extending this, I have two questions: Let's use the same code stated there:
class C0(object):
def func0(self, arg): # added self
...
def func1(self, rdd): # added self
func = self.func0
result = rdd.map(lambda x: func(x))
Is
result = rdd.map(lambda x:func(x))the same thing asresult = rdd.map(func)? Specially in this situation where I usefunc = self.func0before?Let's say func0 calls another method from the class:
class C0(object):
def func2(self, arg):
...
def func0(self, arg): # added self
self.func2(arg)
...
def func1(self, rdd): # added self
func = self.func0
result = rdd.map(lambda x: func(x))
How does Spark handle this? Should I do func2 = self.func2 inside func0?
staticmethod