I'm trying to get myself familiar with importlib hooks. I want to implement an ability to directly import non-pythonic files written in other language, and maintain source maps, so raising SyntaxErrors with line numbers will still give meaningful stacktraces.
My approach to loading foreign files is assembling Pythonic source, then compiling it and executing it in the desired context.
I've read in the documentation that implementing importlib.abc.SourceLoader seems to be my choice — however, the only method that is being called is exec_module. At that stage, to my understanding, module object is fully ready. So why are get_source, get_data, get_code and others are not called?
My stub implementation:
import sys
import os
import importlib.abc
import importlib.machinery
class MyFinder(importlib.abc.MetaPathFinder):
def __init__(self):
pass
def find_spec(self, fullname, path, target=None):
print('find_spec', fullname, path, target)
# filename = '{}.npy'.format(fullname)
# if not os.path.exists(filename):
# return
if fullname != 'foobar':
return
filename = 'foobar://ponyworld/foo.py'
spec = importlib.machinery.ModuleSpec(
name = fullname,
loader = MyLoader(fullname, path, target),
origin = filename,
loader_state = 1234,
is_package = False,
)
return spec
class MyLoader(importlib.abc.SourceLoader):
def __init__(self, fullname, path, target):
pass
def get_data(self, path):
print('get_data', path)
def get_filename(self, fullname):
print('get_filename', fullname)
def path_stats(self, path):
print('path_stats', path)
def set_data(self, path, data):
print('set_data', path, data)
def get_code(self, fullname):
print('get_code', fullname)
def exec_module(self, module):
print('exec_module', module)
print(dir(module))
def get_source(self, fullname):
print('get_source', fullname)
def is_package(self, fullname):
print('is_package', fullname)
sys.meta_path.append(MyFinder())
# import fake module to see if it works
import foobar
Loader's__init__you can combineMyLoaderandMyFinderintoMyImporterby decoratingfind_specas aclassmethodThe__init__then becomes the init for theSourceLoaderparent. Supposedly this saves one from instantiating theLoader. Though this makes it a little less flexible if theFinderneeds localized information, as this is only really passed in by instantiation.