Python pkg_resources module dynamic loading plugin example analysis

Using the Standard Libraryimportlib(used form a nominal expression)import_module()function, django's import_string(), all of which dynamically load the specified Python module.

Give two examples of dynamic loading:

Example one:

If you have a test function in your project, located in your_project/demo/, then you can use import_module to dynamically load and call this function without importing it where it is used.

module_path = 'your_project/demo'
module = import_module(module_path)
()

Example two:

django's middleware have used it, just need to be configured in the setting django can automatically be called, which is also the use of import_string dynamically loaded.

#
MIDDLEWARE = [
    '',
    '',
     ...
]
# Dynamically load the code at the call
for middleware_path in reversed():
    middleware = import_string(middleware_path)
    ...

The above approach will have some drawbacks:

• If the referenced module does not exist, the error will not be thrown in time when the project is started, and will only be detected when it is actually called.
• The referenced module should be written in advance and placed in the specified location, and should not be deleted.
• Semi-automatic pluggability, want to disable a plug-in features need to manually modify the code. For example, if you want to remove django's SessionMiddleware feature, you need to manually modify the settings configuration file.

pkg_resources implements dynamic loading of plugins

Here is another way to dynamically load plug-ins, the software library pkg_resources, which is installed together with the setuptools library. It basically solves the above problem and has in fact become a popular plug-in implementation. The main unit of operation of pkg_resources is the Distribution.Refer hereWhen the Python script starts, pkg_resources recognizes all of Distribution's namespace packages in the search path, so we'll find paths containing many pip-installed packages and can perform import operations correctly.

pkg_resources comes with a global WorkingSet object that represents the default working set for the search path, which is the working set we commonly use. With this working set, then it's easy to dynamically import any module.

Case in point below:

This case is an event handling plugin for ansible-runner, project addressGitHub - ansible/ansible-runner-httpThe package will automatically recognize and call the status_handler and event_handler event handlers. Simply install this package into your virtual environment and ansible-runner will automatically recognize and call the status_handler and event_handler event handler functions. When the package is uninstalled, ansible-runner handles events in the default way. Compared to the import_module approach described earlier, this dynamic loading approach is good in that it is less invasive to the source code, and realizes true plug-and-play. Here's how to do it with pkg_resources.

Structure of the ansible-runner-http project source directory:

├──

├── ansible_runner_http

│├── __init__.py

│└──

└──

:

...
def status_handler(runner_config, data):
    plugin_config = get_configuration(runner_config)
    if plugin_config['runner_url'] is not None:
        status = send_request(plugin_config['runner_url'],
                              data=data,
                              headers=plugin_config['runner_headers'],
                              urlpath=plugin_config['runner_path'])
        ("POST Response {}".format(status))
    else:
        ("HTTP Plugin Skipped")
def event_handler(runner_config, data):
    status_handler(runner_config, data)

__init__.py:

from .events import status_handler, event_handler # noqa

:

from setuptools import setup, find_packages
with open('', 'r') as f:
    long_description = ()
setup(
    name="ansible-runner-http",
    version="1.0.0",
    author="Red Hat Ansible",
    url="/ansible/ansible-runner-http",
    license='Apache',
    packages=find_packages(),
    long_description=long_description,
    long_description_content_type='text/markdown',
    install_requires=[
        'requests',
        'requests-unixsocket',
    ],
    # Way 1: Specific to a module
    entry_points={'ansible_runner.plugins': ['http = ansible_runner_http']},
    # Way 2: Specific to a function under a certain module
    #entry_points={'ansible_runner.plugins': [
    #    'status_handler = ansible_runner_http:status_handler',
    #    'event_handler = ansible_runner_http:event_handler',
    #    ]
    #},
    zip_safe=False,
)

Focus on the entry_points option in setup:

• Group names, separated by dots for hierarchical organization, are not associated with a Package, such asansible_runner.plugin
• name, e.g. http
• The location in Distribution can point to a module such as ansible_runner_http or to a function within a module such as ansible_runner_http:status_handler, preceded by the Module and followed by the function within the module.

This way pkg_resources can dynamically recognize the plugin once this package is installed.

Look at the caller below:

...
plugins = {
    # Call the load method and get the object pointing to python
    entry_point.name: entry_point.load()
    for entry_point
    #Call WorkingSet.iter_entry_points method to iterate through all EntryPoints, parameter is group name.
    in pkg_resources.iter_entry_points('ansible_runner.plugins')
}
...
def event_callback(self, event_data):
        '''
        Invoked for every Ansible event to collect stdout with the event data and store it for
        later use
        '''
    for plugin in plugins:
        plugins[plugin].event_handler(, event_data)
        ...

Mode 1 writeup gets the plugins.

The plugins you get from the way 2 writeup.

to this article on the Python pkg_resources module dynamically loaded plug-ins example analysis of the article is introduced to this, more related Python dynamically loaded plug-ins content, please search for my previous posts or continue to browse the following related articles I hope you will support me in the future more!