Difference between revisions of "Projects/Plugin support improvements"

Revision as of 16:13, 21 July 2010

Motivations, Priorities & Requirements

Motivations: there are a number of motivations behind the creation of the plugin architecture framework.

• Desire to separate plugin interface from its implementation;
• Desire to provide simple and clear mechanism that facilitates additions of new plugin interfaces and their implementations(modules);
• Handles both built-in and dynamic plugin modules;
• Allows multiple implementation of the same plugin interface;
• Provides uniform way to supply parameters for plugin configuration;
• Allows one plugin implementation to use services provided by the other plugin implementations.

Requirements: from these items we have developed a more formal set of requirements covering the design and the implementation of the framework to support the plugins. These are as follows:

1. Allow third parties to implement multiple plugin modules for each plugin interface.
2. Allow a plugin module to build as dynamic or built-in from the same source code.
3. Allow third parties to more easily create new plugin modules.
4. Provide a uniform method for configuring discovery of plugin modules.
5. Improve readability of code that calls plugin interfaces.
6. Allow easier creation of new plugin interfaces.
7. Allow incremental transition of existing plugin interfaces to the new framework.

Architecture Overview and Concepts

Introduction

TBD

Participants

The following is a summary of participants and components within the architecture. Further details are provided in the sections below.

Plugin Manager: The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services. It is responsible for managing plugin handles and provides discovery capabilities for consumers or callers.

Plugin Interface: A plugin interface is an interface that can be implemented by a third party in a modular manner. An implementation of a plugin interfaces is referred to as a plugin module. Furthermore, a plugin interface itself consist of a consumer interface and provider interface (see below).

Plugin Module: A plugin module is an implementation of a plugin interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

Consumer: The consumer or caller is the entity that uses the plugin module. The consumer or caller may also perform discovery of available modules prior to using them.

Collaboration: Flows

Architecture Components

In this section we provide further details on the components of the architecture, describing its features and behaviors.

Plugin Manager

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of plugin interfaces do not call it directly, but instead call a loader function of the specific plugin interface, which in-turn calls the plugin manager.

In the current architecture, the krb5_init_context() functions will create and configure a plugin manager context that will exist in the krb5_context. An consumer application may use krb5_get_plm() to retrieve the plugin manager context so it can register its own built-in plugin modules. Libraries such as libkadm5 register their own built-in plugin modules this way too.

The plugin manager locates plugin modules using both a numeric identifier that designates a plugin interface and a string that names a module that implements that pluggable interface. The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

As a lower-level interface that supports backward compatibility for older loadable-module provider interfaces, the plugin manager can create an opaque locator handle for a specified module. The plugin manager also supports looking up sets of locator handles (for all modules implementing a specified pluggable interface).

Additional functions in the plugin manager interface allow for lookups of symbols in the plugin module (if supported by that module).

The plugin manager keeps track of modules through its registries.

Registry of built-in modules

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Applications can also register private built-in plugin modules using this registry.

Registry of loadable modules

This registry keeps track of a few additional items needed for locating loadable modules. This includes a base directory pathname to a directory tree of plugin modules. Each subdirectory of the base directory has a name that corresponds to the pluggable interface identifier for the modules that intended to be in that subdirectory. Caching of vtable constructors of previously-loaded dynamically loadable modules can occur.

Plugin Interfaces

A plugin interface is an interface, possibly internal to a library, that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of plugin interfaces are called plugin modules. plugin interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a plugin interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Plugin interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A plugin interface has two parts: a consumer interface and a provider interface. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface. The provider interface is what the plugin module implements.

Consumer interface

The consumer interface isolates the consumer from implementation details of the plugin interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any plugin interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a plugin interface uses an opaque handle, obtained from a loader function that is part of the plugin interface, to call the methods of a plugin module. Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these plugin interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

A handle can represent:

• the plugin module itself
• a resource to which the plugin module provides access (e.g., a ccache handle)
• a set of plugin modules (e.g., the set of all available preauth mechanisms)

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the plugin interface. One reason is for type safety: there will be a distinct opaque handle type for each plugin interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a plugin interface to support plugin modules that implement an older provider interface.

Provider interface

A plugin module is a unit of code that implements the provider interface portion of a plugin interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using dlopen().

Loadable module provider interface

The domain-independent part of the provider interface of a loadable module consists of a single exported function symbol, which denotes the vtable constructor function for that module. The signature of the constructor is specific to the plugin interface the module implements. The contents of the vtable are also specific to the plugin interface.

The constructor usually takes as arguments the following:

• an interface version number
• a pointer to a caller-allocated vtable structure for the interface,
• the size of the structure (as an added precaution against version mismatches).

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

DELETE- Although the caller (actually the plugin support code) allocates the vtable structure in the above description, one alternative is to have the module perform the allocation of the structure itself. This can cause problems if the module uses a different memory allocator than the caller.

DELETE - Another alternative is to have the vtable constructor instead return a pointer to a compiled-in vtable. This might cause performance problems related to copy relocations.

Built-in module provider interface

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a plugin interface.

Operational Flow

Startup

The krb5_init_context() function initializes a plugin manager context that resides in the krb5_context. It registers libkrb5 built-in modules (from static tables), and reads location parameters of dynamically-loaded plugin modules (if any) from the krb5.conf (or other configuration profile), calling plugin manager methods to configure the locations of dynamically-loaded plugin modules.

Consumer

• The consumer calls the plugin loader function for the desired plugin interface.

• The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module and builds an opaque module handle to give to the consumer.

• The plugin manager looks up the vtable constructor to give to the loader function of the pluggable interface.

• The consumer calls wrapper functions of the pluggable interface, passing the opaque module handle, to access the capabilities of the pluggable interface.

Adding new modules at run time

An application can register its own plugin modules at run time by calling the plugin manager function to update the registry.

Backward compatibility

Older dynamically loadable modules that don't conform to the "vtable constructor function" provider model can be handled with specialized loader functions. If a pluggable interface needs to support a legacy loadable module of this sort, it can contain a more elaborate loader function that uses the lower-level capabilities of the plugin manager to:

• obtain an opaque locator handle for a module (or set of modules)
• look up data or function symbols from a module locator handle (or set of locator handles)

Configuration

Sample Code and Proof of Concept

Deliverables