K5Wiki - User contributions [en]

Projects/Documentation Tasks

2011-11-08T19:34:16Z

Hardjono: /* Most commonly used API functions (in alphabetical order) */

{{project-early}}

== Purpose ==

To keep track of the various tasks that need to be documented such as function documentation, administration, troubleshooting etc.

{| class="wikitable"
|+ Matrix of Document-Type VS Intended Readership
|-
! Doc-type/Reader
! Architectural Guide
! Setup & Config of Kerberos
! Admin & Operations of Kerberos
! Custom Build
! API Description
! API Details
|-
|-
| End-users || || || || || ||
|-
| Architects || || || || || ||
|-
|System Admins || || || || || ||
|-
|Application Developers || || || || || ||
|-
|GSSAPI Developers || || || || || ||
|-
|Kerberos Developers || || || || || ||
|}

== Application development ==

{| class="wikitable"
|+
|-
! task
! Proposed Author
! Target Date
! Reviewer
! Reviewer Comments
|-
|-
| Designing a new protocol, or extending existing one, to use GSS-API || || || ||
|-
| Choosing security API|| || || ||
|-
| <ul><li> GSS-API vs SASL vs KRB5 </ul>|| || || ||
|-
| <ul><li> A guide to the similarities and differences between Heimdal and MIT Kerberos API </ul>|| || || ||
|-
| GSS-API || || || ||
|-
| <ul><li> A basic introduction to GSS-API, making use of the sample client and server, with special attention paid to Kerberos-related GSS-API issues</ul>|| || || ||
|-
| <ul><li> How to tell the GSS-API library on the client side where the existing Kerberos ticket cache is </ul>|| || || ||
|-
| <ul><li> How to write mechanism-independent GSS-API code</ul>|| || || ||
|-
| <ul><li> Acceptor naming - How to get servers to use any key in a keytab</ul>|| GH|| || ||
|-
| <ul><li> A guide to GSS-API naming as compared to Kerberos principal naming</ul>|| || || ||
|-
| <ul><li> Using IAKERB</ul>|| || || ||
|-
| <ul><li> Anonymous credentials</ul>|| || || ||
|-
| <ul><li> Delegating credentials</ul>|| MIT || || ||
|-
| <ul><li> Available extensions</ul>|| || || ||
|-
| <ul><li> Thread safety</ul>|| KR || || ||
|-
| <ul><li> Validating the flags set on the connection to ensure things like mutual authentication, confidentiality, integrity, replay protection, and sequence protection</ul>|| || || ||
|-
| Developing plugins|| GH || || ||
|-
| <ul><li> A guide to developing plugins </ul>|| || || ||
|-
| <ul><li>Overview of existing pluggable interfaces </ul>|| || || ||
|-
| Krb5 library guide|| || || ||
|-
| <ul><li> A more advanced introduction to using the Kerberos libraries for initial authentication, focusing on the authentication steps, validating initial credential</ul>|| TY || || ||
|-
| <ul><li> Kerberos prompter behavior</ul>|| || || ||
|-
| <ul><li> An introduction to ticket caches and keytabs and their corresponding APIs </ul>|| KR || || ||
|-
| <ul><li> An advanced guide to the pre-auth mechanisms, FAST</ul>|| || || ||
|-
| <ul><li> An advanced guide to the principal manipulation and parsing</ul>|| TY || || ||
|-
| <ul><li> Thread safety</ul>|| KR || || ||
|-
| <ul><li> Password change including the automatic internal support for password change on expired passwords if a prompter is provided</ul>|| || || ||
|-
| <ul><li> krb5_appdefault_* functions and their alternatives </ul>|| || || ||
|-
| MIT Kerberos features : quick facts || ZT || || || ongoing
|-
| How to build Kerberos from source || ZT || || || ready for review
|-
|}

== Administration ==

{| class="wikitable"
|+
|-
! task
! Proposed Author
! Target Date
! Reviewer
! Reviewer Comments
|-
|-
|Setting a new realm|| || || ||
|-
|<ul><li>Choosing backend: LDAP vs DB2</ul>|| || || ||
|-
|<ul><li>Replication</ul>|| ZT|| || ||ready for review
|-
|<ul><li> DNS configuration and SRV records - how they are used, in what order</ul>|| ZT|| || ||
|-
| Integration Kerberos with Login System|| || || ||
|-
| <ul><li> Difference between real Kerberos authentication, Kerberos password verification on the server side, and "LDAP authentication" in a Kerberos environment</ul>|| || || ||
|-
| <ul><li> Validating Kerberos tickets</ul>|| || || ||
|-
| <ul><li> Clear text password over HTTPS </ul>|| || || ||
|-
| <ul><li> Configuring with pam_krb5 module</ul>|| || || ||
|-
| <ul><li> Storing/locating keytab</ul>|| || || ||
|-
| Cross-realm|| || || ||
|-
| <ul><li>cross-realm interaction with AD </ul>|| || || ||
|-
| <ul><li> Transitive trust</ul>|| || || ||
|-
| <ul><li> Referrals</ul>|| || || ||
|-
| Performance|| || || ||
|-
| <ul><li> Performance tuning tips</ul>|| || || ||
|-
| <ul><li> Performance tradeoffs</ul>|| || || ||
|-
| kadmin interface|| || || ||
|-
| <ul><li> Keying workstation/ host key setting</ul>|| || || ||
|-
| Using Smartcard with PKINIT|| || || ||
|-
| Kerberized ssh|| || || ||
|-
| <ul><li> Configuration</ul>|| || || ||
|-
| <ul><li>Cross-realm and ssh</ul>|| || || ||
|-
| Selecting and configuring plugins|| GH || || ||
|-
| Anonymity support|| || || ||
|-
| A guide to principal naming basics and structure|| || || ||
|-
| Troubleshooting|| || || ||
|-
| <ul><li>Troubleshooting errors</ul> || ZT || || || ongoing
|-
| <ul><li>Trace logging</ul>||GH || || ||
|-
| <ul><li>Realm renaming </ul>|| || || ||
|-
| Using LDAP server for Kerberos backend|| ZT || ||Ubuntu 10.4 (lucid)||
|-
|}

== API documentation ==

===Most commonly used API functions (in alphabetical order)===

{| class="wikitable"
|+ Tier 1 - Highest priority
|-
! API
! Proposed Author
! Target Date
! Reviewer
! Reviewer Comments
|-
|-
| krb5_build_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_build_principal.html]|| ZT || GH|| ||
|-
|krb5_build_principal_alloc_va [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_build_principal_alloc_va.html] || ZT || GH|| ||
|-
| krb5_build_principal_ext [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_build_principal_ext.html]|| ZT ||GH || ||
|-
| krb5_cc_close [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_close.html] ||ZT ||GH || ||
|-
| krb5_cc_default [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_default.html]|| ZT|| GH|| ||
|-
| krb5_cc_default_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_default_name.html]|| ZT|| GH|| ||
|-
| krb5_cc_destroy [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_destroy.html]|| ZT|| GH|| ||
|-
| krb5_cc_dup [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_dup.html]|| ZT|| GH|| ||
|-
| krb5_cc_get_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_get_name.html]|| ZT || GH|| ||
|-
| krb5_cc_get_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_get_principal.html]|| ZT ||GH || ||
|-
| krb5_cc_get_type [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_get_type.html]|| ZT ||GH || ||
|-
| krb5_cc_initialize [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_initialize.html]|| ZT||GH || ||
|-
| krb5_cc_new_unique [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_new_unique.html]|| ZT|| GH|| ||
|-
| krb5_cc_resolve [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_resolve.html]|| ZT|| GH|| ||
|-
| krb5_change_password [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_change_password.html]|| ZT||GH || ||
|-
| krb5_free_context [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_free_context.html]|| ZT|| GH|| ||
|-
| krb5_free_error_message [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_free_error_message.html]|| ZT ||GH || ||
|-
| krb5_free_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_free_principal.html]|| ZT ||GH || ||
|-
| krb5_fwd_tgt_cred [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_fwd_tgt_cred.html]|| ZT || GH|| || Needs example
|-
| krb5_get_default_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_default_realm.html]|| ZT || GH|| ||
|-
| krb5_get_error_message [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_error_message.html]|| ZT || GH|| ||
|-
| krb5_get_host_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_host_realm.html]|| ZT ||GH || ||
|-
| krb5_get_credentials [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_credentials.html]|| ZT ||GH || ||
|-
| krb5_get_fallback_host_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_fallback_host_realm.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_keytab [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_keytab.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_alloc [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_alloc.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_free [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_free.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_get_fast_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_get_fast_flags.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_init [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_init.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_address_list [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_address_list.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_anonymous [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_anonymous.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_canonicalize [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_canonicalize.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_change_password_prompt [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_change_password_prompt.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_etype_list [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_etype_list.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_expire_callback [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_expire_callback.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_fast_ccache [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_fast_ccache.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_fast_ccache_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_fast_ccache_name.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_fast_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_fast_flags.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_forwardable [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_forwardable.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_out_ccache [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_out_ccache.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_pa [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_pa.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_preauth_list [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_preauth_list.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_proxiable [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_proxiable.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_renew_life [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_renew_life.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_salt [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_salt.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_tkt_life [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_tkt_life.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_password [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_password.html]|| ZT || GH|| ||
|-
| krb5_get_profile [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_profile.html]|| ZT || GH|| ||
|-
| krb5_get_prompt_types [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_prompt_types.html]|| ZT ||GH || ||
|-
| krb5_get_renewed_creds [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_renewed_creds.html]|| ZT || GH|| ||
|-
| krb5_get_validated_creds [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_validated_creds.html]|| ZT || GH|| ||
|-
| krb5_init_context [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_init_context.html]|| ZT ||GH || ||
|-
| krb5_init_secure_context [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_init_secure_context.html]|| ZT || GH|| ||
|-
| krb5_is_config_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_is_config_principal.html]|| ZT ||GH || ||
|-
| krb5_is_thread_safe [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_is_thread_safe.html]|| ZT ||GH || ||
|-
| krb5_kt_close [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_close.html]|| ZT || GH|| ||
|-
| krb5_kt_default [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_default.html]|| ZT ||GH || ||
|-
| krb5_kt_default_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_default_name.html]|| ZT ||GH || ||
|-
| krb5_kt_get_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_get_name.html]|| ZT ||GH || ||
|-
| krb5_kt_get_type [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_get_type.html] || ZT ||GH || ||
|-
| krb5_kt_resolve [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_resolve.html]|| ZT || GH|| ||
|-
| krb5_kuserok [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kuserok.html] || ZT ||GH || ||
|-
| krb5_parse_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_parse_name.html]|| ZT ||GH || ||
|-
| krb5_parse_name_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_parse_name_flags.html]|| ZT || GH|| ||
|-
| krb5_principal_compare [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_principal_compare.html]|| ZT ||GH || ||
|-
| krb5_principal_compare_any_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_principal_compare_any_realm.html]|| ZT || GH|| ||
|-
| krb5_principal_compare_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_principal_compare_flags.html]|| ZT ||GH || ||
|-
| krb5_prompter_posix [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_prompter_posix.html]|| ZT||GH || ||
|-
| krb5_realm_compare [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_realm_compare.html]|| ZT ||GH || ||
|-
| krb5_recvauth [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_recvauth.html]||ZT ||GH || ||
|-
| krb5_recvauth_version [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_recvauth_version.html] ||ZT ||GH || ||
|-
| krb5_set_default_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_default_realm.html]|| ZT ||GH || ||
|-
| krb5_set_password [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_password.html]|| ZT || GH|| ||
|-
| krb5_set_password_using_ccache [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_password_using_ccache.html] || ZT ||GH || ||
|-
| krb5_set_principal_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_principal_realm.html] || ZT || GH|| ||
|-
| krb5_set_trace_callback [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_trace_callback.html]|| ZT ||GH || ||
|-
| krb5_set_trace_filename [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_trace_filename.html]|| ZT ||GH || ||
|-
| krb5_sname_to_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_sname_to_principal.html]|| ZT ||GH || ||
|-
| krb5_unparse_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name.html]|| ZT || GH|| ||
|-
| krb5_unparse_name_ext [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name_ext.html]|| ZT ||GH || ||
|-
| krb5_unparse_name_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name_flags.html] || ZT || GH|| ||
|-
| krb5_unparse_name_flags_ext [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name_flags_ext.html] || ZT ||GH || ||
|-
| krb5_us_timeofday [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_us_timeofday.html]|| ZT || GH|| ||
|-
| krb5_verify_authdata_kdc_issued [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_verify_authdata_kdc_issued.html]|| ZT || GH|| ||
|-
|}

== Abbreviations ==

{| class="wikitable"
|+
|-
! abbreviation
! full names?
|-
|-
| GH || Greg Hudson
|-
| KR || Ken Raeburn
|-
| MIT || MITKC group
|-
| TY|| Tom Yu
|-
| ZT || Zhanna Tsitkova
|-
|}

Projects/Documentation Tasks

2011-11-08T19:33:27Z

Hardjono: /* Administration */

{{project-early}}

== Purpose ==

To keep track of the various tasks that need to be documented such as function documentation, administration, troubleshooting etc.

{| class="wikitable"
|+ Matrix of Document-Type VS Intended Readership
|-
! Doc-type/Reader
! Architectural Guide
! Setup & Config of Kerberos
! Admin & Operations of Kerberos
! Custom Build
! API Description
! API Details
|-
|-
| End-users || || || || || ||
|-
| Architects || || || || || ||
|-
|System Admins || || || || || ||
|-
|Application Developers || || || || || ||
|-
|GSSAPI Developers || || || || || ||
|-
|Kerberos Developers || || || || || ||
|}

== Application development ==

{| class="wikitable"
|+
|-
! task
! Proposed Author
! Target Date
! Reviewer
! Reviewer Comments
|-
|-
| Designing a new protocol, or extending existing one, to use GSS-API || || || ||
|-
| Choosing security API|| || || ||
|-
| <ul><li> GSS-API vs SASL vs KRB5 </ul>|| || || ||
|-
| <ul><li> A guide to the similarities and differences between Heimdal and MIT Kerberos API </ul>|| || || ||
|-
| GSS-API || || || ||
|-
| <ul><li> A basic introduction to GSS-API, making use of the sample client and server, with special attention paid to Kerberos-related GSS-API issues</ul>|| || || ||
|-
| <ul><li> How to tell the GSS-API library on the client side where the existing Kerberos ticket cache is </ul>|| || || ||
|-
| <ul><li> How to write mechanism-independent GSS-API code</ul>|| || || ||
|-
| <ul><li> Acceptor naming - How to get servers to use any key in a keytab</ul>|| GH|| || ||
|-
| <ul><li> A guide to GSS-API naming as compared to Kerberos principal naming</ul>|| || || ||
|-
| <ul><li> Using IAKERB</ul>|| || || ||
|-
| <ul><li> Anonymous credentials</ul>|| || || ||
|-
| <ul><li> Delegating credentials</ul>|| MIT || || ||
|-
| <ul><li> Available extensions</ul>|| || || ||
|-
| <ul><li> Thread safety</ul>|| KR || || ||
|-
| <ul><li> Validating the flags set on the connection to ensure things like mutual authentication, confidentiality, integrity, replay protection, and sequence protection</ul>|| || || ||
|-
| Developing plugins|| GH || || ||
|-
| <ul><li> A guide to developing plugins </ul>|| || || ||
|-
| <ul><li>Overview of existing pluggable interfaces </ul>|| || || ||
|-
| Krb5 library guide|| || || ||
|-
| <ul><li> A more advanced introduction to using the Kerberos libraries for initial authentication, focusing on the authentication steps, validating initial credential</ul>|| TY || || ||
|-
| <ul><li> Kerberos prompter behavior</ul>|| || || ||
|-
| <ul><li> An introduction to ticket caches and keytabs and their corresponding APIs </ul>|| KR || || ||
|-
| <ul><li> An advanced guide to the pre-auth mechanisms, FAST</ul>|| || || ||
|-
| <ul><li> An advanced guide to the principal manipulation and parsing</ul>|| TY || || ||
|-
| <ul><li> Thread safety</ul>|| KR || || ||
|-
| <ul><li> Password change including the automatic internal support for password change on expired passwords if a prompter is provided</ul>|| || || ||
|-
| <ul><li> krb5_appdefault_* functions and their alternatives </ul>|| || || ||
|-
| MIT Kerberos features : quick facts || ZT || || || ongoing
|-
| How to build Kerberos from source || ZT || || || ready for review
|-
|}

== Administration ==

{| class="wikitable"
|+
|-
! task
! Proposed Author
! Target Date
! Reviewer
! Reviewer Comments
|-
|-
|Setting a new realm|| || || ||
|-
|<ul><li>Choosing backend: LDAP vs DB2</ul>|| || || ||
|-
|<ul><li>Replication</ul>|| ZT|| || ||ready for review
|-
|<ul><li> DNS configuration and SRV records - how they are used, in what order</ul>|| ZT|| || ||
|-
| Integration Kerberos with Login System|| || || ||
|-
| <ul><li> Difference between real Kerberos authentication, Kerberos password verification on the server side, and "LDAP authentication" in a Kerberos environment</ul>|| || || ||
|-
| <ul><li> Validating Kerberos tickets</ul>|| || || ||
|-
| <ul><li> Clear text password over HTTPS </ul>|| || || ||
|-
| <ul><li> Configuring with pam_krb5 module</ul>|| || || ||
|-
| <ul><li> Storing/locating keytab</ul>|| || || ||
|-
| Cross-realm|| || || ||
|-
| <ul><li>cross-realm interaction with AD </ul>|| || || ||
|-
| <ul><li> Transitive trust</ul>|| || || ||
|-
| <ul><li> Referrals</ul>|| || || ||
|-
| Performance|| || || ||
|-
| <ul><li> Performance tuning tips</ul>|| || || ||
|-
| <ul><li> Performance tradeoffs</ul>|| || || ||
|-
| kadmin interface|| || || ||
|-
| <ul><li> Keying workstation/ host key setting</ul>|| || || ||
|-
| Using Smartcard with PKINIT|| || || ||
|-
| Kerberized ssh|| || || ||
|-
| <ul><li> Configuration</ul>|| || || ||
|-
| <ul><li>Cross-realm and ssh</ul>|| || || ||
|-
| Selecting and configuring plugins|| GH || || ||
|-
| Anonymity support|| || || ||
|-
| A guide to principal naming basics and structure|| || || ||
|-
| Troubleshooting|| || || ||
|-
| <ul><li>Troubleshooting errors</ul> || ZT || || || ongoing
|-
| <ul><li>Trace logging</ul>||GH || || ||
|-
| <ul><li>Realm renaming </ul>|| || || ||
|-
| Using LDAP server for Kerberos backend|| ZT || ||Ubuntu 10.4 (lucid)||
|-
|}

== API documentation ==

===Most commonly used API functions (in alphabetical order)===

{| class="wikitable"
|+ Tier 1 - Highest priority
|-
! API
! who writes?
! who reviews?
! reviewed?
! comments
|-
|-
| krb5_build_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_build_principal.html]|| ZT || GH|| ||
|-
|krb5_build_principal_alloc_va [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_build_principal_alloc_va.html] || ZT || GH|| ||
|-
| krb5_build_principal_ext [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_build_principal_ext.html]|| ZT ||GH || ||
|-
| krb5_cc_close [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_close.html] ||ZT ||GH || ||
|-
| krb5_cc_default [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_default.html]|| ZT|| GH|| ||
|-
| krb5_cc_default_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_default_name.html]|| ZT|| GH|| ||
|-
| krb5_cc_destroy [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_destroy.html]|| ZT|| GH|| ||
|-
| krb5_cc_dup [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_dup.html]|| ZT|| GH|| ||
|-
| krb5_cc_get_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_get_name.html]|| ZT || GH|| ||
|-
| krb5_cc_get_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_get_principal.html]|| ZT ||GH || ||
|-
| krb5_cc_get_type [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_get_type.html]|| ZT ||GH || ||
|-
| krb5_cc_initialize [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_initialize.html]|| ZT||GH || ||
|-
| krb5_cc_new_unique [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_new_unique.html]|| ZT|| GH|| ||
|-
| krb5_cc_resolve [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_resolve.html]|| ZT|| GH|| ||
|-
| krb5_change_password [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_change_password.html]|| ZT||GH || ||
|-
| krb5_free_context [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_free_context.html]|| ZT|| GH|| ||
|-
| krb5_free_error_message [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_free_error_message.html]|| ZT ||GH || ||
|-
| krb5_free_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_free_principal.html]|| ZT ||GH || ||
|-
| krb5_fwd_tgt_cred [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_fwd_tgt_cred.html]|| ZT || GH|| || Needs example
|-
| krb5_get_default_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_default_realm.html]|| ZT || GH|| ||
|-
| krb5_get_error_message [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_error_message.html]|| ZT || GH|| ||
|-
| krb5_get_host_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_host_realm.html]|| ZT ||GH || ||
|-
| krb5_get_credentials [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_credentials.html]|| ZT ||GH || ||
|-
| krb5_get_fallback_host_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_fallback_host_realm.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_keytab [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_keytab.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_alloc [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_alloc.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_free [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_free.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_get_fast_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_get_fast_flags.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_init [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_init.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_address_list [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_address_list.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_anonymous [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_anonymous.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_canonicalize [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_canonicalize.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_change_password_prompt [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_change_password_prompt.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_etype_list [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_etype_list.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_expire_callback [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_expire_callback.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_fast_ccache [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_fast_ccache.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_fast_ccache_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_fast_ccache_name.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_fast_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_fast_flags.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_forwardable [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_forwardable.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_out_ccache [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_out_ccache.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_pa [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_pa.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_preauth_list [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_preauth_list.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_proxiable [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_proxiable.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_renew_life [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_renew_life.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_salt [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_salt.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_tkt_life [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_tkt_life.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_password [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_password.html]|| ZT || GH|| ||
|-
| krb5_get_profile [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_profile.html]|| ZT || GH|| ||
|-
| krb5_get_prompt_types [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_prompt_types.html]|| ZT ||GH || ||
|-
| krb5_get_renewed_creds [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_renewed_creds.html]|| ZT || GH|| ||
|-
| krb5_get_validated_creds [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_validated_creds.html]|| ZT || GH|| ||
|-
| krb5_init_context [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_init_context.html]|| ZT ||GH || ||
|-
| krb5_init_secure_context [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_init_secure_context.html]|| ZT || GH|| ||
|-
| krb5_is_config_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_is_config_principal.html]|| ZT ||GH || ||
|-
| krb5_is_thread_safe [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_is_thread_safe.html]|| ZT ||GH || ||
|-
| krb5_kt_close [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_close.html]|| ZT || GH|| ||
|-
| krb5_kt_default [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_default.html]|| ZT ||GH || ||
|-
| krb5_kt_default_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_default_name.html]|| ZT ||GH || ||
|-
| krb5_kt_get_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_get_name.html]|| ZT ||GH || ||
|-
| krb5_kt_get_type [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_get_type.html] || ZT ||GH || ||
|-
| krb5_kt_resolve [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_resolve.html]|| ZT || GH|| ||
|-
| krb5_kuserok [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kuserok.html] || ZT ||GH || ||
|-
| krb5_parse_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_parse_name.html]|| ZT ||GH || ||
|-
| krb5_parse_name_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_parse_name_flags.html]|| ZT || GH|| ||
|-
| krb5_principal_compare [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_principal_compare.html]|| ZT ||GH || ||
|-
| krb5_principal_compare_any_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_principal_compare_any_realm.html]|| ZT || GH|| ||
|-
| krb5_principal_compare_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_principal_compare_flags.html]|| ZT ||GH || ||
|-
| krb5_prompter_posix [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_prompter_posix.html]|| ZT||GH || ||
|-
| krb5_realm_compare [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_realm_compare.html]|| ZT ||GH || ||
|-
| krb5_recvauth [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_recvauth.html]||ZT ||GH || ||
|-
| krb5_recvauth_version [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_recvauth_version.html] ||ZT ||GH || ||
|-
| krb5_set_default_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_default_realm.html]|| ZT ||GH || ||
|-
| krb5_set_password [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_password.html]|| ZT || GH|| ||
|-
| krb5_set_password_using_ccache [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_password_using_ccache.html] || ZT ||GH || ||
|-
| krb5_set_principal_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_principal_realm.html] || ZT || GH|| ||
|-
| krb5_set_trace_callback [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_trace_callback.html]|| ZT ||GH || ||
|-
| krb5_set_trace_filename [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_trace_filename.html]|| ZT ||GH || ||
|-
| krb5_sname_to_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_sname_to_principal.html]|| ZT ||GH || ||
|-
| krb5_unparse_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name.html]|| ZT || GH|| ||
|-
| krb5_unparse_name_ext [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name_ext.html]|| ZT ||GH || ||
|-
| krb5_unparse_name_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name_flags.html] || ZT || GH|| ||
|-
| krb5_unparse_name_flags_ext [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name_flags_ext.html] || ZT ||GH || ||
|-
| krb5_us_timeofday [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_us_timeofday.html]|| ZT || GH|| ||
|-
| krb5_verify_authdata_kdc_issued [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_verify_authdata_kdc_issued.html]|| ZT || GH|| ||
|-
|}

== Abbreviations ==

{| class="wikitable"
|+
|-
! abbreviation
! full names?
|-
|-
| GH || Greg Hudson
|-
| KR || Ken Raeburn
|-
| MIT || MITKC group
|-
| TY|| Tom Yu
|-
| ZT || Zhanna Tsitkova
|-
|}

Projects/Documentation Tasks

2011-11-08T16:29:24Z

Hardjono: /* Application development */

{{project-early}}

== Purpose ==

To keep track of the various tasks that need to be documented such as function documentation, administration, troubleshooting etc.

{| class="wikitable"
|+ Matrix of Document-Type VS Intended Readership
|-
! Doc-type/Reader
! Architectural Guide
! Setup & Config of Kerberos
! Admin & Operations of Kerberos
! Custom Build
! API Description
! API Details
|-
|-
| End-users || || || || || ||
|-
| Architects || || || || || ||
|-
|System Admins || || || || || ||
|-
|Application Developers || || || || || ||
|-
|GSSAPI Developers || || || || || ||
|-
|Kerberos Developers || || || || || ||
|}

== Application development ==

{| class="wikitable"
|+
|-
! task
! Proposed Author
! Target Date
! Reviewer
! Reviewer Comments
|-
|-
| Designing a new protocol, or extending existing one, to use GSS-API || || || ||
|-
| Choosing security API|| || || ||
|-
| <ul><li> GSS-API vs SASL vs KRB5 </ul>|| || || ||
|-
| <ul><li> A guide to the similarities and differences between Heimdal and MIT Kerberos API </ul>|| || || ||
|-
| GSS-API || || || ||
|-
| <ul><li> A basic introduction to GSS-API, making use of the sample client and server, with special attention paid to Kerberos-related GSS-API issues</ul>|| || || ||
|-
| <ul><li> How to tell the GSS-API library on the client side where the existing Kerberos ticket cache is </ul>|| || || ||
|-
| <ul><li> How to write mechanism-independent GSS-API code</ul>|| || || ||
|-
| <ul><li> Acceptor naming - How to get servers to use any key in a keytab</ul>|| GH|| || ||
|-
| <ul><li> A guide to GSS-API naming as compared to Kerberos principal naming</ul>|| || || ||
|-
| <ul><li> Using IAKERB</ul>|| || || ||
|-
| <ul><li> Anonymous credentials</ul>|| || || ||
|-
| <ul><li> Delegating credentials</ul>|| MIT || || ||
|-
| <ul><li> Available extensions</ul>|| || || ||
|-
| <ul><li> Thread safety</ul>|| KR || || ||
|-
| <ul><li> Validating the flags set on the connection to ensure things like mutual authentication, confidentiality, integrity, replay protection, and sequence protection</ul>|| || || ||
|-
| Developing plugins|| GH || || ||
|-
| <ul><li> A guide to developing plugins </ul>|| || || ||
|-
| <ul><li>Overview of existing pluggable interfaces </ul>|| || || ||
|-
| Krb5 library guide|| || || ||
|-
| <ul><li> A more advanced introduction to using the Kerberos libraries for initial authentication, focusing on the authentication steps, validating initial credential</ul>|| TY || || ||
|-
| <ul><li> Kerberos prompter behavior</ul>|| || || ||
|-
| <ul><li> An introduction to ticket caches and keytabs and their corresponding APIs </ul>|| KR || || ||
|-
| <ul><li> An advanced guide to the pre-auth mechanisms, FAST</ul>|| || || ||
|-
| <ul><li> An advanced guide to the principal manipulation and parsing</ul>|| TY || || ||
|-
| <ul><li> Thread safety</ul>|| KR || || ||
|-
| <ul><li> Password change including the automatic internal support for password change on expired passwords if a prompter is provided</ul>|| || || ||
|-
| <ul><li> krb5_appdefault_* functions and their alternatives </ul>|| || || ||
|-
| MIT Kerberos features : quick facts || ZT || || || ongoing
|-
| How to build Kerberos from source || ZT || || || ready for review
|-
|}

== Administration ==

{| class="wikitable"
|+
|-
! task
! who writes/due date
! who reviews/due date
! comments
! status
|-
|-
|Setting a new realm|| || || ||
|-
|<ul><li>Choosing backend: LDAP vs DB2</ul>|| || || ||
|-
|<ul><li>Replication</ul>|| ZT|| || ||ready for review
|-
|<ul><li> DNS configuration and SRV records - how they are used, in what order</ul>|| ZT|| || ||
|-
| Integration Kerberos with Login System|| || || ||
|-
| <ul><li> Difference between real Kerberos authentication, Kerberos password verification on the server side, and "LDAP authentication" in a Kerberos environment</ul>|| || || ||
|-
| <ul><li> Validating Kerberos tickets</ul>|| || || ||
|-
| <ul><li> Clear text password over HTTPS </ul>|| || || ||
|-
| <ul><li> Configuring with pam_krb5 module</ul>|| || || ||
|-
| <ul><li> Storing/locating keytab</ul>|| || || ||
|-
| Cross-realm|| || || ||
|-
| <ul><li>cross-realm interaction with AD </ul>|| || || ||
|-
| <ul><li> Transitive trust</ul>|| || || ||
|-
| <ul><li> Referrals</ul>|| || || ||
|-
| Performance|| || || ||
|-
| <ul><li> Performance tuning tips</ul>|| || || ||
|-
| <ul><li> Performance tradeoffs</ul>|| || || ||
|-
| kadmin interface|| || || ||
|-
| <ul><li> Keying workstation/ host key setting</ul>|| || || ||
|-
| Using Smartcard with PKINIT|| || || ||
|-
| Kerberized ssh|| || || ||
|-
| <ul><li> Configuration</ul>|| || || ||
|-
| <ul><li>Cross-realm and ssh</ul>|| || || ||
|-
| Selecting and configuring plugins|| GH || || ||
|-
| Anonymity support|| || || ||
|-
| A guide to principal naming basics and structure|| || || ||
|-
| Troubleshooting|| || || ||
|-
| <ul><li>Troubleshooting errors</ul> || ZT || || || ongoing
|-
| <ul><li>Trace logging</ul>||GH || || ||
|-
| <ul><li>Realm renaming </ul>|| || || ||
|-
| Using LDAP server for Kerberos backend|| ZT || ||Ubuntu 10.4 (lucid)||
|-
|}

== API documentation ==

===Most commonly used API functions (in alphabetical order)===

{| class="wikitable"
|+ Tier 1 - Highest priority
|-
! API
! who writes?
! who reviews?
! reviewed?
! comments
|-
|-
| krb5_build_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_build_principal.html]|| ZT || GH|| ||
|-
|krb5_build_principal_alloc_va [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_build_principal_alloc_va.html] || ZT || GH|| ||
|-
| krb5_build_principal_ext [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_build_principal_ext.html]|| ZT ||GH || ||
|-
| krb5_cc_close [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_close.html] ||ZT ||GH || ||
|-
| krb5_cc_default [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_default.html]|| ZT|| GH|| ||
|-
| krb5_cc_default_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_default_name.html]|| ZT|| GH|| ||
|-
| krb5_cc_destroy [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_destroy.html]|| ZT|| GH|| ||
|-
| krb5_cc_dup [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_dup.html]|| ZT|| GH|| ||
|-
| krb5_cc_get_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_get_name.html]|| ZT || GH|| ||
|-
| krb5_cc_get_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_get_principal.html]|| ZT ||GH || ||
|-
| krb5_cc_get_type [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_get_type.html]|| ZT ||GH || ||
|-
| krb5_cc_initialize [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_initialize.html]|| ZT||GH || ||
|-
| krb5_cc_new_unique [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_new_unique.html]|| ZT|| GH|| ||
|-
| krb5_cc_resolve [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_cc_resolve.html]|| ZT|| GH|| ||
|-
| krb5_change_password [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_change_password.html]|| ZT||GH || ||
|-
| krb5_free_context [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_free_context.html]|| ZT|| GH|| ||
|-
| krb5_free_error_message [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_free_error_message.html]|| ZT ||GH || ||
|-
| krb5_free_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_free_principal.html]|| ZT ||GH || ||
|-
| krb5_fwd_tgt_cred [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_fwd_tgt_cred.html]|| ZT || GH|| || Needs example
|-
| krb5_get_default_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_default_realm.html]|| ZT || GH|| ||
|-
| krb5_get_error_message [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_error_message.html]|| ZT || GH|| ||
|-
| krb5_get_host_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_host_realm.html]|| ZT ||GH || ||
|-
| krb5_get_credentials [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_credentials.html]|| ZT ||GH || ||
|-
| krb5_get_fallback_host_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_fallback_host_realm.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_keytab [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_keytab.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_alloc [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_alloc.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_free [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_free.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_get_fast_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_get_fast_flags.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_init [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_init.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_address_list [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_address_list.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_anonymous [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_anonymous.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_canonicalize [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_canonicalize.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_change_password_prompt [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_change_password_prompt.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_etype_list [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_etype_list.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_expire_callback [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_expire_callback.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_fast_ccache [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_fast_ccache.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_fast_ccache_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_fast_ccache_name.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_fast_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_fast_flags.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_forwardable [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_forwardable.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_out_ccache [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_out_ccache.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_pa [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_pa.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_preauth_list [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_preauth_list.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_proxiable [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_proxiable.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_renew_life [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_renew_life.html]|| ZT || GH|| ||
|-
| krb5_get_init_creds_opt_set_salt [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_salt.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_opt_set_tkt_life [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_opt_set_tkt_life.html]|| ZT ||GH || ||
|-
| krb5_get_init_creds_password [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_init_creds_password.html]|| ZT || GH|| ||
|-
| krb5_get_profile [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_profile.html]|| ZT || GH|| ||
|-
| krb5_get_prompt_types [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_prompt_types.html]|| ZT ||GH || ||
|-
| krb5_get_renewed_creds [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_renewed_creds.html]|| ZT || GH|| ||
|-
| krb5_get_validated_creds [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_get_validated_creds.html]|| ZT || GH|| ||
|-
| krb5_init_context [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_init_context.html]|| ZT ||GH || ||
|-
| krb5_init_secure_context [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_init_secure_context.html]|| ZT || GH|| ||
|-
| krb5_is_config_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_is_config_principal.html]|| ZT ||GH || ||
|-
| krb5_is_thread_safe [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_is_thread_safe.html]|| ZT ||GH || ||
|-
| krb5_kt_close [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_close.html]|| ZT || GH|| ||
|-
| krb5_kt_default [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_default.html]|| ZT ||GH || ||
|-
| krb5_kt_default_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_default_name.html]|| ZT ||GH || ||
|-
| krb5_kt_get_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_get_name.html]|| ZT ||GH || ||
|-
| krb5_kt_get_type [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_get_type.html] || ZT ||GH || ||
|-
| krb5_kt_resolve [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kt_resolve.html]|| ZT || GH|| ||
|-
| krb5_kuserok [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_kuserok.html] || ZT ||GH || ||
|-
| krb5_parse_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_parse_name.html]|| ZT ||GH || ||
|-
| krb5_parse_name_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_parse_name_flags.html]|| ZT || GH|| ||
|-
| krb5_principal_compare [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_principal_compare.html]|| ZT ||GH || ||
|-
| krb5_principal_compare_any_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_principal_compare_any_realm.html]|| ZT || GH|| ||
|-
| krb5_principal_compare_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_principal_compare_flags.html]|| ZT ||GH || ||
|-
| krb5_prompter_posix [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_prompter_posix.html]|| ZT||GH || ||
|-
| krb5_realm_compare [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_realm_compare.html]|| ZT ||GH || ||
|-
| krb5_recvauth [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_recvauth.html]||ZT ||GH || ||
|-
| krb5_recvauth_version [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_recvauth_version.html] ||ZT ||GH || ||
|-
| krb5_set_default_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_default_realm.html]|| ZT ||GH || ||
|-
| krb5_set_password [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_password.html]|| ZT || GH|| ||
|-
| krb5_set_password_using_ccache [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_password_using_ccache.html] || ZT ||GH || ||
|-
| krb5_set_principal_realm [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_principal_realm.html] || ZT || GH|| ||
|-
| krb5_set_trace_callback [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_trace_callback.html]|| ZT ||GH || ||
|-
| krb5_set_trace_filename [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_set_trace_filename.html]|| ZT ||GH || ||
|-
| krb5_sname_to_principal [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_sname_to_principal.html]|| ZT ||GH || ||
|-
| krb5_unparse_name [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name.html]|| ZT || GH|| ||
|-
| krb5_unparse_name_ext [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name_ext.html]|| ZT ||GH || ||
|-
| krb5_unparse_name_flags [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name_flags.html] || ZT || GH|| ||
|-
| krb5_unparse_name_flags_ext [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_unparse_name_flags_ext.html] || ZT ||GH || ||
|-
| krb5_us_timeofday [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_us_timeofday.html]|| ZT || GH|| ||
|-
| krb5_verify_authdata_kdc_issued [http://web.mit.edu/tsitkova/www/build/refs/api/krb5_verify_authdata_kdc_issued.html]|| ZT || GH|| ||
|-
|}

== Abbreviations ==

{| class="wikitable"
|+
|-
! abbreviation
! full names?
|-
|-
| GH || Greg Hudson
|-
| KR || Ken Raeburn
|-
| MIT || MITKC group
|-
| TY|| Tom Yu
|-
| ZT || Zhanna Tsitkova
|-
|}

Release 1.10

2011-05-19T15:17:02Z

Hardjono:

== Timeline ==

This is only an approximate timeline.

* Jul. 2010 -- make release branch
* Oct. 2010 -- final release

== Goals ==

* Kernel / user split (for NFS, etc.): Add build infrastructure demonstrating and testing a message-processing subset of the gss-krb5 mechanism suitable for kernel filesystems.
* Localization: Create infrastructure for localization of client user interface messages using gettext.
* Improve API documentation: Create documentation for the libkrb5 API.
* Selective refactoring of KDC (to support libKDC etc.): Reorganize parts of the KDC code for improved modularity and maintainability.
* Pluggable configuration back-end: Allow applications and integrators to override krb5.conf as the source of krb5 configuration data.
* Credential selection: Add a facility to select between credentials for different Kerberos identities based on the service being contacted. (This will be confirmed).
* Referrals: Finish implementation following IETF updates.
* PKINIT hash agility: Allow PKINIT to use newer hash algorithms than SHA-1.

Projects/Documentation Tasks

2011-01-13T17:14:21Z

Hardjono: /* API documentation */

{{project-early}}

== Purpose ==

To keep track of the various tasks that need to be documented such as function documentation, administration, troubleshooting etc.

=== API documentation ===

{| class="wikitable"
|+ Matrix of Document-Type VS Intended Readership
|-
! Doc-type/Reader
! Architectural Guide
! Setup & Config of Kerberos
! Admin & Operations of Kerberos
! Custom Build
! API Description
! API Details
|-
|-
| End-users || || || || || || ||
|-
| Architects || || || || || || ||
|-
|System Admins || || || || || || ||
|-
|Application Developers || || || || || || ||
|-
|GSSAPI Developers || || || || || || ||
|-
|Kerberos Developers || || || || || || ||
|}

{| class="wikitable"
|+ Tier 1 - most commonly used API functions (in alphabetical order)
|-
! API
! who writes?
! who reviews?
! reviewed?
! comments
|-
|-
| krb5_build_principal || || || || ||
|-
| krb5_build_principal || || || || ||
|-
|krb5_build_principal_alloc_val || || || || ||
|-
| krb5_build_principal_ext || || || || ||
|-
| krb5_cc_close || || || || ||
|-
| krb5_cc_default || || || || ||
|-
| krb5_cc_default_name || || || || ||
|-
| krb5_cc_destroy || || || || ||
|-
| krb5_cc_dup || || || || ||
|-
| krb5_cc_get_name || || || || ||
|-
| krb5_cc_get_principal || || || || ||
|-
| krb5_cc_get_type || || || || ||
|-
| krb5_cc_initialize || || || || ||
|-
| krb5_cc_new_unique || || || || ||
|-
| krb5_cc_resolve || || || || ||
|-
| krb5_change_password || || || || ||
|-
| krb5_free_context || || || || ||
|-
| krb5_free_error_message || || || || ||
|-
| krb5_free_principal || || || || ||
|-
| krb5_fwd_tgt_cred || || || || ||
|-
| krb5_get_default_realm || || || || ||
|-
| krb5_get_error_message || || || || ||
|-
| krb5_get_host_realm || || || || ||
|-
| krb5_get_credentials || || || || ||
|-
| krb5_get_fallback_host_realm || || || || ||
|-
| krb5_get_init_creds_keytab || || || || ||
|-
| krb5_get_init_creds_opt_alloc || || || || ||
|-
| krb5_get_init_creds_opt_free || || || || ||
|-
| krb5_get_init_creds_opt_get_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_init || || || || ||
|-
| krb5_get_init_creds_opt_set_address_list || || || || ||
|-
| krb5_get_init_creds_opt_set_anonymous || || || || ||
|-
| krb5_get_init_creds_opt_set_canonicalize || || || || ||
|-
| krb5_get_init_creds_opt_set_change_password_prompt || || || || ||
|-
| krb5_get_init_creds_opt_set_etype_list || || || || ||
|-
| krb5_get_init_creds_opt_set_expire_callback || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache_name || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_set_forwardable || || || || ||
|-
| krb5_get_init_creds_opt_set_out_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_pa || || || || ||
|-
| krb5_get_init_creds_opt_set_preauth_list || || || || ||
|-
| krb5_get_init_creds_opt_set_proxiable || || || || ||
|-
| krb5_get_init_creds_opt_set_renew_life || || || || ||
|-
| krb5_get_init_creds_opt_set_salt || || || || ||
|-
| krb5_get_init_creds_opt_set_tkt_life || || || || ||
|-
| krb5_get_init_creds_password || || || || ||
|-
| krb5_get_profile || || || || ||
|-
| krb5_get_prompt_types || || || || ||
|-
| krb5_get_renewed_creds || || || || ||
|-
| krb5_get_validated_creds || || || || ||
|-
| krb5_init_context || || || || ||
|-
| krb5_init_secure_context || || || || ||
|-
| krb5_is_config_principal || || || || ||
|-
| krb5_is_thread_safe || || || || ||
|-
| krb5_kt_close || || || || ||
|-
| krb5_kt_default || || || || ||
|-
| krb5_kt_default_name || || || || ||
|-
| krb5_kt_get_name || || || || ||
|-
| krb5_kt_get_type || || || || ||
|-
| krb5_kt_resolve || || || || ||
|-
| krb5_kuserok || || || || ||
|-
| krb5_parse_name || || || || ||
|-
| krb5_parse_name_flags || || || || ||
|-
| krb5_principal_compare || || || || ||
|-
| krb5_principal_compare_any_realm || || || || ||
|-
| krb5_principal_compare_flags || || || || ||
|-
| krb5_prompter_posix || || || || ||
|-
| krb5_realm_compare || || || || ||
|-
| krb5_recvauth || || || || ||
|-
| krb5_recvauth_version || || || || ||
|-
| krb5_set_default_realm || || || || ||
|-
| krb5_set_password || || || || ||
|-
| krb5_set_password_using_ccache || || || || ||
|-
| krb5_set_principal_realm || || || || ||
|-
| krb5_set_trace_callback || || || || ||
|-
| krb5_set_trace_filename || || || || ||
|-
| krb5_sname_to_principal || || || || ||
|-
| krb5_unparse_name || || || || ||
|-
| krb5_unparse_name_ext || || || || ||
|-
| krb5_unparse_name_flags || || || || ||
|-
| krb5_unparse_name_flags_ext || || || || ||
|-
| krb5_us_timeofday || || || || ||
|-
| krb5_verify_authdata_kdc_issued || || || || ||
|-
|}

Projects/Documentation Tasks

2011-01-13T17:13:45Z

Hardjono: /* API documentation */

{{project-early}}

== Purpose ==

To keep track of the various tasks that need to be documented such as function documentation, administration, troubleshooting etc.

=== API documentation ===

{| class="wikitable"
|+ Matrix of Document-Type VS Intended Readership
|-
! Doc-type/Reader
! Architectural Guide
! Setup and Config of Kerberos
! Admin & Operations of Kerberos
! Custom Build
! API Description
! API Details
|-
|-
| End-users || || || || || || ||
|-
| Architects || || || || || || ||
|-
|Sys Admins || || || || || || ||
|-
|Application Developers || || || || || || ||
|-
|GSSAPI Developers || || || || || || ||
|-
|Kerberos Developers || || || || || || ||
|}

{| class="wikitable"
|+ Tier 1 - most commonly used API functions (in alphabetical order)
|-
! API
! who writes?
! who reviews?
! reviewed?
! comments
|-
|-
| krb5_build_principal || || || || ||
|-
| krb5_build_principal || || || || ||
|-
|krb5_build_principal_alloc_val || || || || ||
|-
| krb5_build_principal_ext || || || || ||
|-
| krb5_cc_close || || || || ||
|-
| krb5_cc_default || || || || ||
|-
| krb5_cc_default_name || || || || ||
|-
| krb5_cc_destroy || || || || ||
|-
| krb5_cc_dup || || || || ||
|-
| krb5_cc_get_name || || || || ||
|-
| krb5_cc_get_principal || || || || ||
|-
| krb5_cc_get_type || || || || ||
|-
| krb5_cc_initialize || || || || ||
|-
| krb5_cc_new_unique || || || || ||
|-
| krb5_cc_resolve || || || || ||
|-
| krb5_change_password || || || || ||
|-
| krb5_free_context || || || || ||
|-
| krb5_free_error_message || || || || ||
|-
| krb5_free_principal || || || || ||
|-
| krb5_fwd_tgt_cred || || || || ||
|-
| krb5_get_default_realm || || || || ||
|-
| krb5_get_error_message || || || || ||
|-
| krb5_get_host_realm || || || || ||
|-
| krb5_get_credentials || || || || ||
|-
| krb5_get_fallback_host_realm || || || || ||
|-
| krb5_get_init_creds_keytab || || || || ||
|-
| krb5_get_init_creds_opt_alloc || || || || ||
|-
| krb5_get_init_creds_opt_free || || || || ||
|-
| krb5_get_init_creds_opt_get_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_init || || || || ||
|-
| krb5_get_init_creds_opt_set_address_list || || || || ||
|-
| krb5_get_init_creds_opt_set_anonymous || || || || ||
|-
| krb5_get_init_creds_opt_set_canonicalize || || || || ||
|-
| krb5_get_init_creds_opt_set_change_password_prompt || || || || ||
|-
| krb5_get_init_creds_opt_set_etype_list || || || || ||
|-
| krb5_get_init_creds_opt_set_expire_callback || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache_name || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_set_forwardable || || || || ||
|-
| krb5_get_init_creds_opt_set_out_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_pa || || || || ||
|-
| krb5_get_init_creds_opt_set_preauth_list || || || || ||
|-
| krb5_get_init_creds_opt_set_proxiable || || || || ||
|-
| krb5_get_init_creds_opt_set_renew_life || || || || ||
|-
| krb5_get_init_creds_opt_set_salt || || || || ||
|-
| krb5_get_init_creds_opt_set_tkt_life || || || || ||
|-
| krb5_get_init_creds_password || || || || ||
|-
| krb5_get_profile || || || || ||
|-
| krb5_get_prompt_types || || || || ||
|-
| krb5_get_renewed_creds || || || || ||
|-
| krb5_get_validated_creds || || || || ||
|-
| krb5_init_context || || || || ||
|-
| krb5_init_secure_context || || || || ||
|-
| krb5_is_config_principal || || || || ||
|-
| krb5_is_thread_safe || || || || ||
|-
| krb5_kt_close || || || || ||
|-
| krb5_kt_default || || || || ||
|-
| krb5_kt_default_name || || || || ||
|-
| krb5_kt_get_name || || || || ||
|-
| krb5_kt_get_type || || || || ||
|-
| krb5_kt_resolve || || || || ||
|-
| krb5_kuserok || || || || ||
|-
| krb5_parse_name || || || || ||
|-
| krb5_parse_name_flags || || || || ||
|-
| krb5_principal_compare || || || || ||
|-
| krb5_principal_compare_any_realm || || || || ||
|-
| krb5_principal_compare_flags || || || || ||
|-
| krb5_prompter_posix || || || || ||
|-
| krb5_realm_compare || || || || ||
|-
| krb5_recvauth || || || || ||
|-
| krb5_recvauth_version || || || || ||
|-
| krb5_set_default_realm || || || || ||
|-
| krb5_set_password || || || || ||
|-
| krb5_set_password_using_ccache || || || || ||
|-
| krb5_set_principal_realm || || || || ||
|-
| krb5_set_trace_callback || || || || ||
|-
| krb5_set_trace_filename || || || || ||
|-
| krb5_sname_to_principal || || || || ||
|-
| krb5_unparse_name || || || || ||
|-
| krb5_unparse_name_ext || || || || ||
|-
| krb5_unparse_name_flags || || || || ||
|-
| krb5_unparse_name_flags_ext || || || || ||
|-
| krb5_us_timeofday || || || || ||
|-
| krb5_verify_authdata_kdc_issued || || || || ||
|-
|}

Projects/Documentation Tasks

2011-01-13T17:13:21Z

Hardjono: /* API documentation */

{{project-early}}

== Purpose ==

To keep track of the various tasks that need to be documented such as function documentation, administration, troubleshooting etc.

=== API documentation ===

{| class="wikitable"
|+ Matrix of Document-Type VS Intended Readership
|-
! Doc-type/Reader
! Architectural Guide
! Setup and Config of Kerberos
! Admin & Operations of Kerberos
! Custom Build
! API Description
|-
|-
| End-users || || || || || || ||
|-
| Architects || || || || || || ||
|-
|Sys Admins || || || || || || ||
|-
|Application Developers || || || || || || ||
|-
|GSSAPI Developers || || || || || || ||
|-
|Kerberos Developers || || || || || || ||
|}

{| class="wikitable"
|+ Tier 1 - most commonly used API functions (in alphabetical order)
|-
! API
! who writes?
! who reviews?
! reviewed?
! comments
|-
|-
| krb5_build_principal || || || || ||
|-
| krb5_build_principal || || || || ||
|-
|krb5_build_principal_alloc_val || || || || ||
|-
| krb5_build_principal_ext || || || || ||
|-
| krb5_cc_close || || || || ||
|-
| krb5_cc_default || || || || ||
|-
| krb5_cc_default_name || || || || ||
|-
| krb5_cc_destroy || || || || ||
|-
| krb5_cc_dup || || || || ||
|-
| krb5_cc_get_name || || || || ||
|-
| krb5_cc_get_principal || || || || ||
|-
| krb5_cc_get_type || || || || ||
|-
| krb5_cc_initialize || || || || ||
|-
| krb5_cc_new_unique || || || || ||
|-
| krb5_cc_resolve || || || || ||
|-
| krb5_change_password || || || || ||
|-
| krb5_free_context || || || || ||
|-
| krb5_free_error_message || || || || ||
|-
| krb5_free_principal || || || || ||
|-
| krb5_fwd_tgt_cred || || || || ||
|-
| krb5_get_default_realm || || || || ||
|-
| krb5_get_error_message || || || || ||
|-
| krb5_get_host_realm || || || || ||
|-
| krb5_get_credentials || || || || ||
|-
| krb5_get_fallback_host_realm || || || || ||
|-
| krb5_get_init_creds_keytab || || || || ||
|-
| krb5_get_init_creds_opt_alloc || || || || ||
|-
| krb5_get_init_creds_opt_free || || || || ||
|-
| krb5_get_init_creds_opt_get_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_init || || || || ||
|-
| krb5_get_init_creds_opt_set_address_list || || || || ||
|-
| krb5_get_init_creds_opt_set_anonymous || || || || ||
|-
| krb5_get_init_creds_opt_set_canonicalize || || || || ||
|-
| krb5_get_init_creds_opt_set_change_password_prompt || || || || ||
|-
| krb5_get_init_creds_opt_set_etype_list || || || || ||
|-
| krb5_get_init_creds_opt_set_expire_callback || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache_name || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_set_forwardable || || || || ||
|-
| krb5_get_init_creds_opt_set_out_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_pa || || || || ||
|-
| krb5_get_init_creds_opt_set_preauth_list || || || || ||
|-
| krb5_get_init_creds_opt_set_proxiable || || || || ||
|-
| krb5_get_init_creds_opt_set_renew_life || || || || ||
|-
| krb5_get_init_creds_opt_set_salt || || || || ||
|-
| krb5_get_init_creds_opt_set_tkt_life || || || || ||
|-
| krb5_get_init_creds_password || || || || ||
|-
| krb5_get_profile || || || || ||
|-
| krb5_get_prompt_types || || || || ||
|-
| krb5_get_renewed_creds || || || || ||
|-
| krb5_get_validated_creds || || || || ||
|-
| krb5_init_context || || || || ||
|-
| krb5_init_secure_context || || || || ||
|-
| krb5_is_config_principal || || || || ||
|-
| krb5_is_thread_safe || || || || ||
|-
| krb5_kt_close || || || || ||
|-
| krb5_kt_default || || || || ||
|-
| krb5_kt_default_name || || || || ||
|-
| krb5_kt_get_name || || || || ||
|-
| krb5_kt_get_type || || || || ||
|-
| krb5_kt_resolve || || || || ||
|-
| krb5_kuserok || || || || ||
|-
| krb5_parse_name || || || || ||
|-
| krb5_parse_name_flags || || || || ||
|-
| krb5_principal_compare || || || || ||
|-
| krb5_principal_compare_any_realm || || || || ||
|-
| krb5_principal_compare_flags || || || || ||
|-
| krb5_prompter_posix || || || || ||
|-
| krb5_realm_compare || || || || ||
|-
| krb5_recvauth || || || || ||
|-
| krb5_recvauth_version || || || || ||
|-
| krb5_set_default_realm || || || || ||
|-
| krb5_set_password || || || || ||
|-
| krb5_set_password_using_ccache || || || || ||
|-
| krb5_set_principal_realm || || || || ||
|-
| krb5_set_trace_callback || || || || ||
|-
| krb5_set_trace_filename || || || || ||
|-
| krb5_sname_to_principal || || || || ||
|-
| krb5_unparse_name || || || || ||
|-
| krb5_unparse_name_ext || || || || ||
|-
| krb5_unparse_name_flags || || || || ||
|-
| krb5_unparse_name_flags_ext || || || || ||
|-
| krb5_us_timeofday || || || || ||
|-
| krb5_verify_authdata_kdc_issued || || || || ||
|-
|}

Projects/Documentation Tasks

2011-01-13T17:12:46Z

Hardjono: /* API documentation */

{{project-early}}

== Purpose ==

To keep track of the various tasks that need to be documented such as function documentation, administration, troubleshooting etc.

=== API documentation ===

{| class="wikitable"
|+ Matrix of Document-Type VS Intended Readership
|-
! Architectural Guide
! Setup and Config of Kerberos
! Admin & Operations of Kerberos
! Custom Build
! API Description
|-
|-
| End-users || || || || || || ||
|-
| Architects || || || || || || ||
|-
|Sys Admins || || || || || || ||
|-
|Application Developers || || || || || || ||
|-
|GSSAPI Developers || || || || || || ||
|-
|Kerberos Developers || || || || || || ||
|}

{| class="wikitable"
|+ Tier 1 - most commonly used API functions (in alphabetical order)
|-
! API
! who writes?
! who reviews?
! reviewed?
! comments
|-
|-
| krb5_build_principal || || || || ||
|-
| krb5_build_principal || || || || ||
|-
|krb5_build_principal_alloc_val || || || || ||
|-
| krb5_build_principal_ext || || || || ||
|-
| krb5_cc_close || || || || ||
|-
| krb5_cc_default || || || || ||
|-
| krb5_cc_default_name || || || || ||
|-
| krb5_cc_destroy || || || || ||
|-
| krb5_cc_dup || || || || ||
|-
| krb5_cc_get_name || || || || ||
|-
| krb5_cc_get_principal || || || || ||
|-
| krb5_cc_get_type || || || || ||
|-
| krb5_cc_initialize || || || || ||
|-
| krb5_cc_new_unique || || || || ||
|-
| krb5_cc_resolve || || || || ||
|-
| krb5_change_password || || || || ||
|-
| krb5_free_context || || || || ||
|-
| krb5_free_error_message || || || || ||
|-
| krb5_free_principal || || || || ||
|-
| krb5_fwd_tgt_cred || || || || ||
|-
| krb5_get_default_realm || || || || ||
|-
| krb5_get_error_message || || || || ||
|-
| krb5_get_host_realm || || || || ||
|-
| krb5_get_credentials || || || || ||
|-
| krb5_get_fallback_host_realm || || || || ||
|-
| krb5_get_init_creds_keytab || || || || ||
|-
| krb5_get_init_creds_opt_alloc || || || || ||
|-
| krb5_get_init_creds_opt_free || || || || ||
|-
| krb5_get_init_creds_opt_get_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_init || || || || ||
|-
| krb5_get_init_creds_opt_set_address_list || || || || ||
|-
| krb5_get_init_creds_opt_set_anonymous || || || || ||
|-
| krb5_get_init_creds_opt_set_canonicalize || || || || ||
|-
| krb5_get_init_creds_opt_set_change_password_prompt || || || || ||
|-
| krb5_get_init_creds_opt_set_etype_list || || || || ||
|-
| krb5_get_init_creds_opt_set_expire_callback || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache_name || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_set_forwardable || || || || ||
|-
| krb5_get_init_creds_opt_set_out_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_pa || || || || ||
|-
| krb5_get_init_creds_opt_set_preauth_list || || || || ||
|-
| krb5_get_init_creds_opt_set_proxiable || || || || ||
|-
| krb5_get_init_creds_opt_set_renew_life || || || || ||
|-
| krb5_get_init_creds_opt_set_salt || || || || ||
|-
| krb5_get_init_creds_opt_set_tkt_life || || || || ||
|-
| krb5_get_init_creds_password || || || || ||
|-
| krb5_get_profile || || || || ||
|-
| krb5_get_prompt_types || || || || ||
|-
| krb5_get_renewed_creds || || || || ||
|-
| krb5_get_validated_creds || || || || ||
|-
| krb5_init_context || || || || ||
|-
| krb5_init_secure_context || || || || ||
|-
| krb5_is_config_principal || || || || ||
|-
| krb5_is_thread_safe || || || || ||
|-
| krb5_kt_close || || || || ||
|-
| krb5_kt_default || || || || ||
|-
| krb5_kt_default_name || || || || ||
|-
| krb5_kt_get_name || || || || ||
|-
| krb5_kt_get_type || || || || ||
|-
| krb5_kt_resolve || || || || ||
|-
| krb5_kuserok || || || || ||
|-
| krb5_parse_name || || || || ||
|-
| krb5_parse_name_flags || || || || ||
|-
| krb5_principal_compare || || || || ||
|-
| krb5_principal_compare_any_realm || || || || ||
|-
| krb5_principal_compare_flags || || || || ||
|-
| krb5_prompter_posix || || || || ||
|-
| krb5_realm_compare || || || || ||
|-
| krb5_recvauth || || || || ||
|-
| krb5_recvauth_version || || || || ||
|-
| krb5_set_default_realm || || || || ||
|-
| krb5_set_password || || || || ||
|-
| krb5_set_password_using_ccache || || || || ||
|-
| krb5_set_principal_realm || || || || ||
|-
| krb5_set_trace_callback || || || || ||
|-
| krb5_set_trace_filename || || || || ||
|-
| krb5_sname_to_principal || || || || ||
|-
| krb5_unparse_name || || || || ||
|-
| krb5_unparse_name_ext || || || || ||
|-
| krb5_unparse_name_flags || || || || ||
|-
| krb5_unparse_name_flags_ext || || || || ||
|-
| krb5_us_timeofday || || || || ||
|-
| krb5_verify_authdata_kdc_issued || || || || ||
|-
|}

Projects/Documentation Tasks

2011-01-13T17:11:59Z

Hardjono: /* API documentation */

{{project-early}}

== Purpose ==

To keep track of the various tasks that need to be documented such as function documentation, administration, troubleshooting etc.

=== API documentation ===

{| class="wikitable"
|+ Matrix of Document-Type VS Intended Readership
|-
! Architectural Guide
! Setup and Config of Kerberos
! Admin & Operations of Kerberos
! Custom Build
! API Description
|-
|-
| End-users || || || || ||
|-
| Architects || || || || ||
|-
|Sys Admins || || || || ||
|-
|Application Developers || || || || ||
|-
|GSSAPI Developers || || || || ||
|-
|Kerberos Developers || || || || ||
|}

{| class="wikitable"
|+ Tier 1 - most commonly used API functions (in alphabetical order)
|-
! API
! who writes?
! who reviews?
! reviewed?
! comments
|-
|-
| krb5_build_principal || || || || ||
|-
| krb5_build_principal || || || || ||
|-
|krb5_build_principal_alloc_val || || || || ||
|-
| krb5_build_principal_ext || || || || ||
|-
| krb5_cc_close || || || || ||
|-
| krb5_cc_default || || || || ||
|-
| krb5_cc_default_name || || || || ||
|-
| krb5_cc_destroy || || || || ||
|-
| krb5_cc_dup || || || || ||
|-
| krb5_cc_get_name || || || || ||
|-
| krb5_cc_get_principal || || || || ||
|-
| krb5_cc_get_type || || || || ||
|-
| krb5_cc_initialize || || || || ||
|-
| krb5_cc_new_unique || || || || ||
|-
| krb5_cc_resolve || || || || ||
|-
| krb5_change_password || || || || ||
|-
| krb5_free_context || || || || ||
|-
| krb5_free_error_message || || || || ||
|-
| krb5_free_principal || || || || ||
|-
| krb5_fwd_tgt_cred || || || || ||
|-
| krb5_get_default_realm || || || || ||
|-
| krb5_get_error_message || || || || ||
|-
| krb5_get_host_realm || || || || ||
|-
| krb5_get_credentials || || || || ||
|-
| krb5_get_fallback_host_realm || || || || ||
|-
| krb5_get_init_creds_keytab || || || || ||
|-
| krb5_get_init_creds_opt_alloc || || || || ||
|-
| krb5_get_init_creds_opt_free || || || || ||
|-
| krb5_get_init_creds_opt_get_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_init || || || || ||
|-
| krb5_get_init_creds_opt_set_address_list || || || || ||
|-
| krb5_get_init_creds_opt_set_anonymous || || || || ||
|-
| krb5_get_init_creds_opt_set_canonicalize || || || || ||
|-
| krb5_get_init_creds_opt_set_change_password_prompt || || || || ||
|-
| krb5_get_init_creds_opt_set_etype_list || || || || ||
|-
| krb5_get_init_creds_opt_set_expire_callback || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache_name || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_set_forwardable || || || || ||
|-
| krb5_get_init_creds_opt_set_out_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_pa || || || || ||
|-
| krb5_get_init_creds_opt_set_preauth_list || || || || ||
|-
| krb5_get_init_creds_opt_set_proxiable || || || || ||
|-
| krb5_get_init_creds_opt_set_renew_life || || || || ||
|-
| krb5_get_init_creds_opt_set_salt || || || || ||
|-
| krb5_get_init_creds_opt_set_tkt_life || || || || ||
|-
| krb5_get_init_creds_password || || || || ||
|-
| krb5_get_profile || || || || ||
|-
| krb5_get_prompt_types || || || || ||
|-
| krb5_get_renewed_creds || || || || ||
|-
| krb5_get_validated_creds || || || || ||
|-
| krb5_init_context || || || || ||
|-
| krb5_init_secure_context || || || || ||
|-
| krb5_is_config_principal || || || || ||
|-
| krb5_is_thread_safe || || || || ||
|-
| krb5_kt_close || || || || ||
|-
| krb5_kt_default || || || || ||
|-
| krb5_kt_default_name || || || || ||
|-
| krb5_kt_get_name || || || || ||
|-
| krb5_kt_get_type || || || || ||
|-
| krb5_kt_resolve || || || || ||
|-
| krb5_kuserok || || || || ||
|-
| krb5_parse_name || || || || ||
|-
| krb5_parse_name_flags || || || || ||
|-
| krb5_principal_compare || || || || ||
|-
| krb5_principal_compare_any_realm || || || || ||
|-
| krb5_principal_compare_flags || || || || ||
|-
| krb5_prompter_posix || || || || ||
|-
| krb5_realm_compare || || || || ||
|-
| krb5_recvauth || || || || ||
|-
| krb5_recvauth_version || || || || ||
|-
| krb5_set_default_realm || || || || ||
|-
| krb5_set_password || || || || ||
|-
| krb5_set_password_using_ccache || || || || ||
|-
| krb5_set_principal_realm || || || || ||
|-
| krb5_set_trace_callback || || || || ||
|-
| krb5_set_trace_filename || || || || ||
|-
| krb5_sname_to_principal || || || || ||
|-
| krb5_unparse_name || || || || ||
|-
| krb5_unparse_name_ext || || || || ||
|-
| krb5_unparse_name_flags || || || || ||
|-
| krb5_unparse_name_flags_ext || || || || ||
|-
| krb5_us_timeofday || || || || ||
|-
| krb5_verify_authdata_kdc_issued || || || || ||
|-
|}

Projects/Documentation Tasks

2011-01-13T17:10:14Z

Hardjono: /* Purpose */

{{project-early}}

== Purpose ==

To keep track of the various tasks that need to be documented such as function documentation, administration, troubleshooting etc.

=== API documentation ===

{| class="wikitable"
|+ Tier 1 - most commonly used API functions (in alphabetical order)
|-
! Architectural Guide
! Setup and Config of Kerberos
! Admin & Operations of Kerberos
! Custom Build
! API Description
|-
|-
| krb5_build_principal || || || || ||
|-
| krb5_build_principal || || || || ||
|-
|krb5_build_principal_alloc_val || || || || ||
|}

{| class="wikitable"
|+ Tier 1 - most commonly used API functions (in alphabetical order)
|-
! API
! who writes?
! who reviews?
! reviewed?
! comments
|-
|-
| krb5_build_principal || || || || ||
|-
| krb5_build_principal || || || || ||
|-
|krb5_build_principal_alloc_val || || || || ||
|-
| krb5_build_principal_ext || || || || ||
|-
| krb5_cc_close || || || || ||
|-
| krb5_cc_default || || || || ||
|-
| krb5_cc_default_name || || || || ||
|-
| krb5_cc_destroy || || || || ||
|-
| krb5_cc_dup || || || || ||
|-
| krb5_cc_get_name || || || || ||
|-
| krb5_cc_get_principal || || || || ||
|-
| krb5_cc_get_type || || || || ||
|-
| krb5_cc_initialize || || || || ||
|-
| krb5_cc_new_unique || || || || ||
|-
| krb5_cc_resolve || || || || ||
|-
| krb5_change_password || || || || ||
|-
| krb5_free_context || || || || ||
|-
| krb5_free_error_message || || || || ||
|-
| krb5_free_principal || || || || ||
|-
| krb5_fwd_tgt_cred || || || || ||
|-
| krb5_get_default_realm || || || || ||
|-
| krb5_get_error_message || || || || ||
|-
| krb5_get_host_realm || || || || ||
|-
| krb5_get_credentials || || || || ||
|-
| krb5_get_fallback_host_realm || || || || ||
|-
| krb5_get_init_creds_keytab || || || || ||
|-
| krb5_get_init_creds_opt_alloc || || || || ||
|-
| krb5_get_init_creds_opt_free || || || || ||
|-
| krb5_get_init_creds_opt_get_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_init || || || || ||
|-
| krb5_get_init_creds_opt_set_address_list || || || || ||
|-
| krb5_get_init_creds_opt_set_anonymous || || || || ||
|-
| krb5_get_init_creds_opt_set_canonicalize || || || || ||
|-
| krb5_get_init_creds_opt_set_change_password_prompt || || || || ||
|-
| krb5_get_init_creds_opt_set_etype_list || || || || ||
|-
| krb5_get_init_creds_opt_set_expire_callback || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_ccache_name || || || || ||
|-
| krb5_get_init_creds_opt_set_fast_flags || || || || ||
|-
| krb5_get_init_creds_opt_set_forwardable || || || || ||
|-
| krb5_get_init_creds_opt_set_out_ccache || || || || ||
|-
| krb5_get_init_creds_opt_set_pa || || || || ||
|-
| krb5_get_init_creds_opt_set_preauth_list || || || || ||
|-
| krb5_get_init_creds_opt_set_proxiable || || || || ||
|-
| krb5_get_init_creds_opt_set_renew_life || || || || ||
|-
| krb5_get_init_creds_opt_set_salt || || || || ||
|-
| krb5_get_init_creds_opt_set_tkt_life || || || || ||
|-
| krb5_get_init_creds_password || || || || ||
|-
| krb5_get_profile || || || || ||
|-
| krb5_get_prompt_types || || || || ||
|-
| krb5_get_renewed_creds || || || || ||
|-
| krb5_get_validated_creds || || || || ||
|-
| krb5_init_context || || || || ||
|-
| krb5_init_secure_context || || || || ||
|-
| krb5_is_config_principal || || || || ||
|-
| krb5_is_thread_safe || || || || ||
|-
| krb5_kt_close || || || || ||
|-
| krb5_kt_default || || || || ||
|-
| krb5_kt_default_name || || || || ||
|-
| krb5_kt_get_name || || || || ||
|-
| krb5_kt_get_type || || || || ||
|-
| krb5_kt_resolve || || || || ||
|-
| krb5_kuserok || || || || ||
|-
| krb5_parse_name || || || || ||
|-
| krb5_parse_name_flags || || || || ||
|-
| krb5_principal_compare || || || || ||
|-
| krb5_principal_compare_any_realm || || || || ||
|-
| krb5_principal_compare_flags || || || || ||
|-
| krb5_prompter_posix || || || || ||
|-
| krb5_realm_compare || || || || ||
|-
| krb5_recvauth || || || || ||
|-
| krb5_recvauth_version || || || || ||
|-
| krb5_set_default_realm || || || || ||
|-
| krb5_set_password || || || || ||
|-
| krb5_set_password_using_ccache || || || || ||
|-
| krb5_set_principal_realm || || || || ||
|-
| krb5_set_trace_callback || || || || ||
|-
| krb5_set_trace_filename || || || || ||
|-
| krb5_sname_to_principal || || || || ||
|-
| krb5_unparse_name || || || || ||
|-
| krb5_unparse_name_ext || || || || ||
|-
| krb5_unparse_name_flags || || || || ||
|-
| krb5_unparse_name_flags_ext || || || || ||
|-
| krb5_us_timeofday || || || || ||
|-
| krb5_verify_authdata_kdc_issued || || || || ||
|-
|}

Projects/Plugin support improvements

2010-08-03T19:13:20Z

Hardjono: /* Background */

{{project-early}}

== Background ==

* Project inception and first proposal: Zhanna Tsitkova (see [[Projects/Plugin_support_improvements_%28first_version%29]]).
* Further revisions (this page): Greg Hudson, Tom Yu and Thomas Hardjono.

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The architecture is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin support:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin support:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin support which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support architecture uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-08-03T19:12:47Z

Hardjono: /* Background */

{{project-early}}

== Background ==

* Project inception and first proposal: Zhanna Tsitkova (see [[Projects/Plugin_support_improvements_%28first_version%29]]).
* Further revisions: Greg Hudson, Tom Yu and Thomas Hardjono.

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The architecture is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin support:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin support:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin support which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support architecture uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-08-03T19:12:39Z

Hardjono: /* Background */

{{project-early}}

== Background ==

* Project inception and first proposal: Zhanna Tsitkova (see [[Projects/Plugin_support_improvements_%28first_version%29]]).
* Further revision: Greg Hudson, Tom Yu and Thomas Hardjono.

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The architecture is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin support:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin support:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin support which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support architecture uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-08-03T19:12:12Z

Hardjono:

{{project-early}}

== Background ==

* Project inception: Zhanna Tsitkova (see [[Projects/Plugin_support_improvements_%28first_version%29]]).
* Further revision: Greg Hudson, Tom Yu and Thomas Hardjono.

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The architecture is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin support:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin support:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin support which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support architecture uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-08-03T19:05:45Z

Hardjono:

{{project-early}}

== Background ==

* Project inception: Zhanna Tsitkova [[Projects/Plugin_support_improvements_%28first_version%29]]
* Further revision: Greg Hudson and Tom Yu.

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The architecture is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin support:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin support:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin support which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support architecture uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements (first version)

2010-08-03T18:54:45Z

Hardjono: Projects/Plugin support improvements moved to Projects/Plugin support improvements (first version)

{{project-early}}

==Motivation ==

We set up our goal on creating a framework that

* Separates plugin interface from its implementation;
* Provides simple and clear mechanism that facilitates additions of new plugin interfaces and their implementations(modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same plugin interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation to use services provided by the other plugin implementations.

==Structure==
[[Image:Slide1.jpg]]

==Participants==

* Plugin Manager ('''PM''') - Module that implements operations that manage plugin configuration and registry services. PM is responsible to maintain a container populated with plugin handles (references to plugin objects) and provide the search capabilities for callers requesting the plugin services.
* Plugin Loader ('''PL''') - Abstract module that declares an interface for operations that facilitate manipulation with the set of plugin implementations;
* Plugin Loader Implementation ('''PLI''') - Concrete implementation of PL. It knows about the set of the available implementations and how to create them;
* Plugin Interface ('''PI''') - Abstract module that declares plugin interface.
* Plugin Interface Implementation ('''PII''') - Concrete implementation of the plugin interface;
* Caller - Plugin caller.

==Collaboration==

* An instance ''pl_manager'' of PM is created as part of ''krb5_init_context'' operation at run-time (1). This instance will use PM methods to configure and register the desired plugin implementations;
* The list of the desired plugin implementations is known to the specific PLI which is aggregated within PM (2);
* PM delegates acquiring of plugin interfaces to PLI (3);
* The caller uses ''pl_manager'' to invoke the desired plugin interface (4).

==User Interface==

; plugin_manager_get_service: - perform a search in the plugin manager registry to locate the particular interface based on plugin interface name and plugin id. If the search is successful, the plhandle structure gets populated with reference to the plugin interface and the function return code is set to success. Otherwise, this function returns an error;
; plugin_manager_configure: - parse a configuration file. Based on the obtained information construct plugin objects and load them into the plugin registry;
; plugin_manager_start: - complete necessary initializations of plugin objects and make them available for use;
; plugin_manager_stop: - free resources;
; plugin_manager_get_instance: - instantiate an instance of plugin manager if it does not exist.
; plugin_loader_create_api: - return the vtable with the implementation pertained to the specific interface;

==Implementation==

; MT safety : Implementation must be MT safe.

; Proof of concept code is available from ''plugins'' branch: ''svn+ssh://svn.mit.edu/krb5/branches/plugins''

; k5-int.h: ''plugin_manager *pl_manager'' is added to _krb5_context;

; PM: The purpose of PM is to provide the configuration and registry services. In future the ability to have multiple implementations of the PM may be desired to allow one to have flexible configuration environment (For example, the configuration service can be implemented using various file formats or be completely hardcoded.)
; PM vs PLI: In general PM is tied to one or more PLIs. In its turn, each PLI is responsible for loading one or more plugins. The notion of the multiple plugins per PLI is justified by the convenience of the grouping based on the plugin functionality, level of optimization or other features;

==Consequences==

The framework is designed to hide the management details, such as configuration and loading, from the plugin writers and callers.

* The responsibility of the writer of the '''new plugin interface (PI)''' is limited to the writing source and header as described at Template_1:

''Template 1''

''HEADER''
#include <plugin_manager.h>
#include <k5-int.h>
typedef struct {
int version;
char plugin_name[MAX_PL_NAME_LEN];
krb5_error_code (*fn1)(...);
} plugin_PLName;
krb5_error_code plugin_PLName_fn1(plhandle handle, ...);

''SOURCE''
krb5_error_code plugin_PLName_fn1(plhandle handle, ...)
{
plugin_PLName* api = (plugin_PLName*) handle.api;
api->plugin_PLName_fn1(...);
return 0;
}

* The responsibility of the writer of the '''new plugin interface implementation (PII)''' is limited to the writing source and header as described at Template_2 and the notifying the plugin loader about its availability Template_3:

''Template 2''

''HEADER''
plhandle plugin_pwd_qlty_krb_create(void);

''SOURCE''
static krb5_error_code
plugin_PLName_fn1(...)
{ /*intended functionality */}

static void
plugin_PLName_cleanup(...)
{ /* cleanup */ }

static krb5_error_code
plugin_PLName_init(...)
{ /* init */ }

plhandle plugin_PLName_PLImpl_create()
{
plhandle handle;;
plugin_PLName* api = malloc(sizeof(plugin_PLName));
api->fn1 = plugin_PLName_fn1;
api->PLName_init = plugin_PLName_init;
api->PLName_cleanup = plugin_PLName_cleanup;
handle.api = api;
return handle;
}

''Template 3''

static plugin_descr _table[] = {
{"plugin_PLImpl_PLName", plugin_PLName_PLImpl_create},
};

''For better readability some of the verification, clean-up, error handling code is omitted from the templates''

==Built-in Plugins==

*; Examples: PreAuth, AuthData, Password Quality (see ''Current plugins'' section for details)

*; Invoking multiple implementations of the same plugin: This is achieved by using the aimed ''plugin_id'''s in ''plugin_manager_get_service'' calls. For example, two specific implementations of the password quality plugins may be invoked in password check.

*; Invoking ''all available'' plugin implementations: See also "Defaults" section

==Dynamic Plugins==

*; Examples: Audit, Password sync

*; Requirements:

# The difference between dynamic and built-in plugins (both interface and implementation) should be on the level of linkage.
# Plugin consumer should not have the knowledge about the type of plugins it is using (in terms of dynamic/built-in). On the caller side the code invoking plugin should stay the same for both dynamic and built-in plugins.
# Similar to built-in plugins, the plugin consumer may call the multiple implementations of the same plugin interface. For example, two implementations of the audit plugin interface may be used to handle password change and intruder lockout events.
# Similar to built-in plugins, one can choose to use ''all available'' plugin implementations. Their location may be some default one, or passed via env variable, or defined in the configuration.
# In the case one wants to have better control over the configuration and management of the plugins selection, it responsibility of the administrator to point to the location of the dynamic plugin as part of the configuration.

*; Creating dynamic plugin: The appropriate loader is linked with the set of the desired plugins resulting into the shared library. Details:
# One of the deliverables of the plugin framework is the loader file. Its responsibility is to load modules that are listed in its local table (see below). ;
# The dynamic plugin creator must fill this table with the list of the desired pluginmodules.
# The dynamic plugin creator then links the loader with the set of the desired plugin modules resulting into the shared library;
# Administrator points to the location of the dynamic plugin by setting ''plugin_loader_path'' configuration attribute to this library.

NOTE: This approach allows to bundle dynamic plugins.
It also hides loading details from the plugin writer making code for dynamic and built-in plugin indistinguishable.

''The following is the example of how the table in the loader may look:''
static plugin_descr plugin_dyn_factory_table[] = {
{"plugin_pwd_qlty_DYN", plugin_pwd_qlty_DYN_create},
{"plugin_pwd_qlty_XYZ", plugin_pwd_qlty_XYZ_create},
{NULL,NULL}
};

*; Invoking dynamic plugin: Similar to builtin plugins with the following additions. The option ''plugin_loader_path'' in the configuration file refers to the location of the dynamic plugin shared library and the option ''plugin_loader_type'' must be set to ''dynamic''.

==Configuration==
If no plugin configuration is provided the new plugin framework should act upon some default behavior.

*; '''Configuration file attributes'''
:: ''plugin list'' - list of the desired plugins;
:: ''plugin_api'' - name of the plugin interface;
:: ''plugin_name'' - concrete plugin implementation;
:: ''plugin_version'' - desired version of plugin implementation; This may be used by the administrator to point to the most recent plugin implementation (perhaps, bug fix).
:: ''plugin_loader_name'' - name of the loader the particular implementation is associated with;
:: ''plugin_loader_type'' - defines the plugin type: built-in or dynamic;
:: ''plugin_loader_path'' - path to the plugin dynamic library
:: ''plugin_disable'' - don't use this plugin implementation
:: ''plugin_properties'' - plugin implementation initialization parameters (when needed)

*; '''Example of the plugin section in krb5.conf'''
[plugins]
/* By default, all built-in plugin impls are enabled. To disable a particular built-in plugin one need to specify it */
plugin_list = PQ1
PQ1 = {
plugin_api = plugin_pwd_qlty
plugin_name = plugin_pwd_qlty_krb
plugin_disable = yes
}

/* dynamic plugin */
plugin_list = PQ_DYN
PQ_DYN = {
plugin_api = plugin_pwd_qlty
plugin_name = plugin_pwd_qlty_DYN
plugin_loader_type = dynamic
plugin_loader_path = /path-to-the-lib/libplugin_dynamic.so # if default, may be omitted
}

*; '''Alternatives'''
The choice of plugin configuration format is a matter of preference. It may be presented in XML, YAML, JSON etc. One needs only an appropriate parser.

*; '''Configuration change at run-time'''
TBD

==Directory Structure==

*; src/plugin_core: plugin_manager.[ch], plugin_loader.[ch], impl/plugin_XXX_manager.[ch], impl/plugin_XXX_loader.[ch]

*; src/plugins: pluginA/plugin_A.[ch], pluginA/plugin_A_implN/plugin_A_implN.[ch]

*; src/plugin_dynamic: plugin_dyn_factory.[ch]

==Defaults==
If no plugin configuration is provided new plugin framework should act upon some default behavior.

There are two default loaders.
The first loader aggregates built-in plugin implementations, while the second one - dynamically loadable ones located in some default directory. If the administrator wishes to have all of these plugin implementations available and they need no additional configuration (for example, for their initialization), the configuration file may not have any plugin related information.

==Build process==

*; Options: DEBUG_PLUGINS - output more info for debugging

==Sample Code==

*; Plugin initialization
The following is an example of the plugin manager initialization:

plugin_manager_get_instance(&ctx->pl_manager);
plugin_manager_configure(ctx->pl_manager, conf_path);
plugin_manager_start(ctx->pl_manager);

*; How to call plugins
The following is an example of invoking Password Quality plugin:

plugin_manager_get_service(ctx->pl_manager, PWD_QLTY, PWD_QLTY_KRB, &plugin_handle);
plugin_pwd_qlty_check(plugin_handle, srv_handle, password, use_policy, pol, principal);

*;How to construct plugin interface (PI)
The following is an example of how plugin interface called "password quality" may look:

#ifndef PLUGIN_PWD_QLTY_H_
#define PLUGIN_PWD_QLTY_H_

typedef struct {
int version;
char plugin_id[MAX_PL_NAME_LEN];
kadm5_ret_t (*pwd_qlty_init)(kadm5_server_handle_t);
void (*pwd_qlty_cleanup)();
kadm5_ret_t (*pwd_qlty_check)(kadm5_server_handle_t, char*, int, kadm5_policy_ent_t, krb5_principal);
} plugin_pwd_qlty;

kadm5_ret_t plugin_pwd_qlty_init(plhandle, kadm5_server_handle_t);
kadm5_ret_t plugin_pwd_qlty_cleanup(plhandle);
kadm5_ret_t plugin_pwd_qlty_check(plhandle, kadm5_server_handle_t, char*, int, kadm5_policy_ent_t, krb5_principal);

#endif /* PLUGIN_PWD_QLTY_H_ */

''SOURCE''
kadm5_ret_t
plugin_pwd_qlty_check( plhandle handle, kadm5_server_handle_t srv_handle, char *password, int use_policy, kadm5_policy_ent_t pol, krb5_principal principal)
{
plugin_pwd_qlty* api = (plugin_pwd_qlty*) handle.api;
api->pwd_qlty_check(srv_handle, password, use_policy, pol, principal);
}

kadm5_ret_t
plugin_pwd_qlty_init(plhandle, kadm5_server_handle_t)
{ /* init */ }

void
plugin_pwd_qlty_cleanup(plhandle)
{ /* clean-up */ }

*; How to construct plugin interface implementation (PII)
Every PII should have only one public ''create'' function. For PII called "password quality krb" it would be ''plhandle plugin_pwd_qlty_krb_create()''.The following is an example of the this PII:

#ifndef PLUGIN_PWD_QLTY_KRB_IMPL_H_
#define PLUGIN_PWD_QLTY_KRB_IMPL_H_

plhandle plugin_pwd_qlty_krb_create(void);

#endif /* PLUGIN_PWD_QLTY_KRB_IMPL_H_ */

''SOURCE''
plhandle plugin_pwd_qlty_krb_create()
{
plhandle handle;
...
api->pwd_qlty_init = plugin_pwd_qlty_init;
api->pwd_qlty_check = plugin_pwd_qlty_check;
api->pwd_qlty_cleanup = plugin_pwd_qlty_clean;
handle.api = api;
return handle;
}

static kadm5_ret_t
plugin_pwd_qlty_check(kadm5_server_handle_t srv_handle, char *password, int use_policy, kadm5_policy_ent_t pol, krb5_principal principal)
{ /* do some quality verification */ }

static kadm5_ret_t
plugin_pwd_qlty_init(kadm5_server_handle_t handle)
{ /* initialization */ }

static kadm5_ret_t
plugin_pwd_qlty_cleanup()
{ /* cleanup */ }

''For better readability some of the verification, clean-up, error handling code is omitted from the samples''

==Current plugins==

We currently have the following plugin:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

==Future plugins==

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

==Current support infrastructure==

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

==Problem areas==

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (Dependent library shared objects of the calling library can contain "built-in" modules for the calling library.) The distinguishing feature of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

Projects/Plugin Support

2010-08-03T18:49:39Z

Hardjono:

* Original Proposal: [[projects/Plugin_support_improvements]].
* Revised Proposal [[http://k5wiki.kerberos.org/wiki/User:Hardjono/Plugin_Support]]

Projects/Plugin Support

2010-08-03T18:49:12Z

Hardjono:

* Original Proposal: [[Plugin_support_improvements]].
* Revised Proposal [[http://k5wiki.kerberos.org/wiki/User:Hardjono/Plugin_Support]]

Projects/Plugin Support

2010-08-03T18:40:13Z

Hardjono:

* Original Proposal: [[http://k5wiki.kerberos.org/wiki/Projects/Plugin_support_improvements]].
* Revised Proposal [[http://k5wiki.kerberos.org/wiki/User:Hardjono/Plugin_Support]]

Projects/Plugin Support

2010-08-03T18:38:13Z

Hardjono:

* Original Proposal: [[http://k5wiki.kerberos.org/wiki/Projects/Plugin_support_improvements]].
* Revised Proposal.

Projects/Plugin Support

2010-08-03T18:35:48Z

Hardjono:

* [[Original Proposal]].
* Revised Proposal.

Projects/Plugin Support

2010-08-03T18:34:28Z

Hardjono:

* Original Proposal
* Revised Proposal.

Projects/Plugin Support

2010-08-03T18:34:05Z

Hardjono:

* [Original Proposal] http://k5wiki.kerberos.org/wiki/Projects/Plugin_support_improvements.
* Revised Proposal.

Projects/Plugin Support

2010-08-03T18:33:53Z

Hardjono:

* [Original Proposal]http://k5wiki.kerberos.org/wiki/Projects/Plugin_support_improvements.
* Revised Proposal.

Projects/Plugin Support

2010-08-03T18:32:23Z

Hardjono: New page: * Original Proposal. * Revised Proposal.

* Original Proposal.
* Revised Proposal.

Projects/Plugin support improvements

2010-08-03T18:28:00Z

Hardjono: /* Background */

== Background ==

* Project inception: Zhanna Tsitkova.
* Further revision: Greg Hudson and Tom Yu.

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The architecture is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin support:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin support:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin support which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support architecture uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-08-03T18:26:50Z

Hardjono: /* Motivations, Priorities & Requirements */

== Background ==

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The architecture is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin support:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin support:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin support which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support architecture uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:55:04Z

Hardjono: /* Definitions */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The architecture is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin support:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin support:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin support which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support architecture uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:54:38Z

Hardjono: /* Current plugins */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The architecture is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin support:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin support:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin support which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:53:28Z

Hardjono: /* Deliverables */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The architecture is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:52:55Z

Hardjono: /* Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin manager to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:52:19Z

Hardjono: /* Provider interface */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin manager uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:51:57Z

Hardjono: /* Consumer interface */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin manager. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:51:01Z

Hardjono: /* Motivations, Priorities & Requirements */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture:

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the architecture to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new architecture.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:49:58Z

Hardjono: /* Function signatures */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
:k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
: k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
: k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
: k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:49:23Z

Hardjono: /* Configuration */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

<tt>
: [plugins]
: interfacename = {
:: # May take multiple values; only named plugins will be enabled.
:: enable_only = name
:
:: # May take multiple values; named plugins will be disabled.
:: disable = name
:
:: # Establishes a mapping from a module name to a dynamic object.
:: module = modname:pathname
: }
</tt>

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:48:19Z

Hardjono: /* Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

[plugins]
interfacename = {
# May take multiple values; only named plugins will be enabled.
enable_only = name

# May take multiple values; named plugins will be disabled.
disable = name

# Establishes a mapping from a module name to a dynamic object.
module = modname:pathname
}

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:47:13Z

Hardjono: /* Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

[plugins]
interfacename = {
# May take multiple values; only named plugins will be enabled.
enable_only = name

# May take multiple values; named plugins will be disabled.
disable = name

# Establishes a mapping from a module name to a dynamic object.
module = modname:pathname
}

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
:: goto cleanup;

: /* Allocate a large enough list of handles. */
: for (count = 0; modules[count] != NULL; count++);
: list = k5alloc((count + 1) * sizeof(*list), &ret);
: if (list == NULL)
:: goto cleanup;

: /* For each module, allocate a handle and initialize its vtable. Skip
: * modules which don't successfully initialize. */
: count = 0;
: for (mod = modules; *mod != NULL; mod++) {
:: handle = k5alloc(sizeof(*handle), &ret);
:: if (handle == NULL)
::: goto cleanup;
:: ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
:: if (ret == 0)
::: list[count++] = handle;
:: else
::: free(handle);
: }

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:45:07Z

Hardjono: /* Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

[plugins]
interfacename = {
# May take multiple values; only named plugins will be enabled.
enable_only = name

# May take multiple values; named plugins will be disabled.
disable = name

# Establishes a mapping from a module name to a dynamic object.
module = modname:pathname
}

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:
<tt>
: ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
: if (ret != 0)
goto cleanup;

/* Allocate a large enough list of handles. */
for (count = 0; modules[count] != NULL; count++);
list = k5alloc((count + 1) * sizeof(*list), &ret);
if (list == NULL)
goto cleanup;

/* For each module, allocate a handle and initialize its vtable. Skip
* modules which don't successfully initialize. */
count = 0;
for (mod = modules; *mod != NULL; mod++) {
handle = k5alloc(sizeof(*handle), &ret);
if (handle == NULL)
goto cleanup;
ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
if (ret == 0)
list[count++] = handle;
else
free(handle);
}

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:44:25Z

Hardjono: /* Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

[plugins]
interfacename = {
# May take multiple values; only named plugins will be enabled.
enable_only = name

# May take multiple values; named plugins will be disabled.
disable = name

# Establishes a mapping from a module name to a dynamic object.
module = modname:pathname
}

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
:: }
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:

ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
if (ret != 0)
goto cleanup;

/* Allocate a large enough list of handles. */
for (count = 0; modules[count] != NULL; count++);
list = k5alloc((count + 1) * sizeof(*list), &ret);
if (list == NULL)
goto cleanup;

/* For each module, allocate a handle and initialize its vtable. Skip
* modules which don't successfully initialize. */
count = 0;
for (mod = modules; *mod != NULL; mod++) {
handle = k5alloc(sizeof(*handle), &ret);
if (handle == NULL)
goto cleanup;
ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
if (ret == 0)
list[count++] = handle;
else
free(handle);
}

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:44:07Z

Hardjono: /* Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

[plugins]
interfacename = {
# May take multiple values; only named plugins will be enabled.
enable_only = name

# May take multiple values; named plugins will be disabled.
disable = name

# Establishes a mapping from a module name to a dynamic object.
module = modname:pathname
}

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
: for (h = handle->qual_handles; *h != NULL; h++) {
:: ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
:: if (ret != 0)
::: return ret;
}
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:

ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
if (ret != 0)
goto cleanup;

/* Allocate a large enough list of handles. */
for (count = 0; modules[count] != NULL; count++);
list = k5alloc((count + 1) * sizeof(*list), &ret);
if (list == NULL)
goto cleanup;

/* For each module, allocate a handle and initialize its vtable. Skip
* modules which don't successfully initialize. */
count = 0;
for (mod = modules; *mod != NULL; mod++) {
handle = k5alloc(sizeof(*handle), &ret);
if (handle == NULL)
goto cleanup;
ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
if (ret == 0)
list[count++] = handle;
else
free(handle);
}

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:43:20Z

Hardjono: /* Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

[plugins]
interfacename = {
# May take multiple values; only named plugins will be enabled.
enable_only = name

# May take multiple values; named plugins will be disabled.
disable = name

# Establishes a mapping from a module name to a dynamic object.
module = modname:pathname
}

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

<tt>
: ret = k5_pwqual_load(handle->context, &list);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:
<tt>
for (h = handle->qual_handles; *h != NULL; h++) {
ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
if (ret != 0)
return ret;
}
</tt>

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:

ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
if (ret != 0)
goto cleanup;

/* Allocate a large enough list of handles. */
for (count = 0; modules[count] != NULL; count++);
list = k5alloc((count + 1) * sizeof(*list), &ret);
if (list == NULL)
goto cleanup;

/* For each module, allocate a handle and initialize its vtable. Skip
* modules which don't successfully initialize. */
count = 0;
for (mod = modules; *mod != NULL; mod++) {
handle = k5alloc(sizeof(*handle), &ret);
if (handle == NULL)
goto cleanup;
ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
if (ret == 0)
list[count++] = handle;
else
free(handle);
}

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:42:10Z

Hardjono: /* Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

[plugins]
interfacename = {
# May take multiple values; only named plugins will be enabled.
enable_only = name

# May take multiple values; named plugins will be disabled.
disable = name

# Establishes a mapping from a module name to a dynamic object.
module = modname:pathname
}

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

<tt>
: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "dict", pwqual_dict_init);
:: if (ret != 0)
::: return ret;

: ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL, "policy", pwqual_policy_init);
:: if (ret != 0)
::: return ret;
</tt>

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

ret = k5_pwqual_load(handle->context, &list);
if (ret != 0)
return ret;

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:

for (h = handle->qual_handles; *h != NULL; h++) {
ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
if (ret != 0)
return ret;
}

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:

ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
if (ret != 0)
goto cleanup;

/* Allocate a large enough list of handles. */
for (count = 0; modules[count] != NULL; count++);
list = k5alloc((count + 1) * sizeof(*list), &ret);
if (list == NULL)
goto cleanup;

/* For each module, allocate a handle and initialize its vtable. Skip
* modules which don't successfully initialize. */
count = 0;
for (mod = modules; *mod != NULL; mod++) {
handle = k5alloc(sizeof(*handle), &ret);
if (handle == NULL)
goto cleanup;
ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
if (ret == 0)
list[count++] = handle;
else
free(handle);
}

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:39:59Z

Hardjono: /* Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

[plugins]
interfacename = {
# May take multiple values; only named plugins will be enabled.
enable_only = name

# May take multiple values; named plugins will be disabled.
disable = name

# Establishes a mapping from a module name to a dynamic object.
module = modname:pathname
}

=== Code and Proof of Concept ===

The Subversion URL for the proof of concept is:

<tt>svn://anonsvn.mit.edu/krb5/branches/plugins2</tt>

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL,
"dict", pwqual_dict_init);
if (ret != 0)
return ret;

ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL,
"policy", pwqual_policy_init);
if (ret != 0)
return ret;

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

ret = k5_pwqual_load(handle->context, &list);
if (ret != 0)
return ret;

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:

for (h = handle->qual_handles; *h != NULL; h++) {
ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
if (ret != 0)
return ret;
}

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:

ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
if (ret != 0)
goto cleanup;

/* Allocate a large enough list of handles. */
for (count = 0; modules[count] != NULL; count++);
list = k5alloc((count + 1) * sizeof(*list), &ret);
if (list == NULL)
goto cleanup;

/* For each module, allocate a handle and initialize its vtable. Skip
* modules which don't successfully initialize. */
count = 0;
for (mod = modules; *mod != NULL; mod++) {
handle = k5alloc(sizeof(*handle), &ret);
if (handle == NULL)
goto cleanup;
ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
if (ret == 0)
list[count++] = handle;
else
free(handle);
}

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:39:05Z

Hardjono: /* Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

[plugins]
interfacename = {
# May take multiple values; only named plugins will be enabled.
enable_only = name

# May take multiple values; named plugins will be disabled.
disable = name

# Establishes a mapping from a module name to a dynamic object.
module = modname:pathname
}

=== Code and Proof of Concept ===

<tt>
The Subversion URL for the proof of concept is:

svn://anonsvn.mit.edu/krb5/branches/plugins2

There is a README.BRANCH file as the top level containing a walkthrough of the changes on the branch.

This is a consumer registering built-in plugin modules for the password quality interface:

ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL,
"dict", pwqual_dict_init);
if (ret != 0)
return ret;

ret = k5_plugin_register(context, PLUGIN_INTERFACE_PWQUAL,
"policy", pwqual_policy_init);
if (ret != 0)
return ret;

This is a consumer using the pwqual consumer API to create a list of handles for all available password quality modules:

ret = k5_pwqual_load(handle->context, &list);
if (ret != 0)
return ret;

This is a consumer using the pwqual consumer API to check a password against all available password quality modules:

for (h = handle->qual_handles; *h != NULL; h++) {
ret = k5_pwqual_check(handle->context, *h, password, policy, princ);
if (ret != 0)
return ret;
}

This is the password quality loader function invoking the plugin framework to get a list of all available password quality vtable constructors, and then invoking the vtable constructors to create plugin
handles:

ret = k5_plugin_load_all(context, PLUGIN_INTERFACE_PWQUAL, &modules);
if (ret != 0)
goto cleanup;

/* Allocate a large enough list of handles. */
for (count = 0; modules[count] != NULL; count++);
list = k5alloc((count + 1) * sizeof(*list), &ret);
if (list == NULL)
goto cleanup;

/* For each module, allocate a handle and initialize its vtable. Skip
* modules which don't successfully initialize. */
count = 0;
for (mod = modules; *mod != NULL; mod++) {
handle = k5alloc(sizeof(*handle), &ret);
if (handle == NULL)
goto cleanup;
ret = (*mod)(context, 1, 1, (krb5_plugin_vtable)&handle->vt);
if (ret == 0)
list[count++] = handle;
else
free(handle);
}

</tt>

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:37:28Z

Hardjono: /* Configuration */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

Here is a description of the configuration used by the proof of concept:

[plugins]
interfacename = {
# May take multiple values; only named plugins will be enabled.
enable_only = name

# May take multiple values; named plugins will be disabled.
disable = name

# Establishes a mapping from a module name to a dynamic object.
module = modname:pathname
}

=== Code and Proof of Concept ===

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:36:07Z

Hardjono: /* Sample Code and Proof of Concept */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

=== Configuration ===

=== Code and Proof of Concept ===

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:35:32Z

Hardjono: /* Configuration */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Sample Code and Proof of Concept ==

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:33:29Z

Hardjono: /* Function signatures */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id, krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname, krb5_plugin_init_fn module); </code>

== Configuration ==

== Sample Code and Proof of Concept ==

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements

Projects/Plugin support improvements

2010-07-28T18:32:52Z

Hardjono: /* Function signatures */

==Motivations, Priorities & Requirements ==

'''Motivations''': there are a number of motivations behind the creation of the plugin architecture framework.

* Desire to separate pluggable interface from its implementation;
* Desire to provide simple and clear mechanism that facilitates additions of new pluggable interfaces and their implementations (modules);
* Handles both built-in and dynamic plugin modules;
* Allows multiple implementation of the same pluggable interface;
* Provides uniform way to supply parameters for plugin configuration;
* Allows one plugin implementation (module) to use services provided by the other plugin implementations.

'''Requirements''': from these items we have developed a more formal set of requirements
covering the design and the implementation of the framework to
support the plugins. These are as follows:

# Allow third parties to implement multiple plugin modules for each pluggable interface.
# Allow a plugin module to build as dynamic or built-in from the same source code.
# Allow third parties to more easily create new plugin modules.
# Provide a uniform method for configuring discovery of plugin modules.
# Improve readability of code that calls pluggable interfaces.
# Allow easier creation of new pluggable interfaces.
# Allow incremental transition of existing pluggable interfaces to the new framework.

== Architecture Overview and Concepts ==

=== Introduction ===

The architecture for the plugin support is shown in the following figure.
The participants and components are described in the section below.

[[Image:plugin_architecture_v3_png.png]]

=== Participants ===

The following is a summary of participants and components
within the architecture. Further details are provided in the sections below.

'''Plugin Manager''': The plugin manager provides a set of generic capabilities that are independent of individual plugin interfaces. The plugin manager implements operations that manage plugin configuration and plugin registry services.

'''Pluggable Interface''': A pluggable interface is an interface that can be implemented by a third party in a modular manner. An implementation of a pluggable interface is referred to as a ''plugin module''. Furthermore, a pluggable interface itself consist of a ''consumer interface'' and ''provider interface'' (see below).

'''Plugin Module''': A plugin module is an implementation of a pluggable interface. For example, in the Figure Plugin_A is shown to have two implementations (modules).

'''Consumer''': The consumer or caller is the entity that uses the plugin module.

=== Collaboration: Flows ===

As shown in the above Figure,
the plugin architecture is designed based on the notion
of pluggable interfaces, each of which are defined based on an abstract design.

When a third party wishes to develop a loadable plugin module
(e.g. Plugin_Module_A1)
that implements a specific task (e.g. implement password
quality check), the developer of the module must
conform to the pluggable interface (Pluggable Interface A) defined for that
"family" of plugin modules.

The consumer (or caller) that later makes use of the plugin module,
must invoke functions implemented in that module
through a specific consumer interface (Consumer_Interface_A).
Discovery (and filtering) is triggered by the first load operation (within a krb5_context).

== Architecture Components ==

In this section we provide further details on the components
of the architecture, describing its features and behaviors.

=== Plugin Manager ===

The plugin manager provides a set of generic support capabilities that are independent of individual pluggable interfaces. It centralizes the discovery process for plugin modules. Typically, consumers of pluggable interfaces do not call it directly. Instead a consumer calls a loader function (of the specific pluggable interface) which in-turn calls the plugin manager.

In this architecture, the <code>krb5_init_context()</code> functions will create and configure a plugin manager context that will exist in the krb5_context.

The plugin manager locates plugin modules using both a ''numeric identifier'' (that designates a plugin interface) and a ''string'' (that names a module which implements that pluggable interface). The primary way to use the plugin manager is to query it for the vtable constructor for a specified module (or a set of vtable constructors for all modules of that interface).

The plugin manager keeps track of modules through its registries. These are discussed as follows.

==== Registry of built-in modules ====

This registry keeps track of built-in modules. Typically, libkrb5 will initialize this with locators for all of the built-in modules that are linked into it. Other code units can also register private built-in plugin modules using this registry.

==== Registry of loadable modules ====

This registry keeps track of a few additional items needed for loadable modules:

* Each interface's registry starts out empty.

* The consumer (typically) populates the registry by registering vtable constructors for built-in modules.

* When k5_plugin_load() is invoked on an interface for the first time, discovery is performed. This has two steps:

** Dynamic module mappings are read from the profile. Each named dynamic module is dlopened and dlsym'd to obtain the vtable constructor, and that constructor is added to the interface registry.

** Enable/disable information is read from the profile. The interface registry is pruned to contain only enabled modules.

* Thereafter, the interface's registry is unchanging.

=== Pluggable Interfaces ===

A pluggable interface is an interface (possibly internal to a library) that can be implemented by a third party in a modular, well-compartmentalized manner. These implementations of pluggable interfaces are called plugin modules. Pluggable interfaces allow a consumer to use the capabilities of the interface without needing to be aware of the implementation details. In particular, a pluggable interface prevents the consumer from needing to know whether the module is a built-in or a dynamically loadable module.

Pluggable interfaces can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

A pluggable interface has two parts: a ''consumer interface'' and a ''provider interface''. Typically, library code implements the consumer interface, and application code or other library code calls the functions of the consumer interface.

==== Consumer interface ====

The consumer interface isolates the consumer from implementation details of the pluggable interface. The consumer does not generally need to know about whether a given module is built-in or dynamically loaded. The implementation of a consumer interface is essentially a glue layer, and can make use of domain-independent (not specific to any pluggable interface) capabilities of the plugin framework. The consumer might explicitly register a new plugin module that it implements: this capability is part of the plugin manager.

A consumer of a pluggable interface uses an opaque handle (obtained from a loader function that is part of the pluggable interface) to call the methods of a plugin module. Each handle represents one plugin module, and perhaps associated resource information. For one-to-many pluggable interfaces, the loader function will return a list of handles.

Each method of the consumer interface is an ordinary C function that takes the opaque handle either explicitly as its first argument or implicitly by some means such as a module name. In essence, these pluggable interface functions in the architecture are wrapper functions that call through function pointers contained in the opaque plugin module handle object.

One rationale for using wrapper functions instead of having the consumer directly invoke methods through a function pointer is to make it easier for debuggers and analysis tools to recognize when a particular interface method is being called. (Function pointers might have identifier names that look nothing like the actual name of the function they point to, in addition to enabling confusing aliasing.)

The loader function is specific to the pluggable interface. One reason is for type safety: there will be a distinct opaque handle type for each pluggable interface, allowing compile-time checking to catch some sorts of programming errors. Another reason is backward compatibility: it allows a pluggable interface to support plugin modules that implement an older provider interface.

==== Provider interface ====

A plugin module is a unit of code that implements (among others) the provider interface portion of a pluggable interface. Plugin modules can be built in or dynamically loaded. Several alternatives exist for the form of the provider interface, but some have significant advantages in allowing the plugin module to use identical source code for both built-in and loadable modules.

A built-in module is a module whose implementation is already available within the consumer's symbol namespace at the time of module discovery. This typically means a module whose implementation is part of the same code unit as the consumer, though it could also mean a module which was registered by some other code unit.

A dynamically loaded module is a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. (In POSIX systems, this is typically done using <code>dlopen()</code>).

===== Loadable module provider interface =====

The contents of the vtable are specific to the interface, as well as the major version of the interface. The constructor signature uses an abstract type to represent the vtable pointer.

The constructor takes as arguments a major version number, a minor version number, and a pointer to a caller-allocated vtable structure.

The name of the function symbol is constructed from the name of the plugin interface and the name of the plugin module. This allows the caller to see just from the symbol name which interface and plugin it is calling.

===== Built-in-module provider interface =====

A built-in module provides the same interface as a loadable module. In this architecture we use an exported function symbol for each loadable module implementing a pluggable interface.

== Operational Flow ==

=== Startup ===

* The krb5_init_context() function initializes an empty registry for each pluggable interface.

* It then registers libkrb5 built-in modules.

=== Consumer ===

* The consumer registers built-in modules for the desired pluggable interface, if they were not registered by krb5_init_context (because they are not libkrb5 built-in modules).

* The consumer calls the plugin loader function for the desired pluggable interface.

* The loader function calls the plugin manager to retrieve the vtable constructor function for the appropriate module.

* If this is the first load operation for the pluggable interface, the plugin manager performs module discovery and filtering using the appropriate profile variables for the interface.

* The loader function uses the resulting vtable to build an opaque handle to give to the consumer.

* The consumer calls the wrapper functions of the pluggable interface, passing the opaque module handle in order to access the capabilities of the plugin module.

== Interfaces and Functions ==

=== Consumer accessible functions ===

The following functions are meant to be used by a consumer of pluggable interfaces:

; <code> k5_plugin_register</code>: Register a vtable constructor for a built-in module of a specified interface.

=== Loader accessible function ===

The following functions are meant to be used by a loader function of a pluggable interface:

; <code>k5_plugin_load</code>: Obtain a vtable constructor for a named module of a specified interface.

; <code>k5_plugin_load_all</code>: Obtain a list of all available vtable constructors for a specified interface.

; <code>k5_plugin_free_modules</code>: Free a list of vtable constructors allocated by k5_plugin_load_all.

=== Function signatures ===

The function signatures as as follows:

; <code> krb5_error_code
k5_plugin_load(krb5_context context, int interface_id, const char *modname,
krb5_plugin_init_fn *module); </code>

; <code>krb5_error_code
k5_plugin_load_all(krb5_context context, int interface_id,
krb5_plugin_init_fn **modules); </code>

; <code>void
k5_plugin_free_modules(krb5_context context, krb5_plugin_init_fn *modules); </code>

; <code>krb5_error_code
k5_plugin_register(krb5_context context, int interface_id, const char *modname,
krb5_plugin_init_fn module); </code>

== Configuration ==

== Sample Code and Proof of Concept ==

== Deliverables ==

For Release 1.9, the deliverables are (a) plugin framework/manager and pluggable interfaces that can support (b) password strength and (c) password synchronization plugin modules.

These should support the capabilities of two existing extensions written by Russ Allbery -- krb5-strength and krb5-sync. The framework is subject to change in the future, so it doesn't have to accommmodate all eventualities, but we will have a goal of not painting ourselves into a corner with respect to reasonably plausible future requirements.

== Existing Support ==

This section provides some background material
on existing support for pluggable interfaces.

=== Current plugins ===

We currently have the following plugin frameworks:

* Preauth: All shared objects from profile-specified or installation directory are loaded. Two vtables are read from the shared objects, one for libkrb5 and one for the KDC. The preauth framework iterates over the module list invoking functions to generate or handle preauth data. Preauth vtable functions receive a callback function and data object which allow it to request information such as the expected enctype or FAST armor key for the request.

* Authdata: Very similar to the preauth framework.

* KDB: The profile specifies a database library name for each realm. Shared objects matching the library name are loaded from a profile-specified and installation directory; the first matching object with an appropriately-named vtable data object is used, and the rest are ignored. libkdb5 contains wrappers which invoke functions in the library's vtable, or (for some optional functions) default implementations if the vtable left the function pointer as NULL.

* KDC location: All shared objects from an installation directory are located. A vtable is read from the shared objects. The KDC location framework iterates over each vtable and invokes a lookup function; modules can return success with a location, an error (which halts the location process), or a distinguished error code which passes control along to the next module or the built-in location mechanisms.

* GSSAPI: The file /etc/gss/mechs can specify a list of mechanism OIDs and shared object filenames; filenames are taken as relative to an installation directory. Shared objects implementing mechanisms can export either a function returning a vtable, or can export each GSSAPI interface individually.

The following areas of functionality are virtualized but have no exposed plugin framework:

* Serialization: Serialization table entries can be registered with krb5_register_serializer. Data objects are matched to table entries by magic number. The registration function is exported by libkrb5 and is named with the krb5_ prefix, but it and its associated structure are declared in k5-int.h rather than krb5.h. It is not used outside of libkrb5.

* ccache: Very similar to serialization, except that ccache implementations are selected using a URL-style prefix in the ccache name.

* keytab: Very similar to ccache, except that the keytab registration function is used outside of libkrb5 to register a "KDB keytab", which is used by kadmind to serve GSSRPC without requiring a keytab file containing the kadmin keys.

* Replay cache: Very similar to ccache, except that the replay cache registration function is not used anywhere (even inside libkrb5).

Plugin frameworks which are "not exposed" may still be productively used by vendor forks of the krb5 tree.

=== Future planned plugins ===

The following areas are candidates for future plugin support:

* PRNG
* profile / configuration
* DNS / host-realm mapping
* password quality policy
* lockout
* audit
* password synchronization

=== Current support infrastructure ===

In libkrb5support, we have functions to facilitate loading plugins from shared objects. There is a set of functions to load individual plugins from named files and mechglue; these are currently used by the HDB bridge and GSS mechglue:

* krb5int_open_plugin - Create a plugin handle from a filename
* krb5int_close_plugin - Close a plugin handle
* krb5int_get_plugin_data - Retrieve a data object from a plugin handle by symbol name
* krb5int_get_plugin_func - Retrieve a function object from a plugin handle by symbol name

There is another set of functions to scan a list of directories for plugins:

* krb5int_open_plugin_dirs - Create a plugin dir handle from a list of directories and (optionally) filebases
* krb5int_close_plugin_dirs - Close a plugin dir handle
* krb5int_get_plugin_dir_data - Retrieve a list of data objects from a plugin dir handle by symbol name
* krb5int_get_plugin_dir_func - Retrieve a list of function objects from a plugin dir handle by symbol name
* krb5int_free_plugin_dir_data - Free a list of data objects returned by krb5int_get_plugin_dir_data
* krb5int_free_plugin_dir_func - Free a list of function objects returned by krb5int_get_plugin_dir_func

=== Problem areas ===

* Every caller of krb5int_open_plugin_dirs specifies either no filebases (e.g. preauth plugins) or a single filebase (KDB plugins). Accepting and processing a list of filebases is probably needless complexity.

* Callers of krb5int_open_plugin_dirs have to know what directories to supply, which means they need to know the krb5 install root as well as the magic plugin area for OS X, and they need logic for reading a profile variable to determine the alternate plugin directory for the test suite (currently only implemented for KDB and preauth plugins).

* In most uses of plugins, we read a data object containing a list of function pointers. This makes it mostly impossible to supply a plugin which works with multiple versions of krb5. If we instead read a function object which we invoked with a version number to retrieve the vtable, it would be possible (though perhaps awkward) to create a shared object which works with multiple versions.

* We are somewhat schizophrenic about how plugins can access krb5 library functionality, and in particular internal symbols. Sometimes we call functions directly, sometimes we make use of a vtable passed into the plugin (e.g. the preauth_get_client_data_proc function), sometimes we use the accessor to invoke internal functions, and sometimes we call APIs or internal functions directly. Ideally we should have a consistent policy with a sound justification.

* When measuring code coverage with gcov, we cannot use shared libraries; this means we need to link in-tree plugins statically into the libraries or programs which load them. We have an ad-hoc method to do this with KDB plugins, but not with other plugin types.

* Administrators have an easier time writing scripts than creating linkable shared objects. In some cases it might yield a better administrator experience to create plugin interfaces via subprocesses than loading shared objects, although in many cases this might not be feasible.

* In some scenarios such as embedded environments, it may be more useful to allow applications to supply plugin vtables via an API (as we do for keytabs and ccaches, though those APIs are not public) than to load them from shared objects in the filesystem.

== Definitions ==

; pluggable interface: an (internal) interface that can be implemented by a third party. These can be one-to-one, or one-to-many. An example of one-to-one is the DAL, and an example of one-to-many is preauth.

; module: a unit of code that implements a pluggable interface. It can be built in, or it can be dynamically loadable.
:; built-in: a module whose executable code is located within the library shared object or executable program file, or behaves as if it were. (While separate library shared objects that the calling library depends on can contain "built-in" modules for the calling library, this can cause problems with cyclic references.) The distinguishing characteristic of a built-in module is that, as part of program startup, the operating system automatically maps the executable code of the module into the address space of the process that calls it, without any explicit action by the library or program.
:; dynamically loaded: a module whose executable code is located within a file that is distinct from the library or program that calls it. The plugin support framework uses the runtime linker (or equivalent) to explicitly map the executable code of the module into the process address space. In POSIX systems, this is typically done using <code>dlopen()</code>.

; discovery: process of enumerating what modules are available for a pluggable interface. Includes possible filtering of the raw discovered set.
:* compiled-in
:* directory scan
:* explicit inclusion by configuration
:* explicit exclusion by configuration

; loading: the process of making modules available for calling. This can involve dynamically loading a module using the runtime linker, or it can involve registering a vtable provided by an application.
:* built-in
:* dynamic loading
:* application-registered

; selection: the process of a caller invoking one specific module from the set of loaded modules that implement an interface.

; consumer interface: the interface that a caller uses to access the services of a pluggable interface. Typically, but not always, the krb5 library implements the consumer interface.

; provider interface: the interface that a module author implements