View source for Task-List for Samba4 Port (Andrew Bartlett)

== Andrew Bartlett's Task-List (June '09) ==
This document was written by Andrew Bartlett (abartlet@samba.org)
and Don Davis (don@mit.edu), in June 2009.  The original 
document did not take into account MIT's past few years of work 
on kerberos, because Samba4, like Samba3, was built to use 
Heimdal for authentication.  This version of the task-list
includes comments from Luke Howard and others, who know more 
about the current state of interoperability obstacles between 
Samba4 and MIT's kerberos.

<pre>
Copyright Andrew Bartlett <abartlet@samba.org> 2005-2009
Copyright Donald T. Davis <don@mit.edu>        2009

Released under the GPLv3
</pre>

'''Title: Porting Samba4 to MIT-Krb'''

IPA v3 will use a version of Samba4 built on top of MIT's kerberos implementation, instead of Heimdal's version of kerberos.

Task list summary for porting changes needed, from Andrew Bartlett:
* Rewrite or extend the LDAP driver that MIT-KDC will use.
* MIT KDC changes:  rewrite DAL, add TGS-KBAC, enable PACs,...
* Full thread-safety for MIT's library code,
* Many small library changes

=== Introduction: ===

This document should be read alongside
the Samba4 source code, as follows: 
<ol>
<li> For [[DAL and KDC requirements]], please see Samba4's 
     '''source4/kdc/hdb-samba4.c''' in particular.  This file
     is an implementation against Heimdal's HDB abstraction
     layer, and is the biggest part of the samba-to-krb
     glue layer, so the main part of the port to MIT is 
     to replace hdb-samba4 with a similar glue layer
     that's designed for MIT's code. </li>
<li> Samba4's [[PAC requirements]] are implemeneted in 
     '''source4/kdc/pac-glue.c''' </li>
<li> Both of the above two layers are [[Heimdal plugins]], and 
     both get loaded in '''source4/kdc/kdc.c''' </li>
<li> For [[GSSAPI requirements]], see auth/gensec/gensec_gssapi.c 
     (the consumer of GSSAPI in Samba4)</li>
<li> For [[Kerberos library requirements]], see 
     '''auth/kerberos/krb5_init_context.c''' .</li>
<li> [[Samba's credentials system]] wraps GSS creds, 
     just as GSS creds wrap around krb5 creds.  For the 
     interaction between Samba4 credential system and GSSAPI
     and Kerberos, see '''auth/credentials/credentials_krb5''' . </li>
</ol>

----

=== LDAP driver ===
Rewrite or extend the LDAP driver that MIT-KDC will use.
Most or all of the Heimdal LDAP driver code is in three source
files, ~1000 lines in all.  These files are in '''samba4/kdc''' :
* '''hdb-samba4.c'''  (samba4-to-kdb glue-layer plugin)
* '''pac-glue.c'''    (samba4's pac  glue-layer plugin)
* '''kdc.c'''         (loads the above two plugins).

==== Name Canonicalization ====
The LDAP driver for the Samba4 KDB needs to know how to do
Samba4's intricate canonicalization of server names,
user-names, and realm names. 
  <ol>
  <li> [[For hostnames & usernames,]] alternate names appear in
       LDAP as extra values in the multivalued "principal name" 
       attributes:
       <ol>
       <li> For a hostname, the alternate names (other than
           the short name, implied from the CN), are stored in 
           the servicePrincipalName.
* LH: This is implied from samAccountName, I would think; CN is not used by the security subsystem."
       </li>
       <li> For a username, the alternate names are stored in 
           the userPrincipalName attribute, and can be long 
           email-address-like names, such as joe@microsoft.com 
           (see "Type 10 names," below). </li>
       </ol></li>
  <li> [[GSSAPI layer requirements]]: The MIT Krb5 libs (including 
       GSSAPI) do not enable the AS to send kinit a TGT containing
       a different realm-name than what the client asked for,
       even in U/L case differences.  Heimdal has the same problem, 
       and this applies to the krb5 layer too, not just GSSAPI.
       There are two kinds of name-canonicalization that can 
       occur on Windows:
       <ol>
       <li> Lower-to-upper case conversion, because Windows domain 
           names are usually in upper case; </li>
       <li> An unrecognizable subsitution of names, such as might 
           happen when a user requests a ticket for a NetBIOS domain 
           name, but gets back a ticket for the corresponging FQDN.</li>
       </ol>
As developers, we should test if the AD KDC's name-canonicalisation can be turned off with the KDCOption flags in the AS-REQ or TGS-REQ;  Windows clients always send the Canonicalize flags as KDCOption values. </li>
       </li>
  <li> [[Principal Names, long and short names:]]
       AD's KDC does not canonicalize servicePrincipalNames, except 
       for the realm in the KDC reply.  That is, the client gets 
       back the principal it asked for, with the realm portion 
       'fixed' to uppercase, long form. 
 
       Samba4 does some canonicalization, though Heimdal doesn't 
       canonicalize names itself:  For hostnames and usernames, 
       Samba4 canonicalizes the requested name only for the LDAP 
       principal-lookup, but then Samba4 returns the retrieved LDAP 
       record with the request's original, uncanonicalized hostname 
       replacing the canonicalized name that actually was found. </li>
  <li> [[Usernames]]:  AndrewB says that Samba4 used to return 
       the canonicalized username exactly as retrieved from LDAP.  
       The reason Samba4 treated usernames differently was that 
       the user needs to present his own canonicalized username 
       to servers, for ACL-matching.  For hostnames this isn't 
       necessary. </li>
  <li> [[Realm-names]]:  AD seems to accept a realm's short name 
       in krb-requests, at least for AS_REQ operations, but the 
       AD KDC always performs realm-canonicalisation, which 
       converts the short realm-name to the canonical long form.  
       So, this causes pain for current krb client libraries. 
       Luke H says:
       <ol>
       <li> "If you set the canonicalize flag you will be fine. </li> 
       <li> If not, then yes, the server returning a different
            realm will cause a problem for the MIT client libraries. </li>
       <li> Take a look at how '''verify_as_reply()''' selectively
            enforces the KDC_OPT_CANONICALIZE flag, in 
            MIT's '''src/lib/krb5/krb/get_in_tkt.c'''." </li>
       </ol>
[[Punchline]]:  For bug-compatibility, we may need to 
selectively or optionally disable the MIT-KDC's name-
canonicalization. 
  </li>
  <li> [[Application-code:]]
       Name-canonicalisation matters not only for the KDC, but 
       also for app-server-code that has to deal with keytabs.
       <ol><li>
           Luke H. says,
           "MIT's canonicalization works with 1.7 as long as you don't 
           provide a server principal to '''krb5_rd_req()'''; 
           '''rd_req()''' will iterate through the keytab, looking 
           for an entry that will decrypt the ticket."
       </li></ol>
Further, with credential-caches, canonicalization can
lead to cache-misses, but then the client just asks for 
new credentials for the variant server-name.  This could 
happen, for example, if the user asks to access the 
server twice, using different variants of the server-name.
  </li>
</ol>

==== Doubled realm-names ====
Samba4's KDC also needs to handle type 10 
names (NT-ENTERPRISE), which have a full principal name 
in the principal field, unrelated to the realm.  Thus, 
the principal field contains both principal & realm names, 
while the realm field contains a realm name, too, possibly 
different.  
This is used for AD's "email-address-like login-names." 
For example, an NT-ENTERPRISE principal name might look like:
<pre> 
       joeblow@microsoft.com@NTDEV.MICROSOFT.COM ,
       <--principal field-->|<----realm name--->|
</pre>
Where joe@microsoft.com is the leading portion, and 
NTDEV.MICROSOFT.COM is the realm.

Luke H. says, 
* "MIT's client libraries and KDC support this (to the extent necessary). 
* Most of the work needs to happen in Samba4's new LDAP backend.

==== Keytab-sharing amongst HOST/ service names ====
AD keeps a list of service-prefixed aliases for the host's
principal name.  The AD KDC reads & parses this list, so 
as to allow the aliased services to share the HOST/ key.
This means that every ticket-request for a service-alias 
gets a service-ticket encrypted in the HOST/ key.

For example, this is how HTTP/ and CIFS/ can use the 
HOST/ AD-LDAP entry, without any explicitly CIFS-prefixed
entry in the host's servicePrincipalName attribute. In the 
app-server host's AD record, the servicePrincipalName says
only HOST/my.computer@MY.REALM , but the client asks
for  CIFS/my.omputer@MY.REALM tickets.  So, AD looks in 
LDAP for both name-variants, and finds the HOST/ version,
In AD's reply, AD replaces the HOST/ prefix with CIFS/ .
We implement this in '''hdb-ldb'''.
[[(TBD: Andrew, is this correct?:)]]

List of HOST/ aliases:  Samba4 currently uses only a small
set of  HOST/ aliases, called "sPNMappings:"

<pre>
       host=ldap,dns,cifs,http . 
</pre> 
BTW, dns's presence in this short list is a bug, somehow.

AD's real sPNMappings list has 52 entries:
<pre>
       host =
	alerter   eventsystem  netlogon      rpclocator  tapisrv  
	appmgmt   fax          netman        rpcss       time     
	browser   http         nmagent       rsvp        trksvr   
	cifs      ias          oakley        samss       trkwks   
	cisvc     iisadmin     plugplay      scardsvr    ups      
	dcom      mcsvc        policyagent   scesrv      w3svc    
	dhcp      messenger    protectedsto  schedule    wins     
	dmserver  msdtc        rasman        scm         www      
	dns       msiserver    remoteaccess  seclogon    
	dnscache  netdde       replicator    snmp        
	eventlog  netddedsm    rpc           spooler     
</pre>
Domain members that expect the longer list will break in 
Samba4, as of 6/09.  AB says he'll try to fix this right 
away.  There is another post somewhere (ref lost for the 
moment) that details where active directory stores this long
list  of stored  aliases for HOST/.

==== Implicit names for Win2000 Accounts ====
AD keys its 
server-records by CN or by servicePrincipalName, but a
win2k box's server-entry in LDAP doesn't include the
servicePrincipalName attribute,  So, win2k server-accounts 
are keyed by the CN attribute instead.  Because AD's LDAP 
doesn't have a servicePrincipalName for win2k servers'
entries, Samba4 has to have an implicit mapping from 
host/computer.full.name and from host/computer, to the 
computer's CN-keyed entry in the AD LDAP database, so as to 
be able to find the win2k server's host name in the KDB.

==== Principal "types" ====
Samba4 has modified Heimdal's 'hdb' interface to specify 
the 'class' of Principal being requested.  This allows
us to correctly behave with the different 'classes' of 
Principal name.  This is necessary because of AD's LDAP
structure, which uses very different record-structures 
for user-principals, trust principals & server-principals.
We currently define 3 classes:
  * client (kinit)
  * server (tgt)
  * krbtgt  the TGS's own ldap record
Samba4 also now specifies the kerberos principal as an 
explicit parameter to LDB_fetch(), not an in/out value 
on the struct hdb_entry parameter itself.

Luke H says this isn't really necessary:
<ol>
<li> I quite like the principal-classes idea, but it mightn't 
     be a good fit for the existing KDC code. </li>
<li> You can find out whether it is a TGS from the principal 
     name. </li>
<li> You can also find out fairly easily whether it is the 
     TGS from your local realm or not, or outgoing/incoming. </li> 
<li> You can check the get_principal_ext flag 
     KRB5_KDB_FLAG_CLIENT_REFERRALS_ONLY to determine whether 
     you are looking up a client in the AS-REQ path. </li>
</ol>

----

=== * [[MIT KDC changes]] ===

==== Data-Abstraction Layer (DAL) ====
It would be good to
rewrite or circumvent the MIT KDC's DAL, mostly because 
the MIT KDC needs to see & manipulate more LDAP detail, 
on Samba4's behalf.  
<ol>
    <li> MIT's DAL calls lack context parameters (as of 2006), 
         so presumably they rely instead on global storage, and 
         aren't fully thread-safe.
    </li> 
    <li> In Novell's pure DAL approach, the DAL only read in the 
         principalName as the key, so it had trouble performing 
         access-control decisions on things other than the user's
         name (like the addresses).
         <ol>
         <li> Luke H says, "The KDC has a different model here -- 
              a separate authorization check.
         </li>
         <li> But I agree, this could be improved."
         </li>
         </ol>
    </li>
    <li> Here's why Samba4 needs more entry detail than the DAL
         provides:  The AS needs to have ACL rules that will allow
         a TGT to a user only when the user logs in from the
         right desktop addresses, and at the right times of day.
         This coarse-granularity access-control could be enforced 
         directly by the KDC's LDAP driver, without Samba having 
         to see the entry's pertinent authZ attributes.  But, 
         there's a notable exception:  a user whose TGT has 
         expired, and who wants to change his password, should 
         be allowed a restricted-use TGT that gives him access 
         to the kpasswd service.  This ACL-logic could be buried 
         in the LDAP driver, in the same way as the TGS ACL could
         be enforced down there, but to do so would just be even 
         uglier than it was to put the TGS's ACL-logic in the driver.
         <ol>
         <li> Luke H says, "This works today, you just need to set 
              '''KRB5_KDB_PWCHANGE_SERVICE''' appropriately."
         </li>
         </ol>
    </li>
    <li> Yet another complaint is that the DAL always pulls an 
         entire LDAP entry, non-selectively.  The current DAL 
         is OK for Samba4's purposes, because Samba4 only reads, 
         and doesn't write, the KDB.  But this all-or-nothing
         retrieval hurts the KDC's performance, and would do so
         even more, if Samba had to use the DAL to change KDB
         entries.
         <ol>
         <li> Luke H says, "You can have your backend retrieve 
              a subset of attributes based on the flags passed to 
              '''get_principal_ext()'''.
         </li>
         <li> "I don't understand the comment "DAL always pulls 
              an entire LDAP entry", because the DAL is just an 
              abstraction layer. 
         </li>
         <li> "FWIW, I wouldn't use the Novell LDAP backend, 
              I would just port your existing backend to DAL."
         </li>
         </ol>
    </li>
</ol>

There's some confusion about whether the MIT DAL is OK:
<ol>
<li> AB says "The MIT DAL may serve well-enough, mostly as is,  
</li>
<li> ...but Samba4 will need the 
     [[#.2A_Appendix_2:_windc_KDC_Plugin_for_Account-AuthZ |'''private pointer''']]
     part of the KDC plugin API, or the 
     [[#.2A.2A_Turn_on_MIT-krb_1.7.27s_PAC_handling | '''PAC generation''']] 
     won't work."
</li>
<li> Luke H replies, "You can use e_data in '''krb5_db_entry()'''
     to store some private data,"
</li>
<li> but DTD thinks Luke doesn't understand what AB means by
     "[[#.2A_Appendix_2:_windc_KDC_Plugin_for_Account-AuthZ |'''private pointer''']]."
</li>
</ol>

==== *** [[Add access-control to the TGS]] ====
For AD compatibility, Samba4 has to be able to 
refuse TGTs to kinit, based on time-of-day & 
IP-address constraints.  This is standard AD behavior 
that Samba4 needs to get right, 
whether Heimdal or MIT-krb is doing the ticket work.
[[Samba4 doesn't yet have these TGS access-controls with Heimdal.]]

AB asks, "Is a DAL the layer we need for Samba4 HBAC?"  
Looking at what we need to pass around, AB doesn't think
the DAL is the right layer; what we really want instead
is to create an account-authorization abstraction layer 
(e.g., is this account permitted to login to this computer,
at this time?).  Samba4 ended up doing account-authorization
inside Heimdal, via a specialized KDC plugin called '''windc'''.  
For a summary description of the '''windc''' plugin API, see 
[[#.2A_Appendix_2:_windc_KDC_Plugin_for_Account-AuthZ | '''Appendix 2''']].

Luke H says, "Well, this exists today. See above."

===== HBAC for the TGS =====
Samba4 doesn't yet have HBAC hooks in the Heimdal KDC.
       <ol>
       <li> If PADL doesn't publish their patch for this, 
            we'll need to write our own.
       </li>
       <li> LH: "use the '''KRB5_KDB_METHOD_CHECK_POLICY_TGS''' method. 
* Samba4 has access to the complete request. 
* See '''against_local_policy_tgs()''' in MIT's '''src/kdc/policy.c''' ."
       </li>
       <li> The '''windc''' plugin provides a function for the main 
         access control routines.  A new '''windc''' function
         should be added to increment the bad password counter
         on failure.
       </li>
       <li>
         [[AllowedWorkstationNames and Krb5]]:  Microsoft uses the 
         clientAddresses *multiple value* field in the krb5
         protocol (particularly the AS_REQ) to communicate the 
         client's netbios name (legacy undotted name, <14 chars)
         AB guesses that this is to support the userWorkstations 
         field (in user's AD record).  The idea is to support 
         client-address HBAC restrictions, as was standard in NT:
         The AD authentication server probably checks the netbios 
         address against this userWorkstations value (BTW, the 
         NetLogon server does this, too).
       </li>
       </ol>

===== Failed PW lockouts =====
Samba4 needs to lockout accounts (eg, after 10 failed PW-attempts). 
       <ol>
       <li> Tom Yu correctly points out that
         no-one can tell how many failed PW-attempts the user
         makes against kinit's PW-encrypted tickets.  
       </li>
       <li> DTD: Maybe we can count rapidly-repeated kinit requests, 
         instead?
       </li>
       <li> Samba4 doesn't yet handle bad password counts (or good 
         password notification), so that a single policy can be 
         applied against all means of checking a password (NTLM, 
         Kerberos, LDAP Simple Bind, etc).  Novell's original DAL
         did not provide a way to update the PW counts information.
       </li>
       <li> Nevertheless, we know that this is very much required in
         AD, because Samba3 + eDirectory goes to great lengths to
         update this information.  (This may have been addressed in
         Simo's subsequent IPA-KDC design.)
       </li>
       <li> Luke H says this is straightforward:
*  Use a '''KRB5_KDB_METHOD_AUDIT_AS''' method for this.
*  See the end of '''log_as_req()''' in MIT's '''/src/kdc/kdc_util.c'''.  
       </li>
       </ol>

==== ** Turn on MIT-krb 1.7's [[PAC handling]] ====
In addition, Andrew Bartlett has added a new interface '''hdb_fetch_ex()''', 
which returns a structure including a private data-pointer, 
which may be used by the '''windc''' plugin inferface functions.
The '''windc''' plugin provides the hook for the PAC.

----

=== ** [[State-machine safety for krb libraries]] ===
Samba's client-side & app-server-side libraries are built 
on a giant state machine, and as such have very different 
requirements to those traditionally expressed for kerberos 
and GSSAPI libraries. 
[[MIT's code has this level of thread-safety now.]]
     <ol>
     <li>
     Samba requires all of the libraries it uses to be "state 
     machine safe" in their use of internal data.  This does not 
     necessarily mean "thread safe," and an application could be 
     thread safe, but not state machine safe (if it instead used 
     thread-local variables).  so, if MIT's libraries were made 
     thread-safe only by inserting spinlock() code, then the MIT
     libraries aren't yet "state machine safe."
     </li>
     <li>
     So, what does it mean for a library to be state machine safe?  
     This is mostly a question of context, and how the library manages 
     whatever internal state machines it has.  If the library uses a 
     context variable, passed in by the caller, which contains all 
     the information about the current state of the library, then it 
     is safe.  An example of this state is the sequence number and 
     session keys for an ongoing encrypted session).
     </li>
     <li>
     The other issue affecting state machines is 'blocking' (waiting for a
     read on a network socket).  Samba's non-blocking I/O doesn't like
     waiting for libkrb5 to go away for awhile to talk to the KDC.
     </li>
     <li>
     Samba4 provides a hook 'send_to_kdc', that allows Samba4 to take over the 
     IO handling, and run other events in the meantime.  This uses a 
     'nested event context' (which presents the challenges that the kerberos 
     library might be called again, while still in the send_to_kdc hook).
     </li>
     <li>
     Heimdal has this 'state machine safety' in parts, and we have modified
     Samba4's lorikeet branch to improve this behaviour, when using a new,
     non-standard API to tunnelling a ccache (containing a set of tickets)
     through the gssapi, by temporarily casting the ccache pointer to a 
     gss credential pointer.  This new API is Heimdal's samba4-requested 
     '''gss_krb5_import_cred()''' fcn;  this will have to be rewritten or ported 
     in the MIT port.
     </li>
     <li>
     This tunnelling trick replaces an older scheme using the '''KRB5_CCACHE''' 
     environment variable to get the same job done.  The tunnelling trick 
     enables a command-line app-client to run kinit tacitly, before running 
     GSSAPI for service-authentication.  The tunnelling trick avoids the 
     more usual approach of keeping the ccache pointer in a global variable.
     </li>
     <li>
     [Heimdal uses a per-context variable for the '''krb5_auth_context()''', 
     which controls the ongoing encrypted connection, but does use global
     variables for the ubiquitous '''krb5_context''' parameter. (No longer true,
     because the '''krb5_context''' global is gone now.)] 
     </li>
     <li>
     The modification that has added most to 'state machine safety' of
     GSSAPI is the addition of the '''gss_krb5_acquire_creds()''' function.  
     This allows the caller to specify a keytab and ccache, for use by 
     the GSSAPI code.  Therefore there is no need to use global variables 
     to communicate this information about keytab & ccache. 
     </li>
     <li>
     At a more theoretical level (simply counting static and global
     variables) Heimdal is not state machine safe for the GSSAPI layer.
     (But Heimdal is now (6/09) much more nearly free of globals.)
     The Krb5 layer alone is much closer, as far as I can tell, blocking
     excepted. .
     </li>
     <li>
     As an alternate to fixing MIT Kerberos for better safety in this area, 
     a new design might be implemented in Samba, where blocking read/write 
     is made to the KDC in another (fork()ed) child process, and the results 
     passed back to the parent process for use in other non-blocking
     operations. 
     </li>
     <li>
     To deal with blocking, we could have a fork()ed child per context,
     using the 'GSSAPI export context' function to transfer
     the GSSAPI state back into the main code for the wrap()/unwrap() part
     of the operation.  This will still hit issues of static storage (one
     '''gss_krb5_context''' per process, and multiple GSSAPI encrypted sessions
     at a time) but these may not matter in practice.
     </li>
     <li>
     This approach has long been controversial in the Samba team. 
     An alternate way would be to be implement E_AGAIN in libkrb5:  similar
     to the way to way read() works with incomplete operations.  to do this
     in libkrb5 would be difficult, but valuable.
     </li>
     <li>
     In the short-term, we deal with blocking by taking over the network
     send() and recv() functions, therefore making them 'semi-async'.  This
     doens't apply to DNS yet.These thread-safety context-variables will
     probably present porting problems, during the MIT port.  This will
     probably be most of the work in the port to MIT.
     This may require more thorough thread-safe-ing work on the MIT libraries.
     </li>
     </ol>

----

=== Many small library changes (~15) ===
==== MIT libraries ====
===== * Krb5 lib & GSSAPI =====
Some extensions to MIT's [[libkrb5 & GSSAPI libraries]], including 
GSSAPI ticket-forwarding:  This is a general list of the other 
extensions Samba4 has made to / need from the kerberos libraries
<ol>
</li>
<li>   '''[[DCE_STYLE]]''': Microsoft's hard-coded 3-msg Challenge/Response handshake
       emulates DCE's preference for C/R.  Microsoft calls this DCE_STYLE.
       [[MIT already has this nowadays (6/09)]].
</li>
<li>   '''[[gsskrb5_get_initiator_subkey()]]''' (return the exact key that 
       Samba3 has always asked for.  '''gsskrb5_get_subkey()''' might 
       do what we need anyway).  This routine is necessary, because 
       in some spots, Microsoft uses raw Kerberos keys, outside the 
       Kerberos protocols, as a direct input to MD5 and ARCFOUR, 
       without using the '''make_priv()''' or '''make_safe()''' 
       calls, and without GSSAPI wrappings etc.
       <ol>
       <li> 
       Luke H says, "'''gsskrb5_get_subkey()''' doesn't exist in MIT's krb. 
       </li>
       <li> "But you can use '''GSS_C_INQ_SSPI_SESSION_KEY''' instead." 
       </li>
       </ol>
[[So this should be easy enough]], according to LH.
</li>
<li>   '''gsskrb5_acquire_creds()''' (takes keytab and/or ccache as input
       parameters, see keytab and state machine discussion in 
       [[#.2A.2A_State-machine_safety_for_krb_libraries | '''previous section, bullet 8''']])
</li>
<li>   '''gsskrb5_extract_authz_data_from_sec_context()''' (The new function 
       to handle the PAC fully).  
       <ol>
       <li> AB:  We'll need to test that MIT's PAC-handling 
            code checks the PAC's signature.
       </li>
       </li> Luke H says, "It does not, a future API might do that. </li>
       <li>  "in the meantime you need to extract it and verify it explicitly. </li>
       <li>  "You can use '''krb5_pac_verify()''' for this (same API as Heimdal)." </li>
       </ol>
</li>
<li>   '''gsskrb5_wrap_size()''' (find out how big the wrapped 
       packet will be, given input length).
       <ol>
       <li> AB says, "Samba still needs this one." </li>
       <li> Luke says, "You can use '''gss_wrap_iov_length()''' for this, but... </li>
       <li> "...it won't really help you for MIC tokens, </li>
       <li> "You can take advantage of two facts:
* "MIC tokens are never bigger than wrap header tokens for Kerberos and NTLM, 
* "MS RPC permits the authentication trailer value to have redundant bytes at the end."
       </li>
       </ol>
</ol>

===== Some refitting in Samba4's use of the MIT libs =====
===== ** [[Legacy Samba3 clients & GSSAPI]] =====
MIT's GSSAPI code should support some legacy Samba3 
clients that present incorrectly-calculated checksums.
(Luke H: [["This is already in MIT's v1.7.]])
<ol>
<li> Old Clients (samba3 and HPUX clients) use 'selfmade'
     gssapi/krb5 tokens for use in the CIFS session setup.
     These hand-crafted ASN.1 packets don't follow rfc1964
     (GSSAPI) perfectly, so server-side krblib code has to
     be flexible enough to accept these bent tokens.  
</li>
<li> It turns out that Windows' GSSAPI server-side code is
     sloppy about checking some GSSAPI tokens' checksums.
     During initial work to implement an AD client, it was
     easier to make an acceptable solution (acceptable to
     Windows servers) than to correctly implement the
     GSSAPI specification, particularly on top of the 
     (inflexible) MIT Kerberos API.  It did not seem 
     possible to write a correct, separate GSSAPI 
     implementation on top of MIT Kerberos's public 
     krb5lib API, and at the time, the effort did not 
     need to extend beyond what Windows would require.  
</li>
<li> The upshot is that old Samba3 clients send GSSAPI 
     tokens bearing incorrect checksums, which AD's 
     GSSAPI library cheerfully accepts (but accepts 
     the good checksums, too).  Similarly, Samba4's 
     Heimdal krb5lib accepts these incorrect checksums.  
     Accordingly, if MIT's krb5lib wants to interoperate
     with the old Samba3 clients, then MIT's library will 
     have to do the same.
</li>
<li> Because these old clients use krb5_mk_req()
     the app-servers get a chksum field depending on the 
     encryption type, but that's wrong for GSSAPI (see 
     rfc 1964 section 1.1.1). The Checksum type 8003 
     should be used in the Authenticator of the AP-REQ! 
     That (correct use of the 8003 type) would allow 
     the channel bindings, the GCC_C_* req_flags and 
     optional delegation tickets to be passed from the 
     client to the server.  However windows doesn't seem
     to care whether the checksum is of the wrong type,
     and for CIFS SessionSetups, it seems that the 
     req_flags are just set to 0.  This deviant checksum
     can't work for LDAP connections with sign or seal,
     or for any DCERPC connection, because those 
     connections do not require the negotiation of 
     GSS-Wrap paraemters (signing or sealing of whole
     payloads).  Note:  CIFS has an independent SMB 
     signing mechanism, using the Kerberos key.
</li>
<li> For the code that handles the incorrect & correct
     checksums, see heimdal/lib/gssapi/krb5/accept_sec_context.c,
     lines 390-450 or so.
</li>
<li> This bug-compatibility is likely to be controversial 
     in the kerberos community, but a similar need for bug-
     compatibility arose around MIT's & Heimdal's both 
     failing to support TGS_SUBKEYs correctly, and there 
     are numerous other cases.
     see https://lists.anl.gov/pipermail/ietf-krb-wg/2009-May/007630.html
</li>
<li> So, MIT's krb5lib needs to also support old clients!
</li>
</ol>

==== Samba4 / AD libraries ====
===== NTLM support =====
For AD compatibility, Samba4 has to support NTLM authentication,
so the NTLM library has to be able to access the MIT KDB.
AB says we should be able to get an OSS NTLM authentication
library: AB says Likewise software probably will give us their
OSS "NTLM for MIT-krb" implementation.

===== ** [[Samba4's portable socket API]] =====
(This section assumes that we'll build a [[#libkdc | '''libkdc''']], 
which seems unlikely.) Make sure Samba4's portable socket API works:
<ol>
<li>   An important detail in the use of libkdc is that samba4 uses its 
       own socket lib.  This allows the KDC code to be as portable as 
       the rest of samba, but more importantly it ensures consistancy 
       in the handling of requests, binding to sockets etc.
</li>
<li>   To handle TCP, we use of our socket layer in much the same way as
       we deal with TCP for CIFS.  Tridge created a generic packet handling
       layer for this.
</li>
<li>   For the client, samba4 likewise must take over the socket functions, 
       so that our single thread smbd will not lock up talking to itself.  
       (We allow processing while waiting for packets in our socket routines).
       send_to_kdc()  presents to its caller the samba-style socket interface,
       but the MIT port will reimplement send_to_kdc(), and this routine will
       use internally the same socket library that MIT-krb uses.
</li>
<li>   The interface we have defined for libkdc allows for packet injection
       into the post-socket layer, with a defined krb5_context and
       krb5_kdc_configuration structure.  These effectively redirect the
       kerberos warnings, logging and database calls as we require.
</li>
<li>   Samba4 socket-library's current TCP support does not send back 
       'too large' error messages if the high bit is set.  This is 
       needed for a proposed extension mechanism (SSL-armored kinit, 
       by Leif Johansson <leifj@it.su.se>), but is currently unsupported 
       in both Heimdal and MIT.
</li>
</ol>

==== Key-handling changes ====
<ol>
<li> Samba4 app-server-host holds a [[UTF-16 PW]], plus a key bitstring; See
     [[#Appendix_1:__Keytab_Requirements | '''Appendix 1, Keytab Requirements''']].
</li>
<li> In-memory-only credentials cache for forwarded tickets
     Samba4 extracts forwarded tickets from the GSSAPI layer, 
     and puts them into the memory-based credentials cache.  
     We can then use them for proxy work.  This needs to be
     ported, if the MIT library doesn't do it yet.
</li>
<li> [[In-memory-only keytab]] (nice to have):
     Heimdal used to offer "in-memory keytabs" for servers that use 
     passwords.  These server-side passwords were held in a Samba LDB 
     database called secrets.ldb .  The heimdal library would fetch 
     the server's password from the ldb file and would construct an 
     in-memory keytab struct containing the password, somewhat as if 
     the library had read an MIT-style keytab file.  Unfortunately, 
     only later, at recv_auth() time, would the Heimdal library convert 
     the server-PW into a salted-&-hashed AES key, by hashing 10,000 
     times with SHA-1.  Naturally, this is really too slow for recv_auth(), 
     which runs when an app-server authenticates a client's app-service-
     request.  So, nowadays, this password-based in-memory keytab  is 
     falling into disuse.
</li>
</ol>

==== * Extra krb library functions ====
<ol>
<li> Special Heimdal-specific functions;  These functions didn't 
     exist in the MIT code, years ago, when Samba started.  
     <ol>
     <li> '''krb5_free_keyblock_contents()''' </li>
     </ol>
AB will try to build a final list of these functions:
</li>
<li> Principal-manipulation functions:  Samba makes extensive
     use of the principal manipulation functions in Heimdal, 
     including the known structure behind '''krb5_principal''' and
     '''krb5_realm''' (a char *).  For example,
   <ol>
   <li> '''krb5_parse_name_flags'''(smb_krb5_context->krb5_context, name, 
                             KRB5_PRINCIPAL_PARSE_REQUIRE_REALM, &principal);
   </li>
   <li> '''krb5_unparse_name_flags'''(smb_krb5_context->krb5_context,    principal, 
                             KRB5_PRINCIPAL_UNPARSE_NO_REALM,    &new_princ);
   </li>
   <li> '''krb5_principal_get_realm'''()
   </li>
   <li> '''krb5_principal_set_realm'''()
   </ol>
These are needed for juggling the AD 
variant-structures for server names.
(Luke says [[the last two are in MIT's code now.]])
</li>
</ol>

==== Error-handling, logging, testing ====
<ol>
<li> Special [[Short name rules]] check for misconfigured Samba4
     hostnames;  Samba is highly likely to be misconfigured, in 
     many weird and interesting ways.  So, we have a patch for 
     Heimdal that avoids DNS lookups on names without a "." in 
     them.  This should avoid some delay and root server load.
     (These errors need to be caught in MIT's library.)
</li>
<li> Improved krb error-messages;
</li>
<li> Improved Kerberos logging support:  
   <ol>
   <li> '''krb5_log_facility()''': Samba4 now uses this Heimdal function,
       which allows us to redirect the warnings and status from 
       the KDC (and client/server Kerberos code) to Samba's DEBUG() 
       system.  Samba uses this logging routine optionally in the 
       main code, but it's required for KDC errors.  
   </li>
   <li> '''krb5_get_error_string'''():  This Heimdal-specific function
       does a lot to reduce the 'administrator pain' level, by
       providing specific, English text-string error messages 
       instead of just error code translations.  (This isn't 
       necessary for the port, but it's more useful than MIT's 
       default err-handling;  Make sure this works for MIT-krb)
   </li>
   </ol>
</li>
<li> MS GSSMonger test-suite:  Microsoft has released a krb-specific 
     testsuite called gssmonger, which tests interoperability.  We 
     should compile it against lorikeet-heimdal & MIT and see if we 
     can build a 'Samba4' server for it.  GSSMonger wasn't intended 
     to be Windows-specific.
</li>
<li> Testsuite for kpasswd daemon:  I have a partial kpasswd server 
     which needs finishing, and Samba4 needs a client testsuite
     written, either via the krb5 API or directly against GENSEC and 
     the ASN.1 routines.  Samba4 likes to test failure-modes, not 
     just successful behavior.  Currently Samba4's kpasswd only works
     for Heimdal, not MIT clients.  This may be due to call-ordering 
     constraints.
</li>
</ol>

----

=== Appendix 1:  Keytab Requirements ===

Traditional 'MIT' keytab operation is very different from AD's 
account-handling for application-servers:
==== Host PWs vs service-keys ====
<ol>
   <li> Traditional 'MIT' behaviour is for the app-server to use a keytab
        containing several named random-bitstring service-keys, created 
        by the KDC.  An MIT-style keytab holds a different service-key 
        for every kerberized application-service that the server offers 
        to clients.  Heimdal also implements this behaviour.  MIT's model 
        doesn't use AD's UTF-16 'service password', and no salting is 
        necessary for service-keys, because each service-key is random 
        enough to withstand an exhaustive key-search attack.
   </li>
   <li> In the Windows model, the server key's construction is very 
        different:  The app-server itself, not the KDC, generates a 
        random UTF-16 pseudo-textual password, and sends this password 
        to the KDC using SAMR, a DCE-RPC "domain-joining" protocol (but 
        for windows 7, see below).  Then, the KDC shares this server-
        password with every application service on the whole machine.  
   </li>
   <li> Only when the app-server uses kerberos does the password get 
        salted by the member server (ie, an AD server-host).  (That 
        is, no salt information appears to be conveyed from the AD KDC 
        to the member server, and the member server must use the rules 
        described in Luke's mail, in Appendix 3, below).  The salted-
        and-hashed version of the server-host's PW gets stored in the
        server-host's keytab.
   </li>
   <li> Samba file-servers can have many server-names simultaneously 
        (kind of like web servers' software-virtual-hosting), but since 
        these servers are running in AD, these names can be set up to 
        all share the same secret key.  In AD, co-located server names 
        almost always share a secret key like this.  In samba3, this 
        key-sharing was optional, so some samba3 hosts' keytabs did 
        hold multiple keys.  Samba4 abandons this traditional "old MIT"
        style of keytab, and only supports one key per keytab, and 
        multiple server-names can use that keytab key in common.  In
        dealing with this model, Samba4 uses both the traditional file
        keytab and an in-MEMORY keytabs.  
   </li>
   <li> Pre-Windows7 AD and samba3/4 both use SAMR, an older protocol, 
        to jumpstart the member server's PW-sharing with AD (the "windows 
        domain-join process").  This PW-sharing transfers only the PW's
        UTF-16 text, without any salting or hashing, so that non-krb 
        security mechanisms can use the same utf-16 text PW.  For 
        Windows 7, this domain-joining uses LDAP for PW-setting.
   </li>
</ol>
==== Flexible server-naming ====

The other big difference between AD's keytabs and MIT's is that 
Windows offers a lot more flexibility about service-principals' 
names. When the kerberos server-side library receives Windows-style 
tickets from an app-client, MIT's krb library (or GSSAPI) must 
accommodate Windows' flexibility about case-sensitivity and 
canonicalization. 
This means that an incoming application-request to a member server
may use a wide variety of service-principal names.  These include:
<pre>
              machine$@REALM      (samba clients)
          HOST/foo.bar@realm      (win2k clients)
          cifs/foo.bar@realm      (winxp clients)
              HOST/foo@realm      (win2k clients, using netbios)
              cifs/foo@realm      (winxp clients, using netbios),
</pre>
as well as all upper/lower-case variations on the above.
</ol>
==== * [[Keytabs & Name-canonicalization]] ====
<ol>
   <li> Heimdal's GSSAPI expects to to be called with a 
        principal-name & a keytab, possibly containing 
        multiple principals' different keys.  However, 
        AD has a different problem to solve, which is 
        that the client may know the member-server by a 
        non-canonicalized principal name, yet AD knows 
        the keytab contains exactly one key, indexed by
        the canonical name.  So, GSSAPI is unprepared 
        to canonicalize the server-name that the client 
        requested, and is also overprepared to do an 
        unnecessary search through the keytab by 
        principal-name.  So Samba's server-side GSSAPI 
        calls have to "game" the GSSAPI, by supplying 
        the server's known canonical name, with the one-key 
        keytab. This doesn't really affect IPA's port of 
        Samba4 to MIT-krb.
* Luke says, [["This is much easier in 1.7, because all keys are tried.]]"
   </li>
   <li> Because the number of U/L case combinations got 
        'too hard' to put into a keytab in the traditional
        way (with the client to specify the name), we 
        either pre-compute the keys into a traditional 
        keytab or make an in-MEMORY keytab at run time.  
        In both cases we specifiy the principal name to 
        GSSAPI, which avoids the need to store duplicate 
        principals.
   </li>
   <li> Samba4 uses a 'private' keytab in a private directory, 
        referenced from the secrets.ldb by default.
   </li>
</ol>

----

=== * Appendix 2: [[windc KDC Plugin for Account-AuthZ]] ===

Samba4 does account-authorization in Heimdal with
the specialized KDC plugin '''windc'''.  
This plugin helps bridge an important gap:  The user's AD 
record is much richer than the Heimdal HDB format allows, 
so we do AD-specific access-control checks in the plugin's 
AD-specific layer, not in the DB-agnostic KDC server.
For Heimdal, we created a separate KDC plugin, with this API:
<ol> 
<li> '''pac_generate()''' creates an initial PAC from the user's AD record.
</li>
<li> '''pac_verify()''' checks that a AC is correctly signed, 
                        adds additional groups (for cross-realm tickets) 
                        and re-signs with the key of the target kerberos 
                        service's account
</li>
<li> '''client_access()''' performs additional access checks, such as 
                           allowedWorkstations and account expiry.
</li>
<ol>
<li> Luke H says, "Implement 
     [[#HBAC_for_the_TGS | '''KRB5_KDB_METHOD_CHECK_POLICY_TGS''']] &
     '''KRB5_KDB_METHOD_CHECK_POLICY_AS'''."
</li>
</ol>
<li> Here are the main definitions, from '''samba4/heimdal/kdc/windc_plugin.h'''
* Luke H says, "[[There is similar stuff in 1.7.]]"
<pre>
     typedef struct 
          hdb_entry_ex { void     *ctx;
                         hdb_entry entry;
                         void    (*free_entry)(krb5_context, struct hdb_entry_ex *);
          } hdb_entry_ex;
</pre>
The void *ctx is a "private pointer," provided by the 
'get' method's hdb_entry_ex retval.  The APIs below use 
the void *ctx so as to find additional information about
the user, not contained in the hdb_entry structure.  
Both the provider and the APIs below understand how to
cast the private void *ctx pointer. 
<pre>
     typedef krb5_error_code
           (*krb5plugin_windc_pac_generate)(void * krb5_context,
          		                  struct hdb_entry_ex *, 
                                                   krb5_pac*);
     typedef krb5_error_code
           (*krb5plugin_windc_pac_verify)(void *  krb5_context,
          	                        const   krb5_principal,
          	                        struct  hdb_entry_ex *,
          	                        struct  hdb_entry_ex *,
          	                                krb5_pac *);
     typedef krb5_error_code
           (*krb5plugin_windc_client_access)(void * krb5_context, 
                                             struct hdb_entry_ex *, 
                                                    KDC_REQ *, 
                                                    krb5_data *);
</pre>
The krb5_data* here is critical, so that samba's KDC can return 
the right NTSTATUS code in the 'error string' returned to the 
client.  Otherwise, the windows client won't get the right error
message to the user (such as 'password expired' etc).  The pure
Kerberos error is not enough.

Luke H says, "Well, the pure Kerberos error seems to work most 
of the time but yes, we should fix that. I think this will 
require a change. Let's try not to break Novell's backend 
(or keep them in the loop) when doing so."

<pre>
     typedef struct
             krb5plugin_windc_ftable { int                            minor_version;
                                       krb5_error_code                (*init)(krb5_context, void **);
                                       void                           (*fini)(void *);
                                       krb5plugin_windc_pac_generate   pac_generate;
                                       krb5plugin_windc_pac_verify     pac_verify;
                                       krb5plugin_windc_client_access  client_access;
             } krb5plugin_windc_ftable;
</pre>
This API has some Heimdal-specific stuff, that'll 
have to change when we port this KDC plugin to MIT krb.
</li>
<li> For example, to register this plugin, use the kdc's standard 
     plugin-system at Samba4's initialisation:
<pre>
        /* first, setup the table of callback pointers */
      	/* Registar WinDC hooks */
        ret = krb5_plugin_register(krb5_context, PLUGIN_TYPE_DATA, 
                                   "windc", &windc_plugin_table);
        /* once registered, the KDC will invoke the callbacks */
        /* while preparing each new ticket (TGT or app-tkt)   */
</pre>
</li>
<li> An alternative way to register the plugin is with a 
     config-file that names a DSO (Dynamically Shared Object).
</li>
----

=== ** Appendix 3:  [[Samba4 stuff that doesn't need to get ported.]] ===
==== Heimdal oddities ====
<ol>
<li> Heimdal is built such that it should be able to serve multiple realms
   at the same time.  This isn't relevant for Samba's use, but it shows
   up in a lot of generalisations throughout the code.
</li>
<li> Samba4's code originally tried internally to make it possible to use
   Heimdal's multi-realms-per-KDC ability, but this was ill-conceived,
   and AB has recently (6/09) ripped the last of that multi-realms
   stuff out of samba4.  AB says that in AD, it's not really possible
   to make this work;  several AD components structurally assume that
   there's one realm per KDC.  However, we do use this to support
   canonicalization of realm-names:  case variations, plus long-vs-short
   variants of realm-names.  No MIT porting task here, as long as MIT kdc
   doesn't refuse to do some LDAP lookups (eg, alias' realm-name looks 
   wrong).
</li>
<li> Heimdal supports multiple passwords on a client account:  Samba4
   seems to call '''hdb_next_enctype2key()''' in the pre-authentication 
   routines, to allow multiple passwords per account in krb5.  
   (I think this was intended to allow multiple salts).  AD doesn't 
   support this, so the MIT port shouldn't bother with this.
</li>
</ol>

==== Not needed anymore, because MIT's code now handles PACs fully ====
<ol>
<li> '''gss_krb5_copy_service_keyblock()''' (get the key used to actually
   encrypt the ticket to the server, because the same key is used for
   the PAC validation).
</li>
<li> '''gsskrb5_extract_authtime_from_sec_context()''' (get authtime from
   kerberos ticket)
</li>
<li> '''gsskrb5_extract_authz_data_from_sec_context()''' (get authdata from
   ticket, ie the PAC.  Must unwrap the data if in an AD-IFRELEVANT)]
</li>
</ol>

==== Authz data extraction ====
Samba4 uses '''krb5_ticket_get_authorization_data_type()''', and expect 
it to return the correct authz data, even if wrapped in an 
AD-IFRELEVANT container.  This doesn't need to be ported to MIT. 
This should be obsoleted by MIT's new PAC code.

==== libkdc ====
<ol>
<li> Samba4 needs to be built as a single binary (design requirement), 
   and this should include the KDC.  Samba also (and perhaps more
   importantly) needs to control the configuration environment of 
   the KDC.  
</li>
<li> But, libkdc doesn't matter for IPA; Samba invokes the Heimdal kdc 
   as a library call, but this is just a convenience, and the MIT 
   port can do otherwise w/o trouble.
</li>
</ol>

==== Returned Salt for PreAuthentication ====

When the AD-KDC replies to pre-authentication, it returns the 
salt, which may be in the form of a principalName that is in no 
way connected with the current names.  (ie, even if the 
userPrincipalName and samAccountName are renamed, the old salt 
is returned).


This is the kerberos standard salt, kept in the 'Key'.  The
AD generation rules are found in a Mail from Luke Howard dated
10 Nov 2004.  The MIT glue layer doesn't really need to care about
these salt-handling details;  the samba4 code & the LDAP backend 
will conspire to make sure that MIT's KDC gets correct salts.

<pre>
   >  From: Luke Howard <lukeh@padl.com>
   >  Organization: PADL Software Pty Ltd
   >  To: lukeh@padl.com
   >  Date: Wed, 10 Nov 2004 13:31:21 +1100
   >  Cc: huaraz@moeller.plus.com, samba-technical@lists.samba.org
   >  Subject: Re: Samba-3.0.7-1.3E Active Directory Issues
   >  -------
   >
   >  Did some more testing, it appears the behaviour has another
   >  explanation. It appears that the standard Kerberos password salt
   >  algorithm is applied in Windows 2003, just that the source principal
   >  name is different.
   >
   >  Here is what I've been able to deduce from creating a bunch of
   >  different accounts:  
   >  [SAM name in this mail means the AD attribute samAccountName .
   >   E.g., jbob for a user and jbcomputer$ for a computer.]
   >
   >  [UPN is the AD userPrincipalName attribute.  For example, jbob@mydomain.com]
   >  Type of account                        Principal for Salting
   >  ========================================================================
   >  Computer Account                host/<SAM-Name-Without-$>.realm@REALM
   >  User Account Without UPN        <SAM-Name>@REALM
   >  User Account With UPN           <LHS-Of-UPN>@REALM
   >
   >  Note that if the computer account's SAM account name does not include
   >  the trailing '$', then the entire SAM account name is used as input to
   >  the salting principal. Setting a UPN for a computer account has no
   >  effect.
   >
   >  It seems to me odd that the RHS of the UPN is not used in the salting
   >  principal. For example, a user with UPN foo@mydomain.com in the realm
   >  MYREALM.COM would have a salt of MYREALM.COMfoo. Perhaps this is to
   >  allow a user's UPN suffix to be changed without changing the salt. And
   >  perhaps using the UPN for salting signifies a move away SAM names and
   >  their associated constraints.
   >
   >  For more information on how UPNs relate to the Kerberos protocol,
   >  see:
   >
   >  http://www.ietf.org/proceedings/01dec/I-D/draft-ietf-krb-wg-kerberos-referrals-02.txt
   >
   >  -- Luke
</pre>