Projects/Hierarchical iprop - Revision history

Ghudson: /* Related problems */

2015-04-09T18:04:42Z

‎Related problems

Ghudson: /* Commits */

2014-08-12T17:56:07Z

‎Commits

Ghudson: /* Commits */

2014-08-12T17:50:30Z

‎Commits

Ghudson at 15:23, 11 July 2014

2014-07-11T15:23:08Z

Ghudson at 18:03, 30 January 2014

2014-01-30T18:03:08Z

Ghudson at 04:19, 28 January 2014

2014-01-28T04:19:23Z

Ghudson: /* Proposed changes */

2014-01-28T04:14:53Z

‎Proposed changes

Ghudson at 04:05, 28 January 2014

2014-01-28T04:05:14Z

Ghudson at 23:33, 27 January 2014

2014-01-27T23:33:21Z

Ghudson: /* Proposed changes */

2014-01-21T15:43:32Z

‎Proposed changes

@@ Line 108: / Line 108: @@
 . If the master ulog is initialized and performs a full resync to the slave before any updates are added to its ulog, the slave has kdb_last_sno 0 and kdb_last_time equal to the time of the master's reinitialization.  As long as the slave and master states match, the master does not repeatedly resync to the slave.  However, the next time an update is added to the master's ulog, the master discards its serial number 0 state and sets both kdb_first_sno and kdb_last_sno to 1.  When it receives the next poll from the slave, it has no way of knowing whether the slave's serial number 0 state precedes its first update or reflects a different full resync, so it performs a second full resync to the slave.  This is correct but inefficient.
-With hierarchical propagation, the same problem occurs after an intermediate slave receives a full resync and then performs a full resync to a downstream slave; the next update it receives from master overwrites its kdb_first_sno and kdb_last_sno values with the serial number of the receives update, causing it to perform a second full resync to the downstream slave.
+With hierarchical propagation, the same problem occurs after an intermediate slave receives a full resync and then performs a full resync to a downstream slave; the next update it receives from master overwrites its kdb_first_sno and kdb_last_sno values with the serial number of the received update, causing it to perform a second full resync to the downstream slave.
 We could try to solve this problem by redefining kdb_first_sno to be the serial number state preceding the first update, rather than the first update itself, so that kadmind can recognize that a slave's state matches the state the master was in just prior to the first update in its ulog.  This change would create a (perhaps harmless) ambiguity during a code upgrade, because a ulog file which had been maintained with older code would have the old definition of kdb_first_sno.  More seriously, this change might make it impossible to reliably detect incompatible ulog size changes when mapping the ulog file, because we would no longer have a timestamp with which to verify the first serial number.  (It might be sufficient to verify the serial number of the first update entry as deduced from kdb_last_sno and kdb_num; more analysis would be required.)

@@ Line 126: / Line 126: @@
 ==Commits==
-* 444ef5fe9ec8d64a5db27b3a8aaf6813dd7ef0e0 Simplify iprop update locking and avoid deadlock
+ef5fe9ec8d64a5db27b3a8aaf6813dd7ef0e0 Simplify iprop update locking and avoid deadlock
-* d1f9aa3737b2b3e62b5c5ed488d6112b7ce8a5ad Factor out ulog serial number status check
+    d1f9aa3737b2b3e62b5c5ed488d6112b7ce8a5ad Factor out ulog serial number status check
-* 71d028f1054deb186807e7c8048218b82b478422 Lock around more ulog operations
+d028f1054deb186807e7c8048218b82b478422 Lock around more ulog operations
-* 6a4a4b7b5e3265e4a811a9fd72c2534e6c5f5fd4 Simplify ulog_map
+a4a4b7b5e3265e4a811a9fd72c2534e6c5f5fd4 Simplify ulog_map
-* 406c83c835a8ce062d798a2ec4eda2eddd088450 Maintain complete ulog on iprop slaves
+c83c835a8ce062d798a2ec4eda2eddd088450 Maintain complete ulog on iprop slaves
-* 2ed8ebf18809af66aeaa2af6984754bdbefff500 Implement kadmind -proponly
+ed8ebf18809af66aeaa2af6984754bdbefff500 Implement kadmind -proponly
-* 90c11ff42008a90a72ee71444b0ad799e38b7ff0 Implement kpropd -A
+c11ff42008a90a72ee71444b0ad799e38b7ff0 Implement kpropd -A
-* cf090890b4219483412ab89b39a60d92515191eb Test iprop slave ulog management
+    cf090890b4219483412ab89b39a60d92515191eb Test iprop slave ulog management
-* e87bba2e8a8c753b761227dda5f2e216a6771db2 Document hierarchical iprop
+    e87bba2e8a8c753b761227dda5f2e216a6771db2 Document hierarchical iprop
 ==Release notes==

← Older revision		Revision as of 17:50, 12 August 2014
Line 125:		Line 125:

	==Commits==		==Commits==
		+
		+	* 444ef5fe9ec8d64a5db27b3a8aaf6813dd7ef0e0 Simplify iprop update locking and avoid deadlock
		+	* d1f9aa3737b2b3e62b5c5ed488d6112b7ce8a5ad Factor out ulog serial number status check
		+	* 71d028f1054deb186807e7c8048218b82b478422 Lock around more ulog operations
		+	* 6a4a4b7b5e3265e4a811a9fd72c2534e6c5f5fd4 Simplify ulog_map
		+	* 406c83c835a8ce062d798a2ec4eda2eddd088450 Maintain complete ulog on iprop slaves
		+	* 2ed8ebf18809af66aeaa2af6984754bdbefff500 Implement kadmind -proponly
		+	* 90c11ff42008a90a72ee71444b0ad799e38b7ff0 Implement kpropd -A
		+	* cf090890b4219483412ab89b39a60d92515191eb Test iprop slave ulog management
		+	* e87bba2e8a8c753b761227dda5f2e216a6771db2 Document hierarchical iprop

	==Release notes==		==Release notes==

← Older revision		Revision as of 15:23, 11 July 2014
Line 1:		Line 1:
−	{{project-~~review~~\|~~2014-02-14~~}}	+	{{project-rel\|1.13}}
−	{{project-target\|1.13}}

	==Description==		==Description==

← Older revision		Revision as of 18:03, 30 January 2014
Line 1:		Line 1:
−	{{project-~~early~~}}	+	{{project-review\|2014-02-14}}
	{{project-target\|1.13}}		{{project-target\|1.13}}

← Older revision		Revision as of 04:19, 28 January 2014
Line 128:		Line 128:

	==Release notes==		==Release notes==
		+
		+	Administrative experience:
		+
		+	* Add support for hierarchical incremental propagation, where slaves can act as intermediates between an upstream master and other downstream slaves.

@@ Line 90: / Line 90: @@
 The code changes will be broken out into the following commits:
-# Add a new interface ulog_get_sno_status, so that current_sno_in_ulog and ulog_get_entries can do the same thing.  In the new function, check if the ulog is empty (using kdb_num) and only accept an exact match against kdb_last_sno/time if it is.  Otherwise, check the presented serial number and timestamp against the actual ulog entry.
+# Add a new interface ulog_get_sno_status, so that current_sno_in_ulog and ulog_get_entries can do the same thing to check how current a serial number and timestamp are.  In the new function, check if the ulog is empty (using kdb_num) and only accept an exact match against kdb_last_sno/time if it is.  Otherwise, check the presented serial number and timestamp against the actual ulog entry.
 # Add new interfaces ulog_get_last and ulog_set_last, and use them in kdb5_util and kpropd to properly lock around accesses to the ulog header.  Make ulog_init_header acquire an exclusive lock.
 # Remove the ulog_map caller parameter, and make it always behave the same way, so that kpropd is prepared to write ulog entries.  Make kproplog map the ulog directly without going through ulog_map (unless -R is given), since its mapping semantics don't have much in common with other callers.

@@ Line 24: / Line 24: @@
 * kdb_block: The entry size (normally 2048)
-The size of the circular entry array is determined by the iprop_master_ulogsize krb5.conf variable, which is not recorded in the header.  The kdb_num, kdb_first_sno, and kdb_last_sno fields are redundant; this redundancy can be used to detect if the ulogsize parameter changed since the last time the ulog was updated.
+The size of the circular entry array is determined by the iprop_master_ulogsize krb5.conf variable, which is not recorded in the header.  The kdb_num, kdb_first_sno, and kdb_last_sno fields are redundant; when kdb_num is not zero, kdb_first_sno must always be equal to kdb_last_sno - kdb_num + 1.
-When the ulog is initialized or reinitialized (with kproplog -R), the kdb_num, kdb_first_time, kdb_first_sno, and kdb_last_sno fields are set to 0, and the kdb_last_time field is set to the time of initialization.  The first database update after ulog initialization increments the kdb_num, kdb_first_sno, and kdb_last_sno fields to 1, and sets the kdb_first_time and kdb_last_time fields to the time of the update.
+When the ulog is initialized or reinitialized, the kdb_num, kdb_first_time, kdb_first_sno, and kdb_last_sno fields are set to 0, and the kdb_last_time field is set to the time of initialization.  The master ulog may be reinitialized because of a policy change (since the iprop protocol cannot transport policy changes), or because the ulog reached the highest possible serial number, or because of administrative action (kproplog -R).  The first database update after ulog initialization increments the kdb_num, kdb_first_sno, and kdb_last_sno fields to 1, and sets the kdb_first_time and kdb_last_time fields to the time of the update.
 The kdb_first_sno and kdb_first_time fields are used by the following code operations on the master KDC:
@@ Line 66: / Line 66: @@
 Three changes are needed to allow hierarchical iprop:
-. On slaves, maintain a complete ulog.  When update entries are received from the upstream master and processed by ulog_replay(), add them to the ulog in addition to modifying the KDB.
+. On slaves, maintain a complete ulog.  When update entries are received from the upstream master and processed by ulog_replay(), add them to the ulog in addition to modifying the KDB.  When the slave receives a full dump from upstream, its ulog is reset and its kdb_last_sno and kdb_last_time fields reflect the information in the dump file.
 . Add a -proponly option to kadmind so that it can be run on slaves as an iprop server.
@@ Line 98: / Line 98: @@
 # Augment the existing automated iprop tests to verify the new functionality.
 # Add documentation on setting up hierarchical iprop.
 ==Testing==

@@ Line 85: / Line 85: @@
 * kdb5_util dump's current_sno_in_ulog() function must check for an empty ulog using kdb_num, not kdb_last_sno, since a slave can have an empty ulog with non-zero kdb_last_sno.  However, if the dump file matches kdb_last_sno and kdb_last_time, the dump should be considered up to date even if the ulog is empty.
 ==Testing==
 ==Documentation==
 ==Mailing list discussions==

@@ Line 82: / Line 82: @@
 * ulog_map can no longer detect from header fields alone that the ulogsize has shrunk in a way that invalidates the entry array (and the current check would give frequent false positives).  For example, if the ulog is initialized with a ulogsize of 1000, then a single update is added from upstream with serial number 1700, and then the ulog is mapped again with a ulogsize of 600, then the header fields will appear exactly the same as if the update had been added when ulogsize was 600, but the update was written to position 699 and will be read from position 499.  Instead, we need to verify the serial numbers of the first and last update against the entry array.
-* ulog_get_entries must explicitly check if the ulog is empty and require a full resync if the downstream slave does not match kdb_last_sno/kdb_last_time.  kdb_first_sno will be zero in this state, so checking against it would erroneously succeed.
+* ulog_get_entries must explicitly check if the ulog is empty and require a full resync if the downstream slave does not match kdb_last_sno/kdb_last_time.  Otherwise, an intermediate slave which has just taken a full dump from upstream will erroneously believe that it has a wide range of serial numbers in its ulog (because kdb_first_sno is 0 and kdb_last_sno is the serial number from the upstream dump).
 * kdb5_util dump's current_sno_in_ulog() function must check for an empty ulog using kdb_num, not kdb_last_sno, since a slave can have an empty ulog with non-zero kdb_last_sno.  However, if the dump file matches kdb_last_sno and kdb_last_time, the dump should be considered up to date even if the ulog is empty.