linux_dsm_epyc7002/security/keys/gc.c

370 lines
9.9 KiB
C
Raw Normal View History

// SPDX-License-Identifier: GPL-2.0-or-later
/* Key garbage collector
*
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
* Copyright (C) 2009-2011 Red Hat, Inc. All Rights Reserved.
* Written by David Howells (dhowells@redhat.com)
*/
#include <linux/slab.h>
#include <linux/security.h>
#include <keys/keyring-type.h>
#include "internal.h"
/*
* Delay between key revocation/expiry in seconds
*/
unsigned key_gc_delay = 5 * 60;
/*
* Reaper for unused keys.
*/
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
static void key_garbage_collector(struct work_struct *work);
DECLARE_WORK(key_gc_work, key_garbage_collector);
/*
* Reaper for links from keyrings to dead keys.
*/
static void key_gc_timer_func(struct timer_list *);
timer: Remove expires and data arguments from DEFINE_TIMER Drop the arguments from the macro and adjust all callers with the following script: perl -pi -e 's/DEFINE_TIMER\((.*), 0, 0\);/DEFINE_TIMER($1);/g;' \ $(git grep DEFINE_TIMER | cut -d: -f1 | sort -u | grep -v timer.h) Signed-off-by: Kees Cook <keescook@chromium.org> Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> # for m68k parts Acked-by: Guenter Roeck <linux@roeck-us.net> # for watchdog parts Acked-by: David S. Miller <davem@davemloft.net> # for networking parts Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Acked-by: Kalle Valo <kvalo@codeaurora.org> # for wireless parts Acked-by: Arnd Bergmann <arnd@arndb.de> Cc: linux-mips@linux-mips.org Cc: Petr Mladek <pmladek@suse.com> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Lai Jiangshan <jiangshanlai@gmail.com> Cc: Sebastian Reichel <sre@kernel.org> Cc: Kalle Valo <kvalo@qca.qualcomm.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Pavel Machek <pavel@ucw.cz> Cc: linux1394-devel@lists.sourceforge.net Cc: Chris Metcalf <cmetcalf@mellanox.com> Cc: linux-s390@vger.kernel.org Cc: linux-wireless@vger.kernel.org Cc: "James E.J. Bottomley" <jejb@linux.vnet.ibm.com> Cc: Wim Van Sebroeck <wim@iguana.be> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Ursula Braun <ubraun@linux.vnet.ibm.com> Cc: Viresh Kumar <viresh.kumar@linaro.org> Cc: Harish Patil <harish.patil@cavium.com> Cc: Stephen Boyd <sboyd@codeaurora.org> Cc: Michael Reed <mdr@sgi.com> Cc: Manish Chopra <manish.chopra@cavium.com> Cc: Len Brown <len.brown@intel.com> Cc: Arnd Bergmann <arnd@arndb.de> Cc: linux-pm@vger.kernel.org Cc: Heiko Carstens <heiko.carstens@de.ibm.com> Cc: Tejun Heo <tj@kernel.org> Cc: Julian Wiedmann <jwi@linux.vnet.ibm.com> Cc: John Stultz <john.stultz@linaro.org> Cc: Mark Gross <mark.gross@intel.com> Cc: linux-watchdog@vger.kernel.org Cc: linux-scsi@vger.kernel.org Cc: "Martin K. Petersen" <martin.petersen@oracle.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: Stefan Richter <stefanr@s5r6.in-berlin.de> Cc: Guenter Roeck <linux@roeck-us.net> Cc: netdev@vger.kernel.org Cc: Martin Schwidefsky <schwidefsky@de.ibm.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: linuxppc-dev@lists.ozlabs.org Cc: Sudip Mukherjee <sudipm.mukherjee@gmail.com> Link: https://lkml.kernel.org/r/1507159627-127660-11-git-send-email-keescook@chromium.org Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2017-10-05 06:27:04 +07:00
static DEFINE_TIMER(key_gc_timer, key_gc_timer_func);
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
static time64_t key_gc_next_run = TIME64_MAX;
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
static struct key_type *key_gc_dead_keytype;
static unsigned long key_gc_flags;
#define KEY_GC_KEY_EXPIRED 0 /* A key expired and needs unlinking */
#define KEY_GC_REAP_KEYTYPE 1 /* A keytype is being unregistered */
#define KEY_GC_REAPING_KEYTYPE 2 /* Cleared when keytype reaped */
/*
* Any key whose type gets unregistered will be re-typed to this if it can't be
* immediately unlinked.
*/
struct key_type key_type_dead = {
.name = ".dead",
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
};
/*
* Schedule a garbage collection run.
* - time precision isn't particularly important
*/
void key_schedule_gc(time64_t gc_at)
{
unsigned long expires;
time64_t now = ktime_get_real_seconds();
kenter("%lld", gc_at - now);
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
if (gc_at <= now || test_bit(KEY_GC_REAP_KEYTYPE, &key_gc_flags)) {
kdebug("IMMEDIATE");
schedule_work(&key_gc_work);
} else if (gc_at < key_gc_next_run) {
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
kdebug("DEFERRED");
key_gc_next_run = gc_at;
expires = jiffies + (gc_at - now) * HZ;
mod_timer(&key_gc_timer, expires);
}
}
/*
* Schedule a dead links collection run.
*/
void key_schedule_gc_links(void)
{
set_bit(KEY_GC_KEY_EXPIRED, &key_gc_flags);
schedule_work(&key_gc_work);
}
/*
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
* Some key's cleanup time was met after it expired, so we need to get the
* reaper to go through a cycle finding expired keys.
*/
static void key_gc_timer_func(struct timer_list *unused)
{
kenter("");
key_gc_next_run = TIME64_MAX;
key_schedule_gc_links();
}
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
/*
* Reap keys of dead type.
*
* We use three flags to make sure we see three complete cycles of the garbage
* collector: the first to mark keys of that type as being dead, the second to
* collect dead links and the third to clean up the dead keys. We have to be
* careful as there may already be a cycle in progress.
*
* The caller must be holding key_types_sem.
*/
void key_gc_keytype(struct key_type *ktype)
{
kenter("%s", ktype->name);
key_gc_dead_keytype = ktype;
set_bit(KEY_GC_REAPING_KEYTYPE, &key_gc_flags);
smp_mb();
set_bit(KEY_GC_REAP_KEYTYPE, &key_gc_flags);
kdebug("schedule");
schedule_work(&key_gc_work);
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
kdebug("sleep");
sched: Remove proliferation of wait_on_bit() action functions The current "wait_on_bit" interface requires an 'action' function to be provided which does the actual waiting. There are over 20 such functions, many of them identical. Most cases can be satisfied by one of just two functions, one which uses io_schedule() and one which just uses schedule(). So: Rename wait_on_bit and wait_on_bit_lock to wait_on_bit_action and wait_on_bit_lock_action to make it explicit that they need an action function. Introduce new wait_on_bit{,_lock} and wait_on_bit{,_lock}_io which are *not* given an action function but implicitly use a standard one. The decision to error-out if a signal is pending is now made based on the 'mode' argument rather than being encoded in the action function. All instances of the old wait_on_bit and wait_on_bit_lock which can use the new version have been changed accordingly and their action functions have been discarded. wait_on_bit{_lock} does not return any specific error code in the event of a signal so the caller must check for non-zero and interpolate their own error code as appropriate. The wait_on_bit() call in __fscache_wait_on_invalidate() was ambiguous as it specified TASK_UNINTERRUPTIBLE but used fscache_wait_bit_interruptible as an action function. David Howells confirms this should be uniformly "uninterruptible" The main remaining user of wait_on_bit{,_lock}_action is NFS which needs to use a freezer-aware schedule() call. A comment in fs/gfs2/glock.c notes that having multiple 'action' functions is useful as they display differently in the 'wchan' field of 'ps'. (and /proc/$PID/wchan). As the new bit_wait{,_io} functions are tagged "__sched", they will not show up at all, but something higher in the stack. So the distinction will still be visible, only with different function names (gds2_glock_wait versus gfs2_glock_dq_wait in the gfs2/glock.c case). Since first version of this patch (against 3.15) two new action functions appeared, on in NFS and one in CIFS. CIFS also now uses an action function that makes the same freezer aware schedule call as NFS. Signed-off-by: NeilBrown <neilb@suse.de> Acked-by: David Howells <dhowells@redhat.com> (fscache, keys) Acked-by: Steven Whitehouse <swhiteho@redhat.com> (gfs2) Acked-by: Peter Zijlstra <peterz@infradead.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Steve French <sfrench@samba.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: http://lkml.kernel.org/r/20140707051603.28027.72349.stgit@notabene.brown Signed-off-by: Ingo Molnar <mingo@kernel.org>
2014-07-07 12:16:04 +07:00
wait_on_bit(&key_gc_flags, KEY_GC_REAPING_KEYTYPE,
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
TASK_UNINTERRUPTIBLE);
key_gc_dead_keytype = NULL;
kleave("");
}
/*
* Garbage collect a list of unreferenced, detached keys
*/
static noinline void key_gc_unused_keys(struct list_head *keys)
{
while (!list_empty(keys)) {
struct key *key =
list_entry(keys->next, struct key, graveyard_link);
KEYS: Fix race between updating and finding a negative key Consolidate KEY_FLAG_INSTANTIATED, KEY_FLAG_NEGATIVE and the rejection error into one field such that: (1) The instantiation state can be modified/read atomically. (2) The error can be accessed atomically with the state. (3) The error isn't stored unioned with the payload pointers. This deals with the problem that the state is spread over three different objects (two bits and a separate variable) and reading or updating them atomically isn't practical, given that not only can uninstantiated keys change into instantiated or rejected keys, but rejected keys can also turn into instantiated keys - and someone accessing the key might not be using any locking. The main side effect of this problem is that what was held in the payload may change, depending on the state. For instance, you might observe the key to be in the rejected state. You then read the cached error, but if the key semaphore wasn't locked, the key might've become instantiated between the two reads - and you might now have something in hand that isn't actually an error code. The state is now KEY_IS_UNINSTANTIATED, KEY_IS_POSITIVE or a negative error code if the key is negatively instantiated. The key_is_instantiated() function is replaced with key_is_positive() to avoid confusion as negative keys are also 'instantiated'. Additionally, barriering is included: (1) Order payload-set before state-set during instantiation. (2) Order state-read before payload-read when using the key. Further separate barriering is necessary if RCU is being used to access the payload content after reading the payload pointers. Fixes: 146aa8b1453b ("KEYS: Merge the type-specific data with the payload data") Cc: stable@vger.kernel.org # v4.4+ Reported-by: Eric Biggers <ebiggers@google.com> Signed-off-by: David Howells <dhowells@redhat.com> Reviewed-by: Eric Biggers <ebiggers@google.com>
2017-10-04 22:43:25 +07:00
short state = key->state;
list_del(&key->graveyard_link);
kdebug("- %u", key->serial);
key_check(key);
watch_queue: Add a key/keyring notification facility Add a key/keyring change notification facility whereby notifications about changes in key and keyring content and attributes can be received. Firstly, an event queue needs to be created: pipe2(fds, O_NOTIFICATION_PIPE); ioctl(fds[1], IOC_WATCH_QUEUE_SET_SIZE, 256); then a notification can be set up to report notifications via that queue: struct watch_notification_filter filter = { .nr_filters = 1, .filters = { [0] = { .type = WATCH_TYPE_KEY_NOTIFY, .subtype_filter[0] = UINT_MAX, }, }, }; ioctl(fds[1], IOC_WATCH_QUEUE_SET_FILTER, &filter); keyctl_watch_key(KEY_SPEC_SESSION_KEYRING, fds[1], 0x01); After that, records will be placed into the queue when events occur in which keys are changed in some way. Records are of the following format: struct key_notification { struct watch_notification watch; __u32 key_id; __u32 aux; } *n; Where: n->watch.type will be WATCH_TYPE_KEY_NOTIFY. n->watch.subtype will indicate the type of event, such as NOTIFY_KEY_REVOKED. n->watch.info & WATCH_INFO_LENGTH will indicate the length of the record. n->watch.info & WATCH_INFO_ID will be the second argument to keyctl_watch_key(), shifted. n->key will be the ID of the affected key. n->aux will hold subtype-dependent information, such as the key being linked into the keyring specified by n->key in the case of NOTIFY_KEY_LINKED. Note that it is permissible for event records to be of variable length - or, at least, the length may be dependent on the subtype. Note also that the queue can be shared between multiple notifications of various types. Signed-off-by: David Howells <dhowells@redhat.com> Reviewed-by: James Morris <jamorris@linux.microsoft.com>
2020-01-15 00:07:11 +07:00
#ifdef CONFIG_KEY_NOTIFICATIONS
remove_watch_list(key->watchers, key->serial);
key->watchers = NULL;
#endif
KEYS: Fix crash when attempt to garbage collect an uninstantiated keyring The following sequence of commands: i=`keyctl add user a a @s` keyctl request2 keyring foo bar @t keyctl unlink $i @s tries to invoke an upcall to instantiate a keyring if one doesn't already exist by that name within the user's keyring set. However, if the upcall fails, the code sets keyring->type_data.reject_error to -ENOKEY or some other error code. When the key is garbage collected, the key destroy function is called unconditionally and keyring_destroy() uses list_empty() on keyring->type_data.link - which is in a union with reject_error. Subsequently, the kernel tries to unlink the keyring from the keyring names list - which oopses like this: BUG: unable to handle kernel paging request at 00000000ffffff8a IP: [<ffffffff8126e051>] keyring_destroy+0x3d/0x88 ... Workqueue: events key_garbage_collector ... RIP: 0010:[<ffffffff8126e051>] keyring_destroy+0x3d/0x88 RSP: 0018:ffff88003e2f3d30 EFLAGS: 00010203 RAX: 00000000ffffff82 RBX: ffff88003bf1a900 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 000000003bfc6901 RDI: ffffffff81a73a40 RBP: ffff88003e2f3d38 R08: 0000000000000152 R09: 0000000000000000 R10: ffff88003e2f3c18 R11: 000000000000865b R12: ffff88003bf1a900 R13: 0000000000000000 R14: ffff88003bf1a908 R15: ffff88003e2f4000 ... CR2: 00000000ffffff8a CR3: 000000003e3ec000 CR4: 00000000000006f0 ... Call Trace: [<ffffffff8126c756>] key_gc_unused_keys.constprop.1+0x5d/0x10f [<ffffffff8126ca71>] key_garbage_collector+0x1fa/0x351 [<ffffffff8105ec9b>] process_one_work+0x28e/0x547 [<ffffffff8105fd17>] worker_thread+0x26e/0x361 [<ffffffff8105faa9>] ? rescuer_thread+0x2a8/0x2a8 [<ffffffff810648ad>] kthread+0xf3/0xfb [<ffffffff810647ba>] ? kthread_create_on_node+0x1c2/0x1c2 [<ffffffff815f2ccf>] ret_from_fork+0x3f/0x70 [<ffffffff810647ba>] ? kthread_create_on_node+0x1c2/0x1c2 Note the value in RAX. This is a 32-bit representation of -ENOKEY. The solution is to only call ->destroy() if the key was successfully instantiated. Reported-by: Dmitry Vyukov <dvyukov@google.com> Signed-off-by: David Howells <dhowells@redhat.com> Tested-by: Dmitry Vyukov <dvyukov@google.com>
2015-10-15 23:21:37 +07:00
/* Throw away the key data if the key is instantiated */
KEYS: Fix race between updating and finding a negative key Consolidate KEY_FLAG_INSTANTIATED, KEY_FLAG_NEGATIVE and the rejection error into one field such that: (1) The instantiation state can be modified/read atomically. (2) The error can be accessed atomically with the state. (3) The error isn't stored unioned with the payload pointers. This deals with the problem that the state is spread over three different objects (two bits and a separate variable) and reading or updating them atomically isn't practical, given that not only can uninstantiated keys change into instantiated or rejected keys, but rejected keys can also turn into instantiated keys - and someone accessing the key might not be using any locking. The main side effect of this problem is that what was held in the payload may change, depending on the state. For instance, you might observe the key to be in the rejected state. You then read the cached error, but if the key semaphore wasn't locked, the key might've become instantiated between the two reads - and you might now have something in hand that isn't actually an error code. The state is now KEY_IS_UNINSTANTIATED, KEY_IS_POSITIVE or a negative error code if the key is negatively instantiated. The key_is_instantiated() function is replaced with key_is_positive() to avoid confusion as negative keys are also 'instantiated'. Additionally, barriering is included: (1) Order payload-set before state-set during instantiation. (2) Order state-read before payload-read when using the key. Further separate barriering is necessary if RCU is being used to access the payload content after reading the payload pointers. Fixes: 146aa8b1453b ("KEYS: Merge the type-specific data with the payload data") Cc: stable@vger.kernel.org # v4.4+ Reported-by: Eric Biggers <ebiggers@google.com> Signed-off-by: David Howells <dhowells@redhat.com> Reviewed-by: Eric Biggers <ebiggers@google.com>
2017-10-04 22:43:25 +07:00
if (state == KEY_IS_POSITIVE && key->type->destroy)
key->type->destroy(key);
security_key_free(key);
/* deal with the user's key tracking and quota */
if (test_bit(KEY_FLAG_IN_QUOTA, &key->flags)) {
spin_lock(&key->user->lock);
key->user->qnkeys--;
key->user->qnbytes -= key->quotalen;
spin_unlock(&key->user->lock);
}
atomic_dec(&key->user->nkeys);
KEYS: Fix race between updating and finding a negative key Consolidate KEY_FLAG_INSTANTIATED, KEY_FLAG_NEGATIVE and the rejection error into one field such that: (1) The instantiation state can be modified/read atomically. (2) The error can be accessed atomically with the state. (3) The error isn't stored unioned with the payload pointers. This deals with the problem that the state is spread over three different objects (two bits and a separate variable) and reading or updating them atomically isn't practical, given that not only can uninstantiated keys change into instantiated or rejected keys, but rejected keys can also turn into instantiated keys - and someone accessing the key might not be using any locking. The main side effect of this problem is that what was held in the payload may change, depending on the state. For instance, you might observe the key to be in the rejected state. You then read the cached error, but if the key semaphore wasn't locked, the key might've become instantiated between the two reads - and you might now have something in hand that isn't actually an error code. The state is now KEY_IS_UNINSTANTIATED, KEY_IS_POSITIVE or a negative error code if the key is negatively instantiated. The key_is_instantiated() function is replaced with key_is_positive() to avoid confusion as negative keys are also 'instantiated'. Additionally, barriering is included: (1) Order payload-set before state-set during instantiation. (2) Order state-read before payload-read when using the key. Further separate barriering is necessary if RCU is being used to access the payload content after reading the payload pointers. Fixes: 146aa8b1453b ("KEYS: Merge the type-specific data with the payload data") Cc: stable@vger.kernel.org # v4.4+ Reported-by: Eric Biggers <ebiggers@google.com> Signed-off-by: David Howells <dhowells@redhat.com> Reviewed-by: Eric Biggers <ebiggers@google.com>
2017-10-04 22:43:25 +07:00
if (state != KEY_IS_UNINSTANTIATED)
atomic_dec(&key->user->nikeys);
key_user_put(key->user);
key_put_tag(key->domain_tag);
kfree(key->description);
memzero_explicit(key, sizeof(*key));
kmem_cache_free(key_jar, key);
}
}
/*
* Garbage collector for unused keys.
*
* This is done in process context so that we don't have to disable interrupts
* all over the place. key_put() schedules this rather than trying to do the
* cleanup itself, which means key_put() doesn't have to sleep.
*/
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
static void key_garbage_collector(struct work_struct *work)
{
static LIST_HEAD(graveyard);
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
static u8 gc_state; /* Internal persistent state */
#define KEY_GC_REAP_AGAIN 0x01 /* - Need another cycle */
#define KEY_GC_REAPING_LINKS 0x02 /* - We need to reap links */
#define KEY_GC_SET_TIMER 0x04 /* - We need to restart the timer */
#define KEY_GC_REAPING_DEAD_1 0x10 /* - We need to mark dead keys */
#define KEY_GC_REAPING_DEAD_2 0x20 /* - We need to reap dead key links */
#define KEY_GC_REAPING_DEAD_3 0x40 /* - We need to reap dead keys */
#define KEY_GC_FOUND_DEAD_KEY 0x80 /* - We found at least one dead key */
struct rb_node *cursor;
struct key *key;
time64_t new_timer, limit;
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
kenter("[%lx,%x]", key_gc_flags, gc_state);
limit = ktime_get_real_seconds();
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
if (limit > key_gc_delay)
limit -= key_gc_delay;
else
limit = key_gc_delay;
/* Work out what we're going to be doing in this pass */
gc_state &= KEY_GC_REAPING_DEAD_1 | KEY_GC_REAPING_DEAD_2;
gc_state <<= 1;
if (test_and_clear_bit(KEY_GC_KEY_EXPIRED, &key_gc_flags))
gc_state |= KEY_GC_REAPING_LINKS | KEY_GC_SET_TIMER;
if (test_and_clear_bit(KEY_GC_REAP_KEYTYPE, &key_gc_flags))
gc_state |= KEY_GC_REAPING_DEAD_1;
kdebug("new pass %x", gc_state);
new_timer = TIME64_MAX;
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
/* As only this function is permitted to remove things from the key
* serial tree, if cursor is non-NULL then it will always point to a
* valid node in the tree - even if lock got dropped.
*/
spin_lock(&key_serial_lock);
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
cursor = rb_first(&key_serial_tree);
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
continue_scanning:
while (cursor) {
key = rb_entry(cursor, struct key, serial_node);
cursor = rb_next(cursor);
if (refcount_read(&key->usage) == 0)
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
goto found_unreferenced_key;
if (unlikely(gc_state & KEY_GC_REAPING_DEAD_1)) {
if (key->type == key_gc_dead_keytype) {
gc_state |= KEY_GC_FOUND_DEAD_KEY;
set_bit(KEY_FLAG_DEAD, &key->flags);
key->perm = 0;
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
goto skip_dead_key;
} else if (key->type == &key_type_keyring &&
key->restrict_link) {
goto found_restricted_keyring;
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
}
}
if (gc_state & KEY_GC_SET_TIMER) {
if (key->expiry > limit && key->expiry < new_timer) {
kdebug("will expire %x in %lld",
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
key_serial(key), key->expiry - limit);
new_timer = key->expiry;
}
}
if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2))
if (key->type == key_gc_dead_keytype)
gc_state |= KEY_GC_FOUND_DEAD_KEY;
if ((gc_state & KEY_GC_REAPING_LINKS) ||
unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) {
if (key->type == &key_type_keyring)
goto found_keyring;
}
if (unlikely(gc_state & KEY_GC_REAPING_DEAD_3))
if (key->type == key_gc_dead_keytype)
goto destroy_dead_key;
skip_dead_key:
if (spin_is_contended(&key_serial_lock) || need_resched())
goto contended;
}
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
contended:
spin_unlock(&key_serial_lock);
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
maybe_resched:
if (cursor) {
cond_resched();
spin_lock(&key_serial_lock);
goto continue_scanning;
}
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
/* We've completed the pass. Set the timer if we need to and queue a
* new cycle if necessary. We keep executing cycles until we find one
* where we didn't reap any keys.
*/
kdebug("pass complete");
if (gc_state & KEY_GC_SET_TIMER && new_timer != (time64_t)TIME64_MAX) {
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
new_timer += key_gc_delay;
key_schedule_gc(new_timer);
}
if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2) ||
!list_empty(&graveyard)) {
/* Make sure that all pending keyring payload destructions are
* fulfilled and that people aren't now looking at dead or
* dying keys that they don't have a reference upon or a link
* to.
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
*/
kdebug("gc sync");
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
synchronize_rcu();
}
if (!list_empty(&graveyard)) {
kdebug("gc keys");
key_gc_unused_keys(&graveyard);
}
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
if (unlikely(gc_state & (KEY_GC_REAPING_DEAD_1 |
KEY_GC_REAPING_DEAD_2))) {
if (!(gc_state & KEY_GC_FOUND_DEAD_KEY)) {
/* No remaining dead keys: short circuit the remaining
* keytype reap cycles.
*/
kdebug("dead short");
gc_state &= ~(KEY_GC_REAPING_DEAD_1 | KEY_GC_REAPING_DEAD_2);
gc_state |= KEY_GC_REAPING_DEAD_3;
} else {
gc_state |= KEY_GC_REAP_AGAIN;
}
}
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
if (unlikely(gc_state & KEY_GC_REAPING_DEAD_3)) {
kdebug("dead wake");
smp_mb();
clear_bit(KEY_GC_REAPING_KEYTYPE, &key_gc_flags);
wake_up_bit(&key_gc_flags, KEY_GC_REAPING_KEYTYPE);
}
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
if (gc_state & KEY_GC_REAP_AGAIN)
schedule_work(&key_gc_work);
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
kleave(" [end %x]", gc_state);
return;
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
/* We found an unreferenced key - once we've removed it from the tree,
* we can safely drop the lock.
*/
found_unreferenced_key:
kdebug("unrefd key %d", key->serial);
rb_erase(&key->serial_node, &key_serial_tree);
spin_unlock(&key_serial_lock);
list_add_tail(&key->graveyard_link, &graveyard);
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
gc_state |= KEY_GC_REAP_AGAIN;
goto maybe_resched;
/* We found a restricted keyring and need to update the restriction if
* it is associated with the dead key type.
*/
found_restricted_keyring:
spin_unlock(&key_serial_lock);
keyring_restriction_gc(key, key_gc_dead_keytype);
goto maybe_resched;
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
/* We found a keyring and we need to check the payload for links to
* dead or expired keys. We don't flag another reap immediately as we
* have to wait for the old payload to be destroyed by RCU before we
* can reap the keys to which it refers.
*/
found_keyring:
spin_unlock(&key_serial_lock);
keyring_gc(key, limit);
KEYS: Correctly destroy key payloads when their keytype is removed unregister_key_type() has code to mark a key as dead and make it unavailable in one loop and then destroy all those unavailable key payloads in the next loop. However, the loop to mark keys dead renders the key undetectable to the second loop by changing the key type pointer also. Fix this by the following means: (1) The key code has two garbage collectors: one deletes unreferenced keys and the other alters keyrings to delete links to old dead, revoked and expired keys. They can end up holding each other up as both want to scan the key serial tree under spinlock. Combine these into a single routine. (2) Move the dead key marking, dead link removal and dead key removal into the garbage collector as a three phase process running over the three cycles of the normal garbage collection procedure. This is tracked by the KEY_GC_REAPING_DEAD_1, _2 and _3 state flags. unregister_key_type() then just unlinks the key type from the list, wakes up the garbage collector and waits for the third phase to complete. (3) Downgrade the key types sem in unregister_key_type() once it has deleted the key type from the list so that it doesn't block the keyctl() syscall. (4) Dead keys that cannot be simply removed in the third phase have their payloads destroyed with the key's semaphore write-locked to prevent interference by the keyctl() syscall. There should be no in-kernel users of dead keys of that type by the point of unregistration, though keyctl() may be holding a reference. (5) Only perform timer recalculation in the GC if the timer actually expired. If it didn't, we'll get another cycle when it goes off - and if the key that actually triggered it has been removed, it's not a problem. (6) Only garbage collect link if the timer expired or if we're doing dead key clean up phase 2. (7) As only key_garbage_collector() is permitted to use rb_erase() on the key serial tree, it doesn't need to revalidate its cursor after dropping the spinlock as the node the cursor points to must still exist in the tree. (8) Drop the spinlock in the GC if there is contention on it or if we need to reschedule. After dealing with that, get the spinlock again and resume scanning. This has been tested in the following ways: (1) Run the keyutils testsuite against it. (2) Using the AF_RXRPC and RxKAD modules to test keytype removal: Load the rxrpc_s key type: # insmod /tmp/af-rxrpc.ko # insmod /tmp/rxkad.ko Create a key (http://people.redhat.com/~dhowells/rxrpc/listen.c): # /tmp/listen & [1] 8173 Find the key: # grep rxrpc_s /proc/keys 091086e1 I--Q-- 1 perm 39390000 0 0 rxrpc_s 52:2 Link it to a session keyring, preferably one with a higher serial number: # keyctl link 0x20e36251 @s Kill the process (the key should remain as it's linked to another place): # fg /tmp/listen ^C Remove the key type: rmmod rxkad rmmod af-rxrpc This can be made a more effective test by altering the following part of the patch: if (unlikely(gc_state & KEY_GC_REAPING_DEAD_2)) { /* Make sure everyone revalidates their keys if we marked a * bunch as being dead and make sure all keyring ex-payloads * are destroyed. */ kdebug("dead sync"); synchronize_rcu(); To call synchronize_rcu() in GC phase 1 instead. That causes that the keyring's old payload content to hang around longer until it's RCU destroyed - which usually happens after GC phase 3 is complete. This allows the destroy_dead_key branch to be tested. Reported-by: Benjamin Coddington <bcodding@gmail.com> Signed-off-by: David Howells <dhowells@redhat.com> Signed-off-by: James Morris <jmorris@namei.org>
2011-08-22 20:09:36 +07:00
goto maybe_resched;
/* We found a dead key that is still referenced. Reset its type and
* destroy its payload with its semaphore held.
*/
destroy_dead_key:
spin_unlock(&key_serial_lock);
kdebug("destroy key %d", key->serial);
down_write(&key->sem);
key->type = &key_type_dead;
if (key_gc_dead_keytype->destroy)
key_gc_dead_keytype->destroy(key);
memset(&key->payload, KEY_DESTROY, sizeof(key->payload));
up_write(&key->sem);
goto maybe_resched;
}