2d9e1e2f58
The standard ticket spinlocks are very expensive in a virtual environment, because their performance depends on Xen's scheduler giving vcpus time in the order that they're supposed to take the spinlock. This implements a Xen-specific spinlock, which should be much more efficient. The fast-path is essentially the old Linux-x86 locks, using a single lock byte. The locker decrements the byte; if the result is 0, then they have the lock. If the lock is negative, then locker must spin until the lock is positive again. When there's contention, the locker spin for 2^16[*] iterations waiting to get the lock. If it fails to get the lock in that time, it adds itself to the contention count in the lock and blocks on a per-cpu event channel. When unlocking the spinlock, the locker looks to see if there's anyone blocked waiting for the lock by checking for a non-zero waiter count. If there's a waiter, it traverses the per-cpu "lock_spinners" variable, which contains which lock each CPU is waiting on. It picks one CPU waiting on the lock and sends it an event to wake it up. This allows efficient fast-path spinlock operation, while allowing spinning vcpus to give up their processor time while waiting for a contended lock. [*] 2^16 iterations is threshold at which 98% locks have been taken according to Thomas Friebel's Xen Summit talk "Preventing Guests from Spinning Around". Therefore, we'd expect the lock and unlock slow paths will only be entered 2% of the time. Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Cc: Jens Axboe <axboe@kernel.dk> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Christoph Lameter <clameter@linux-foundation.org> Cc: Petr Tesarik <ptesarik@suse.cz> Cc: Virtualization <virtualization@lists.linux-foundation.org> Cc: Xen devel <xen-devel@lists.xensource.com> Cc: Thomas Friebel <thomas.friebel@amd.com> Cc: Nick Piggin <nickpiggin@yahoo.com.au> Signed-off-by: Ingo Molnar <mingo@elte.hu>
25 lines
435 B
C
25 lines
435 B
C
#ifndef __XEN_EVENTS_H
|
|
#define __XEN_EVENTS_H
|
|
|
|
enum ipi_vector {
|
|
XEN_RESCHEDULE_VECTOR,
|
|
XEN_CALL_FUNCTION_VECTOR,
|
|
XEN_CALL_FUNCTION_SINGLE_VECTOR,
|
|
XEN_SPIN_UNLOCK_VECTOR,
|
|
|
|
XEN_NR_IPIS,
|
|
};
|
|
|
|
static inline int xen_irqs_disabled(struct pt_regs *regs)
|
|
{
|
|
return raw_irqs_disabled_flags(regs->flags);
|
|
}
|
|
|
|
static inline void xen_do_IRQ(int irq, struct pt_regs *regs)
|
|
{
|
|
regs->orig_ax = ~irq;
|
|
do_IRQ(regs);
|
|
}
|
|
|
|
#endif /* __XEN_EVENTS_H */
|