System Suspend and Device Interrupts

Copyright (C) 2014 Intel Corp.
Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>


Suspending and Resuming Device IRQs
-----------------------------------

Device interrupt request lines (IRQs) are generally disabled during system
suspend after the "late" phase of suspending devices (that is, after all of the
->prepare, ->suspend and ->suspend_late callbacks have been executed for all
devices).  That is done by suspend_device_irqs().

The rationale for doing so is that after the "late" phase of device suspend
there is no legitimate reason why any interrupts from suspended devices should
trigger and if any devices have not been suspended properly yet, it is better to
block interrupts from them anyway.  Also, in the past we had problems with
interrupt handlers for shared IRQs that device drivers implementing them were
not prepared for interrupts triggering after their devices had been suspended.
In some cases they would attempt to access, for example, memory address spaces
of suspended devices and cause unpredictable behavior to ensue as a result.
Unfortunately, such problems are very difficult to debug and the introduction
of suspend_device_irqs(), along with the "noirq" phase of device suspend and
resume, was the only practical way to mitigate them.

Device IRQs are re-enabled during system resume, right before the "early" phase
of resuming devices (that is, before starting to execute ->resume_early
callbacks for devices).  The function doing that is resume_device_irqs().


The IRQF_NO_SUSPEND Flag
------------------------

There are interrupts that can legitimately trigger during the entire system
suspend-resume cycle, including the "noirq" phases of suspending and resuming
devices as well as during the time when nonboot CPUs are taken offline and
brought back online.  That applies to timer interrupts in the first place,
but also to IPIs and to some other special-purpose interrupts.

The IRQF_NO_SUSPEND flag is used to indicate that to the IRQ subsystem when
requesting a special-purpose interrupt.  It causes suspend_device_irqs() to
leave the corresponding IRQ enabled so as to allow the interrupt to work as
expected during the suspend-resume cycle, but does not guarantee that the
interrupt will wake the system from a suspended state -- for such cases it is
necessary to use enable_irq_wake().

Note that the IRQF_NO_SUSPEND flag affects the entire IRQ and not just one
user of it.  Thus, if the IRQ is shared, all of the interrupt handlers installed
for it will be executed as usual after suspend_device_irqs(), even if the
IRQF_NO_SUSPEND flag was not passed to request_irq() (or equivalent) by some of
the IRQ's users.  For this reason, using IRQF_NO_SUSPEND and IRQF_SHARED at the
same time should be avoided.


System Wakeup Interrupts, enable_irq_wake() and disable_irq_wake()
------------------------------------------------------------------

System wakeup interrupts generally need to be configured to wake up the system
from sleep states, especially if they are used for different purposes (e.g. as
I/O interrupts) in the working state.

That may involve turning on a special signal handling logic within the platform
(such as an SoC) so that signals from a given line are routed in a different way
during system sleep so as to trigger a system wakeup when needed.  For example,
the platform may include a dedicated interrupt controller used specifically for
handling system wakeup events.  Then, if a given interrupt line is supposed to
wake up the system from sleep sates, the corresponding input of that interrupt
controller needs to be enabled to receive signals from the line in question.
After wakeup, it generally is better to disable that input to prevent the
dedicated controller from triggering interrupts unnecessarily.

The IRQ subsystem provides two helper functions to be used by device drivers for
those purposes.  Namely, enable_irq_wake() turns on the platform's logic for
handling the given IRQ as a system wakeup interrupt line and disable_irq_wake()
turns that logic off.

Calling enable_irq_wake() causes suspend_device_irqs() to treat the given IRQ
in a special way.  Namely, the IRQ remains enabled, by on the first interrupt
it will be disabled, marked as pending and "suspended" so that it will be
re-enabled by resume_device_irqs() during the subsequent system resume.  Also
the PM core is notified about the event which causes the system suspend in
progress to be aborted (that doesn't have to happen immediately, but at one
of the points where the suspend thread looks for pending wakeup events).

This way every interrupt from a wakeup interrupt source will either cause the
system suspend currently in progress to be aborted or wake up the system if
already suspended.  However, after suspend_device_irqs() interrupt handlers are
not executed for system wakeup IRQs.  They are only executed for IRQF_NO_SUSPEND
IRQs at that time, but those IRQs should not be configured for system wakeup
using enable_irq_wake().


Interrupts and Suspend-to-Idle
------------------------------

Suspend-to-idle (also known as the "freeze" sleep state) is a relatively new
system sleep state that works by idling all of the processors and waiting for
interrupts right after the "noirq" phase of suspending devices.

Of course, this means that all of the interrupts with the IRQF_NO_SUSPEND flag
set will bring CPUs out of idle while in that state, but they will not cause the
IRQ subsystem to trigger a system wakeup.

System wakeup interrupts, in turn, will trigger wakeup from suspend-to-idle in
analogy with what they do in the full system suspend case.  The only difference
is that the wakeup from suspend-to-idle is signaled using the usual working
state interrupt delivery mechanisms and doesn't require the platform to use
any special interrupt handling logic for it to work.


IRQF_NO_SUSPEND and enable_irq_wake()
-------------------------------------

There are very few valid reasons to use both enable_irq_wake() and the
IRQF_NO_SUSPEND flag on the same IRQ, and it is never valid to use both for the
same device.

First of all, if the IRQ is not shared, the rules for handling IRQF_NO_SUSPEND
interrupts (interrupt handlers are invoked after suspend_device_irqs()) are
directly at odds with the rules for handling system wakeup interrupts (interrupt
handlers are not invoked after suspend_device_irqs()).

Second, both enable_irq_wake() and IRQF_NO_SUSPEND apply to entire IRQs and not
to individual interrupt handlers, so sharing an IRQ between a system wakeup
interrupt source and an IRQF_NO_SUSPEND interrupt source does not generally
make sense.

In rare cases an IRQ can be shared between a wakeup device driver and an
IRQF_NO_SUSPEND user. In order for this to be safe, the wakeup device driver
must be able to discern spurious IRQs from genuine wakeup events (signalling
the latter to the core with pm_system_wakeup()), must use enable_irq_wake() to
ensure that the IRQ will function as a wakeup source, and must request the IRQ
with IRQF_COND_SUSPEND to tell the core that it meets these requirements. If
these requirements are not met, it is not valid to use IRQF_COND_SUSPEND.