2019-04-24 20:07:14 +00:00
|
|
|
/*
|
|
|
|
|
* SPDX-License-Identifier: MIT
|
|
|
|
|
*
|
|
|
|
|
* Copyright © 2019 Intel Corporation
|
|
|
|
|
*/
|
|
|
|
|
|
2019-05-28 09:29:49 +00:00
|
|
|
#include "gem/i915_gem_pm.h"
|
2019-07-12 19:29:53 +00:00
|
|
|
#include "gt/intel_gt.h"
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
#include "gt/intel_gt_pm.h"
|
2019-10-04 13:40:06 +00:00
|
|
|
#include "gt/intel_gt_requests.h"
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
|
2019-04-24 20:07:14 +00:00
|
|
|
#include "i915_drv.h"
|
|
|
|
|
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
static int pm_notifier(struct notifier_block *nb,
|
|
|
|
|
unsigned long action,
|
|
|
|
|
void *data)
|
2019-04-24 20:07:14 +00:00
|
|
|
{
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
struct drm_i915_private *i915 =
|
|
|
|
|
container_of(nb, typeof(*i915), gem.pm_notifier);
|
2019-04-24 20:07:14 +00:00
|
|
|
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
switch (action) {
|
|
|
|
|
case INTEL_GT_UNPARK:
|
|
|
|
|
break;
|
2019-04-24 20:07:14 +00:00
|
|
|
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
case INTEL_GT_PARK:
|
2019-10-04 13:40:07 +00:00
|
|
|
i915_vma_parked(i915);
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
break;
|
|
|
|
|
}
|
2019-04-24 20:07:14 +00:00
|
|
|
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
return NOTIFY_OK;
|
2019-04-24 20:07:14 +00:00
|
|
|
}
|
|
|
|
|
|
2019-07-12 19:29:53 +00:00
|
|
|
static bool switch_to_kernel_context_sync(struct intel_gt *gt)
|
2019-04-24 20:07:14 +00:00
|
|
|
{
|
2019-07-12 19:29:53 +00:00
|
|
|
bool result = !intel_gt_is_wedged(gt);
|
2019-04-24 20:07:14 +00:00
|
|
|
|
2019-10-04 13:40:06 +00:00
|
|
|
if (intel_gt_wait_for_idle(gt, I915_GEM_IDLE_TIMEOUT) == -ETIME) {
|
2019-10-04 13:40:02 +00:00
|
|
|
/* XXX hide warning from gem_eio */
|
|
|
|
|
if (i915_modparams.reset) {
|
|
|
|
|
dev_err(gt->i915->drm.dev,
|
|
|
|
|
"Failed to idle engines, declaring wedged!\n");
|
|
|
|
|
GEM_TRACE_DUMP();
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
}
|
2019-10-04 13:40:02 +00:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Forcibly cancel outstanding work and leave
|
|
|
|
|
* the gpu quiet.
|
|
|
|
|
*/
|
|
|
|
|
intel_gt_set_wedged(gt);
|
|
|
|
|
result = false;
|
|
|
|
|
}
|
2019-04-24 20:07:14 +00:00
|
|
|
|
2019-08-08 20:27:58 +00:00
|
|
|
if (intel_gt_pm_wait_for_idle(gt))
|
|
|
|
|
result = false;
|
|
|
|
|
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
return result;
|
2019-04-24 20:07:14 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
bool i915_gem_load_power_context(struct drm_i915_private *i915)
|
|
|
|
|
{
|
2019-07-12 19:29:53 +00:00
|
|
|
return switch_to_kernel_context_sync(&i915->gt);
|
2019-04-24 20:07:14 +00:00
|
|
|
}
|
|
|
|
|
|
2019-09-12 12:48:13 +00:00
|
|
|
static void user_forcewake(struct intel_gt *gt, bool suspend)
|
|
|
|
|
{
|
|
|
|
|
int count = atomic_read(>->user_wakeref);
|
|
|
|
|
|
|
|
|
|
/* Inside suspend/resume so single threaded, no races to worry about. */
|
|
|
|
|
if (likely(!count))
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
intel_gt_pm_get(gt);
|
|
|
|
|
if (suspend) {
|
|
|
|
|
GEM_BUG_ON(count > atomic_read(>->wakeref.count));
|
|
|
|
|
atomic_sub(count, >->wakeref.count);
|
|
|
|
|
} else {
|
|
|
|
|
atomic_add(count, >->wakeref.count);
|
|
|
|
|
}
|
|
|
|
|
intel_gt_pm_put(gt);
|
|
|
|
|
}
|
|
|
|
|
|
2019-04-24 20:07:14 +00:00
|
|
|
void i915_gem_suspend(struct drm_i915_private *i915)
|
|
|
|
|
{
|
|
|
|
|
GEM_TRACE("\n");
|
|
|
|
|
|
2019-06-13 07:32:54 +00:00
|
|
|
intel_wakeref_auto(&i915->ggtt.userfault_wakeref, 0);
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
flush_workqueue(i915->wq);
|
2019-04-24 20:07:14 +00:00
|
|
|
|
2019-09-12 12:48:13 +00:00
|
|
|
user_forcewake(&i915->gt, true);
|
|
|
|
|
|
2019-04-24 20:07:14 +00:00
|
|
|
/*
|
|
|
|
|
* We have to flush all the executing contexts to main memory so
|
|
|
|
|
* that they can saved in the hibernation image. To ensure the last
|
|
|
|
|
* context image is coherent, we have to switch away from it. That
|
|
|
|
|
* leaves the i915->kernel_context still active when
|
|
|
|
|
* we actually suspend, and its image in memory may not match the GPU
|
|
|
|
|
* state. Fortunately, the kernel_context is disposable and we do
|
|
|
|
|
* not rely on its state.
|
|
|
|
|
*/
|
2019-10-04 13:40:06 +00:00
|
|
|
intel_gt_suspend(&i915->gt);
|
|
|
|
|
intel_uc_suspend(&i915->gt.uc);
|
2019-04-24 20:07:14 +00:00
|
|
|
|
2019-07-12 19:29:53 +00:00
|
|
|
cancel_delayed_work_sync(&i915->gt.hangcheck.work);
|
2019-04-24 20:07:14 +00:00
|
|
|
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
i915_gem_drain_freed_objects(i915);
|
2019-04-24 20:07:14 +00:00
|
|
|
}
|
|
|
|
|
|
2019-06-12 10:57:20 +00:00
|
|
|
static struct drm_i915_gem_object *first_mm_object(struct list_head *list)
|
|
|
|
|
{
|
|
|
|
|
return list_first_entry_or_null(list,
|
|
|
|
|
struct drm_i915_gem_object,
|
|
|
|
|
mm.link);
|
|
|
|
|
}
|
|
|
|
|
|
2019-04-24 20:07:14 +00:00
|
|
|
void i915_gem_suspend_late(struct drm_i915_private *i915)
|
|
|
|
|
{
|
|
|
|
|
struct drm_i915_gem_object *obj;
|
|
|
|
|
struct list_head *phases[] = {
|
2019-06-12 10:57:20 +00:00
|
|
|
&i915->mm.shrink_list,
|
2019-05-30 20:34:59 +00:00
|
|
|
&i915->mm.purge_list,
|
2019-04-24 20:07:14 +00:00
|
|
|
NULL
|
|
|
|
|
}, **phase;
|
2019-06-12 10:57:20 +00:00
|
|
|
unsigned long flags;
|
2019-04-24 20:07:14 +00:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Neither the BIOS, ourselves or any other kernel
|
|
|
|
|
* expects the system to be in execlists mode on startup,
|
|
|
|
|
* so we need to reset the GPU back to legacy mode. And the only
|
|
|
|
|
* known way to disable logical contexts is through a GPU reset.
|
|
|
|
|
*
|
|
|
|
|
* So in order to leave the system in a known default configuration,
|
|
|
|
|
* always reset the GPU upon unload and suspend. Afterwards we then
|
|
|
|
|
* clean up the GEM state tracking, flushing off the requests and
|
|
|
|
|
* leaving the system in a known idle state.
|
|
|
|
|
*
|
|
|
|
|
* Note that is of the upmost importance that the GPU is idle and
|
|
|
|
|
* all stray writes are flushed *before* we dismantle the backing
|
|
|
|
|
* storage for the pinned objects.
|
|
|
|
|
*
|
|
|
|
|
* However, since we are uncertain that resetting the GPU on older
|
|
|
|
|
* machines is a good idea, we don't - just in case it leaves the
|
|
|
|
|
* machine in an unusable condition.
|
|
|
|
|
*/
|
|
|
|
|
|
2019-06-12 10:57:20 +00:00
|
|
|
spin_lock_irqsave(&i915->mm.obj_lock, flags);
|
2019-04-24 20:07:14 +00:00
|
|
|
for (phase = phases; *phase; phase++) {
|
2019-06-12 10:57:20 +00:00
|
|
|
LIST_HEAD(keep);
|
|
|
|
|
|
|
|
|
|
while ((obj = first_mm_object(*phase))) {
|
|
|
|
|
list_move_tail(&obj->mm.link, &keep);
|
|
|
|
|
|
|
|
|
|
/* Beware the background _i915_gem_free_objects */
|
|
|
|
|
if (!kref_get_unless_zero(&obj->base.refcount))
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
|
|
|
|
|
|
2019-05-28 09:29:51 +00:00
|
|
|
i915_gem_object_lock(obj);
|
2019-04-24 20:07:14 +00:00
|
|
|
WARN_ON(i915_gem_object_set_to_gtt_domain(obj, false));
|
2019-05-28 09:29:51 +00:00
|
|
|
i915_gem_object_unlock(obj);
|
2019-06-12 10:57:20 +00:00
|
|
|
i915_gem_object_put(obj);
|
|
|
|
|
|
|
|
|
|
spin_lock_irqsave(&i915->mm.obj_lock, flags);
|
2019-05-28 09:29:51 +00:00
|
|
|
}
|
2019-06-12 10:57:20 +00:00
|
|
|
|
|
|
|
|
list_splice_tail(&keep, *phase);
|
2019-04-24 20:07:14 +00:00
|
|
|
}
|
2019-06-12 10:57:20 +00:00
|
|
|
spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
|
2019-04-24 20:07:14 +00:00
|
|
|
|
|
|
|
|
i915_gem_sanitize(i915);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void i915_gem_resume(struct drm_i915_private *i915)
|
|
|
|
|
{
|
|
|
|
|
GEM_TRACE("\n");
|
|
|
|
|
|
|
|
|
|
intel_uncore_forcewake_get(&i915->uncore, FORCEWAKE_ALL);
|
|
|
|
|
|
2019-09-10 14:38:20 +00:00
|
|
|
if (intel_gt_init_hw(&i915->gt))
|
2019-06-26 15:45:49 +00:00
|
|
|
goto err_wedged;
|
|
|
|
|
|
2019-04-24 20:07:14 +00:00
|
|
|
/*
|
|
|
|
|
* As we didn't flush the kernel context before suspend, we cannot
|
|
|
|
|
* guarantee that the context image is complete. So let's just reset
|
|
|
|
|
* it and start again.
|
|
|
|
|
*/
|
2019-06-26 15:45:49 +00:00
|
|
|
if (intel_gt_resume(&i915->gt))
|
2019-04-24 20:07:14 +00:00
|
|
|
goto err_wedged;
|
|
|
|
|
|
2019-07-13 10:00:13 +00:00
|
|
|
intel_uc_resume(&i915->gt.uc);
|
2019-04-24 20:07:14 +00:00
|
|
|
|
|
|
|
|
/* Always reload a context for powersaving. */
|
|
|
|
|
if (!i915_gem_load_power_context(i915))
|
|
|
|
|
goto err_wedged;
|
|
|
|
|
|
2019-09-12 12:48:13 +00:00
|
|
|
user_forcewake(&i915->gt, false);
|
|
|
|
|
|
2019-04-24 20:07:14 +00:00
|
|
|
out_unlock:
|
|
|
|
|
intel_uncore_forcewake_put(&i915->uncore, FORCEWAKE_ALL);
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
err_wedged:
|
2019-07-12 19:29:53 +00:00
|
|
|
if (!intel_gt_is_wedged(&i915->gt)) {
|
2019-04-24 20:07:14 +00:00
|
|
|
dev_err(i915->drm.dev,
|
|
|
|
|
"Failed to re-initialize GPU, declaring it wedged!\n");
|
2019-07-12 19:29:53 +00:00
|
|
|
intel_gt_set_wedged(&i915->gt);
|
2019-04-24 20:07:14 +00:00
|
|
|
}
|
|
|
|
|
goto out_unlock;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void i915_gem_init__pm(struct drm_i915_private *i915)
|
|
|
|
|
{
|
drm/i915: Invert the GEM wakeref hierarchy
In the current scheme, on submitting a request we take a single global
GEM wakeref, which trickles down to wake up all GT power domains. This
is undesirable as we would like to be able to localise our power
management to the available power domains and to remove the global GEM
operations from the heart of the driver. (The intent there is to push
global GEM decisions to the boundary as used by the GEM user interface.)
Now during request construction, each request is responsible via its
logical context to acquire a wakeref on each power domain it intends to
utilize. Currently, each request takes a wakeref on the engine(s) and
the engines themselves take a chipset wakeref. This gives us a
transition on each engine which we can extend if we want to insert more
powermangement control (such as soft rc6). The global GEM operations
that currently require a struct_mutex are reduced to listening to pm
events from the chipset GT wakeref. As we reduce the struct_mutex
requirement, these listeners should evaporate.
Perhaps the biggest immediate change is that this removes the
struct_mutex requirement around GT power management, allowing us greater
flexibility in request construction. Another important knock-on effect,
is that by tracking engine usage, we can insert a switch back to the
kernel context on that engine immediately, avoiding any extra delay or
inserting global synchronisation barriers. This makes tracking when an
engine and its associated contexts are idle much easier -- important for
when we forgo our assumed execution ordering and need idle barriers to
unpin used contexts. In the process, it means we remove a large chunk of
code whose only purpose was to switch back to the kernel context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190424200717.1686-5-chris@chris-wilson.co.uk
2019-04-24 20:07:17 +00:00
|
|
|
i915->gem.pm_notifier.notifier_call = pm_notifier;
|
|
|
|
|
blocking_notifier_chain_register(&i915->gt.pm_notifications,
|
|
|
|
|
&i915->gem.pm_notifier);
|
2019-04-24 20:07:14 +00:00
|
|
|
}
|
