Move scsi_remove_host call into sas_remove_host and remove it from SAS
HBA drivers, so we don't mess up the ordering. This solves an issue with
double deleting sysfs entries that was introduced by the change of sysfs
behaviour from commit bcdde7e221 ("sysfs: make __sysfs_remove_dir()
recursive").
[mkp: addressed checkpatch complaints]
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Suggested-by: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: James Bottomley <jejb@linux.vnet.ibm.com>
Cc: Jinpu Wang <jinpu.wang@profitbricks.com>
Cc: John Garry <john.garry@huawei.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Jinpu Wang <jinpu.wang@profitbricks.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
For 1 bit ECC errors, those errors can be recovered by hw. But for
multi-bits ECC and AXI errors, there are something wrong with whole
module or system, so try reset the controller to recover those errors
instead of calling panic().
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
If a TMF timeouts (maybe due to unlikely scenario of an expander being
unplugged when TMF for remote device is active), when we eventually try
to free the slot, we crash as we dereference the slot's task, which has
already been released.
As a fix, add checks in the slot release code for a NULL task.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This patch is a workaround for a SoC bug where an internal abort command
may timeout. In v2 hw, the channel should become idle in order to finish
abort process. If the target side has been sending HOLD, host side
channel failed to complete the frame to send, and can not enter the idle
state. Then internal abort command will timeout.
As this issue is only in v2 hw, we deal with it in the hw layer. Our
workaround solution is: If abort is not finished within a certain period
of time, we will check HOLD status. If HOLD has been sending, we will
send break command.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This patch adds a workaround solution for a SoC bug which may cause SoC
logic fatal error when disabling a PHY. Then we find internal abort IO
timeout may occur, and the controller IO breakpoint may be corrupted.
We work around this SoC bug by optimizing the flow of disabling a PHY.
Signed-off-by: Xiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This patch provides a workaround a SoC bug where SATA IPTTs for
different devices may conflict.
The workaround solution requests the following:
1. SATA device id must be even and not equal to SAS IPTT.
2. SATA device can not share the same IPTT with other SAS or
SATA device.
Besides we shall consider IPTT value 0 is reserved for another SoC bug
(STP device open link at firstly after SAS controller reset).
To sum up, the solution is: Each SATA device uses independent and
continuous 32 even IPTT from 64 to 4094, then v2 hw can only support 63
SATA devices. All SAS device(SSP/SMP devices) share odd IPTT value from
1 to 4095.
Signed-off-by: Xiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
After resetting the controller, the process of scanning SATA disks
attached to an expander may fail occasionally. The issue is that the
controller can't close the STP link created by target if the max link
time is 0.
To workaround this issue, we reject STP link after resetting the
controller, and change the corresponding PHY to accept STP link only
after receiving data.
We do this check in cq interrupt handler. In order not to reduce
efficiency, we use an variable to control whether we should check and
change PHY to accept STP link.
The function phys_reject_stp_links_v2_hw() should be called after
resetting the controller.
The solution of another SoC bug "SATA IO timeout", that also uses the
same register to control STP link, is not effective before the PHY
accepts STP link.
Signed-off-by: Xiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Removing the 'select SCSI_SAS_LIBSAS' statement in Kconfig resulted in a
link failure in configurations that have hisi_sas built-in but libsas as
a loadable module:
drivers/scsi/built-in.o: In function `hisi_sas_scan_finished':
hisi_sas_main.c:(.text+0x37ce9): undefined reference to `sas_drain_work'
drivers/scsi/built-in.o: In function `hisi_sas_slave_configure':
hisi_sas_main.c:(.text+0x37d17): undefined reference to `sas_slave_configure'
hisi_sas_main.c:(.text+0x37d40): undefined reference to `sas_change_queue_depth'
drivers/scsi/built-in.o: In function `hisi_sas_remove':
All other libsas users have the 'select' statement, so we should do the
same here for consistency. For all I can tell, the patch that added the
sata softreset does not actually introduce a dependency on SCSI_SAS_ATA
but instead adds calls into libata itself, so we can express that with a
more specific dependency.
We cannot have 'select SCSI_SAS_LIBSAS; depends on SCSI_SAS_ATA' as that
would cause a dependency loop.
Fixes: 7c594f0407 ("scsi: hisi_sas: add softreset function for SATA disk")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
It appears that a break in the TRANS_TX_OPEN_CNX_ERR_NO_DESTINATION case
got accidentally removed in an earlier commit, as it stands, the
ts->stat and ts->open_rej_reason are being updated twice for this case
which looks incorrect. Fix this by adding in the missing break
statement.
Detected by CoverityScan, CID#1422110 ("Missing break in switch")
Fixes: 634a9585f4 ("scsi: hisi_sas: process error codes according to their priority")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add helper function is_sata_phy_v2_hw() to judge whether the attached
device is SATA disk for a root PHY.
Signed-off-by: Xiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
When SMP IO is sent, sas_protocol_ata couldn't judge whether the disk is
SATA or SAS disk. So use dev_is_sata to identify SATA or SAS disk.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Unless we actually get some sort of failure in hisi_sas_lu_reset(),
don't print a message.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
When an SMP task timeouts, it will call lldd_abort_task to release the
associated slot, and then will release the sas_task.
Currently in lldd_abort_task, if we fail to internally abort IO, then
the slot of SMP IO is not released, but sas_task will still be later
released, so the slot's sas_task is NULL, which will cause NULL pointer
when hisi_sas_slot_task_free happens later.
To resolve, check the return value of internal abort, and release the
slot if it failed.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add function for upper-layer to reset controller when all else fails.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
For consistency, remove the "hisi_sas_" prefix.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Handle the situation that PHY UP and DOWN irq happen simultaneously.
There is no mechanism of SoC HW to ensure this situation will never
happen. So, we add this handle just in case.
Signed-off-by: Xiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This patch includes:
(1) Disable transport layer retry
(2) Support CQ time and count interrupt coal
(3) fix link FIFO full issue
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Zhao Nenglong <zhaonenglong@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
There are some rules to decide which error code has the high priority
when errors happen together:
(1) Error phase of CQ decides the error happens on RX or TX;
(2) For TX error, when DMA/TRANS TX error happen simultaneously, the
priority of DMA TX error is higher than TRANS TX error, so for the
priority of TX error: DW2 (DMA TX part) > DW0;
(3) For RX error, when TRANS/DMA/SIPC RX error happen simultaneously,
the priority of TRANS RX error is higher than DMA and SIPC RX error,
and we should also keep the rules (the priority of DW3 > DW2), so
for the priority of RX error: DW1 > DW3 > DW2(SIPC RX part);
(4) There are also a priority we should keep in the same error type.
So, modify slot error code to handle this.
In addition to this, some some error codes are modified according to
recommendation from SoC designer.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
When a TMF or internal abort times-out, do not free slot. We expect this
to be done upon later escalated error handling.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Some more locking needs to be added/modified for when
read-modify-writing sas_task.task_state_flags.
Note: since we can attempt to grab this lock in interrupt
context we should use irq variant of spin_lock.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
After hardreset, we clear up IOs of remote disks, so we need to free
those slots in LLDD.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Check in slot_complete_v2_hw() for whether a task has already been
completed by upper layer.
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
When issuing an LU reset for a SATA target, issue an internal abort and
a hard reset.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Currently an internal abort is executed regardless of the result of the
TMF. We should also check the result of the internal abort to see if we
should free the slot.
So change the status code STAT_IO_COMPLETE to TMF_RESP_FUNC_SUCC,
meaning the slot has been successfully aborted.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
For error codes which need abort-and-retry, simulate IO timeout and let
SCSI+ATA layers process those errors.
Previously for SSP, we should try to abort the IO in the LLDD and then
pass back to upper layer, but sometimes this would also error. So
Instead of adding special error handling for this scenario in the LLDD,
allow the upper layer to handle completely.
No performance hit is seen by taking this approach.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
We currently do a hard reset for a link reset. Change this to do a link
reset only.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiang Chen <chenxiang@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
When sas_port is NULL, then return SAS_PHY_DOWN.
In addition, when the sas_dev is gone then explicitly return
SAS_PHY_DOWN.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Currently when a root PHY is deformed from a asd_sas_port we try to
release the slots in the LLDD, and fail.
Regardless, it is not right to release this early.
This patch removes the deformed function. As it was before, port
deformation is still done in hisi_sas_phy_down().
It would be nice to actually remove the hisi_sas_port_{de}formed() pair,
however we cannot as we need to know the asd_sas_port index libsas has
associated with an asd_sas_phy.
The hw does actually generate a port id for a PHY, but this seems to a
random number, so ignored for this purpose.
This patch also changes the code to link slots to the hisi_sas_device,
and not hisi_sas_port.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add softreset to clear IO after internal abort device for SATA disk.
The SATA error handling for the controller is based on device internal
abort and softreset function.
The controller does not support internal abort for single IO, so we need
to execute internal abort for device.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Relocate the PHY init code from LLDD hw init path to
hisi_sas_scan_start().
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
There are some scenarios that we need to warm-reset to reset registers
of SAS controller. During reset we disable interrupts/DQs/PHYs, and
after reset we re-init the hardware and rescan the topology to see if
anything changed.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Xiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Introduce function to get hisi_sas_port from asd_sas_port.
Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
There is an issue that hisi_sas_dev.running_req is not
decremented properly for internal abort and TMF.
To resolve, only decrease running_req in hisi_sas_slot_task_free()
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
There is a potential probe issue in how we trigger the hw initialisation.
Although we use 1s timer to delay hw initialisation, there is still a
potential that sas_register_ha() is not be finished before we start
the PHY init from hw->hw_init().
To avoid this issue, initialise the hw after sas_register_ha() in the
same probe context.
Note: it is not necessary to use 1s timer now (modified v2 hw only).
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Downgrade the exit print in hisi_sas_internal_task_abort()
to dbg level, as info is not required.
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Correctly set registers in v2 for root PHY hardreset for directly
attached disk.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The message to inform that the controller has no refclk
is currently at warning level, which is unnecessary, so
downgrade to debug.
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Set SMP connection timeout and continue AWT timer;
Clear ITCT table when dev gone.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The v2 SAS controller needs more time to detect channel idle
and send setup link request than SATA disk does, so it is
difficult for the SAS controller to setup an STP link. Therefore
it may cause some IO timeouts.
We need to periodically configure the SAS controller so it
doesn't receive STP setup requests from SATA disks for a while,
so IO can be sent during this period.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
When we call hisi_sas_slot_task_free() we should grab the hisi_hba.lock,
as hisi_sas_slot_task_free() accesses common hisi_hba elements.
Function hisi_sas_slot_abort() is missing this, so add it.
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
There is a bug in the current driver in that certain hisi_hba and port
structure elements which we access when servicing the CQ interrupt do
not use thread-safe accesses; these include hisi_sas_port linked-list of
active slots (hisi_sas_port.entry), bitmap of currently allocated IPTT
(in hisi_hba.slot_index_tags), and completion queue read pointer.
As a solution, lock these elements with the hisi_hba.lock.
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Currently the all the slot processing for the completion queue is done
in ISR context. It is judged that the slot processing can take a long
time, especially when a SATA NCQ completes (upto 32 slots).
So, as a solution, defer the bulk of the ISR processing to tasklet
context. Each CQ will have its down tasklet.
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
In the hip06 and hip07 SoCs, the interrupt lines from the SAS
controllers are connected to mbigen hw module [1]. The mbigen module is
probed with module_init, and, as such, is not guaranteed to probe before
the SAS driver. So we need to support deferred probe.
We check for probe deferral in the hw layer probe, so we not probe into
the main layer and allocate shost, memories, etc., to later learn that
we need to defer the probe.
[1] ./Documentation/devicetree/bindings/interrupt-controller/hisilicon,mbigen-v2.txt
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This patch addresses 4 problems in the module probe/remove:
- When hisi_sas_shost_alloc() fails after we alloc shost memory, we
should free shost memory before the function returns.
- When hisi_sas_probe() fails after we alloc the HBA memories, we
should also free the HBA memories.
- We should free shost memory at the end of hisi_sas_remove().
- sha->core.shost is set twice, so remove extra set.
Signed-off-by: Xiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Quentin Lambert <lambert.quentin@gmail.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
There are some typos where we intended "<<" but have "<". Seems likely
to cause a bunch of problems.
Fixes: d3b688d3c6 ("scsi: hisi_sas: add v2 hw support for ECC and AXI bus fatal error")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Acked-by: John Garry <john.garry@huawei.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add the function to set PHY min and max linkrate through
sysfs interface.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Sometimes the value of hisi_sas_device.running_req
would go negative unless we have the check for
running_req >= 0 before trying to decrement.
This is because using running_req is not thread-safe.
As such, the value for running_req may be actually incorrect,
so use atomic64_t instead.
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Check ERR bit of status to decide whether there is something wrong with
initial register-D2H FIS. If error exists, PHY reset the channel to
restart OOB.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Modify and add some SATA commands according to SATA protocol.
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Delete repeated configuration items for hisi_sas_device() when
we free a device. These items are now only set in
hisi_sas_dev_gone().
Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
