With the commit d75b1ade56 ("net: less interrupt masking in NAPI") napi repoll
is done only when work_done == budget. When we are in busy_poll we return 0 in
napi_poll. We should return budget.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
When allocation of all RQs fail, we do not free previously allocated buffers,
before returning error. This causes memory leak.
This patch fixes this by calling vnic_rq_clean(), which frees all the rq
buffers.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds support for setting/getting rss hash key using ethtool.
v2:
respin patch to support RSS hash function changes.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use of well known RSS key might increase attack surface.
Switch to a random one, using generic helper so that all
ports share a common key.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Christian Benvenuti <benve@cisco.com>
Cc: Govindarajulu Varadarajan <_govind@gmx.com>
Cc: Sujith Sankar <ssujith@cisco.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
All the access to wq has been moved out of hardirq context. We no longer need to
use spin_lock_irqsave.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
enic_isr_legacy(), enic_isr_msix() & enic_isr_msi() run from hard
interrupt context.
They can use napi_schedule_irqoff() instead of napi_schedule()
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Check and update posted_index only when skb->xmit_more is 0 or tx queue is full.
v2:
use txq_map instead of skb_get_queue_mapping(skb)
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
With the commit d75b1ade56 ("net: less interrupt masking in NAPI") napi repoll
is done only when work_done == budget. In tx napi poll we always return 0.
So tx napi is not called again and we do not clean up the tx ring.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The following warning is shown when spinlock debug is enabled.
This occurs when enic_flow_may_expire timer function is running and
enic_stop is called on same CPU.
Fix this by using spink_lock_bh().
=================================
[ INFO: inconsistent lock state ]
3.17.0-netnext-05504-g59f35b8 #268 Not tainted
---------------------------------
inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage.
ifconfig/443 [HC0[0]:SC0[0]:HE1:SE1] takes:
(&(&enic->rfs_h.lock)->rlock){+.?...}, at:
enic_rfs_flw_tbl_free+0x34/0xd0 [enic]
{IN-SOFTIRQ-W} state was registered at:
[<ffffffff810a25af>] __lock_acquire+0x83f/0x21c0
[<ffffffff810a45f2>] lock_acquire+0xa2/0xd0
[<ffffffff814913fc>] _raw_spin_lock+0x3c/0x80
[<ffffffffa029c3d5>] enic_flow_may_expire+0x25/0x130[enic]
[<ffffffff810bcd07>] call_timer_fn+0x77/0x100
[<ffffffff810bd8e3>] run_timer_softirq+0x1e3/0x270
[<ffffffff8105f9ae>] __do_softirq+0x14e/0x280
[<ffffffff8105fdae>] irq_exit+0x8e/0xb0
[<ffffffff8103da0f>] smp_apic_timer_interrupt+0x3f/0x50
[<ffffffff81493742>] apic_timer_interrupt+0x72/0x80
[<ffffffff81018143>] default_idle+0x13/0x20
[<ffffffff81018a6a>] arch_cpu_idle+0xa/0x10
[<ffffffff81097676>] cpu_startup_entry+0x2c6/0x330
[<ffffffff8103b7ad>] start_secondary+0x21d/0x290
irq event stamp: 2997
hardirqs last enabled at (2997): [<ffffffff81491865>] _raw_spin_unlock_irqrestore+0x65/0x90
hardirqs last disabled at (2996): [<ffffffff814915e6>] _raw_spin_lock_irqsave+0x26/0x90
softirqs last enabled at (2968): [<ffffffff813b57a3>] dev_deactivate_many+0x213/0x260
softirqs last disabled at (2966): [<ffffffff813b5783>] dev_deactivate_many+0x1f3/0x260
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
----
lock(&(&enic->rfs_h.lock)->rlock);
<Interrupt>
lock(&(&enic->rfs_h.lock)->rlock);
*** DEADLOCK ***
Reported-by: Jan Stancek <jstancek@redhat.com>
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use the much more common pr_warn instead of pr_warning.
Other miscellanea:
o Typo fixes submiting/submitting
o Coalesce formats
o Realign arguments
o Add missing terminating '\n' to formats
Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds support for setting/getting rx_copybreak using
generic ethtool tunable.
Defines enic_get_tunable() & enic_set_tunable() to get/set rx_copybreak.
As of now, these two function supports only rx_copybreak.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Calling dma_map_single()/dma_unmap_single() is quite expensive compared
to copying a small packet. So let's copy short frames and keep the buffers
mapped.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
We should prefer `struct pci_device_id` over `DEFINE_PCI_DEVICE_TABLE` to
meet kernel coding style guidelines. This issue was reported by checkpatch.
A simplified version of the semantic patch that makes this change is as
follows (http://coccinelle.lip6.fr/):
// <smpl>
@@
identifier i;
declarer name DEFINE_PCI_DEVICE_TABLE;
initializer z;
@@
- DEFINE_PCI_DEVICE_TABLE(i)
+ const struct pci_device_id i[]
= z;
// </smpl>
[bhelgaas: add semantic patch]
Signed-off-by: Benoit Taine <benoit.taine@lip6.fr>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Remove the now unnecessary memset too.
Signed-off-by: Joe Perches <joe@perches.com>
Cc: Christian Benvenuti <benve@cisco.com>
Cc: Sujith Sankar <ssujith@cisco.com>
Cc: Govindarajulu Varadarajan <_govind@gmx.com>
Cc: Neel Patel <neepatel@cisco.com>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This patch impliments ethtool_ops->get_rxnfc() to display the classifier
filter added by the driver.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch removes the #ifdef CONFIG_RFS_ACCEL around the classifier filter
structures. This makes the filter structures available when CONFIG_RFS_ACCEL = n.
Introduce enic_rfs_timer_start() & enic_rfs_timer_stop() to start/stop the
timer. These two functions are nop when CONFIG_RFS_ACCEL = n.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
enic_set_coalesce() has two problems.
* It should return -EINVAL and not -EOPNOTSUPP for invalid coalesce values.
* In case of MSIX, enic_set_coalesce return error after applying requested
coalescing setting partially. We should either apply all the setting requeste
and return success or apply non and return error.
* This patch also simplifies the algo.
This was introduced by
'7c2ce6e60f703 enic: Add support for adaptive interrupt coalescing'
These changes were suggested by Ben Hutchings here
http://www.spinics.net/lists/netdev/msg283972.html
Also change enic driver version.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
If CONFIG_RFS_ACCEL=n:
drivers/net/ethernet/cisco/enic/enic_main.c: In function 'enic_open':
drivers/net/ethernet/cisco/enic/enic_main.c:1603:2: error: implicit declaration of function 'enic_rfs_flw_tbl_init' [-Werror=implicit-function-declaration]
drivers/net/ethernet/cisco/enic/enic_main.c: In function 'enic_stop':
drivers/net/ethernet/cisco/enic/enic_main.c:1630:2: error: implicit declaration of function 'enic_rfs_flw_tbl_free' [-Werror=implicit-function-declaration]
Introduced in commit a145df23ef ("enic: Add
Accelerated RFS support").
Dummy functions are provided, but their prototypes are missing, causing the
build failure. Provide dummy static inline functions instead to fix this.
Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Till now enic had been doing tx clean in isr.
Using napi infrastructure to move the tx clean up out of isr to softirq.
Now, wq isr schedules napi poll. In enic_poll_msix_wq we clean up the tx queus.
This is applicable only on MSIX. In INTx and MSI we use single napi to clean
both rx & tx queues.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds support for low latency busy_poll.
* Introduce drivers ndo_busy_poll function enic_busy_poll, which is called by
socket waiting for data.
* Introduce locking between napi_poll nad busy_poll
* enic_busy_poll cleans up all the rx pkts possible. While in busy_poll, rq
holds the state ENIC_POLL_STATE_POLL. While in napi_poll, rq holds the state
ENIC_POLL_STATE_NAPI.
* in napi_poll we return if we are in busy_poll. Incase of INTx & msix, we just
service wq and return if busy_poll is going on.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
We were experiencing occasional "BUG: scheduling while atomic" splats
in our testing. Enabling DEBUG_SPINLOCK and DEBUG_LOCKDEP in the kernel
exposed a lockdep in the enic driver.
enic 0000:0b:00.0 eth2: Link UP
======================================================
[ INFO: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected ]
3.12.0-rc1.x86_64-dbg+ #2 Tainted: GF W
------------------------------------------------------
NetworkManager/4209 [HC0[0]:SC0[2]:HE1:SE0] is trying to acquire:
(&(&enic->devcmd_lock)->rlock){+.+...}, at: [<ffffffffa026b7e4>] enic_dev_packet_filter+0x44/0x90 [enic]
The fix was to replace spin_lock with spin_lock_bh for the enic
devcmd_lock, so that soft irqs would be disabled while the lock
is held.
Signed-off-by: Sujith Sankar <ssujith@cisco.com>
Signed-off-by: Tony Camuso <tcamuso@redhat.com>
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds supports for Accelerated Receive Flow Steering.
When the desired rx is different from current rq, for a flow, kernel calls the
driver function enic_rx_flow_steer(). enic_rx_flow_steer adds a IP-TCP/UDP
hardware filter.
Driver registers a timer function enic_flow_may_expire. This function is called
every HZ/4 seconds. In this function we check if the added filter has expired
by calling rps_may_expire_flow(). If the flow has expired, it removes the hw
filter.
As of now adaptor supports only IPv4 - TCP/UDP filters.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
rx_cpu_rmap provides the reverse irq cpu affinity. This patch allocates and
sets drivers netdev->rx_cpu_rmap accordingly.
rx_cpu_rmap is set in enic_request_intr() which is called by enic_open and
rx_cpu_rmap is freed in enic_free_intr() which is called by enic_stop.
This is used by Accelerated RFS.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds interface to add and delete IP 5 tuple filter. This interface
is used by Accelerated RFS code to steer a flow to corresponding receive
queue.
As of now adaptor supports only ipv4 + tcp/udp packet steering.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Hardware (in readq(&devcmd->args[0])) returns positive number in case of error.
But _vnic_dev_cmd should return a negative value in case of error.
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This change updates the enic driver to make use of __dev_uc_sync and
__dev_mc_sync calls. Previously the driver was doing its own list
management by storing the mc_addr and uc_addr list in a 32 address array.
With this change the sync data is stored in the netdev_addr_list structures
and instead we just track how many addresses we have written to the device.
When we encounter 32 we stop and print a message as occurred previously with
the old approach.
Other than the core change the only other bit needed was to propagate the
constant attribute with the MAC address as there were several spots where
is twas only passed as a u8 * instead of a const u8 *.
This patch is meant to maintain the original functionality without the use
of the mc_addr and uc_addr arrays.
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
Acked-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Division of a 32 bit number by a 64 bit number causes the following link
error introduced by
7c2ce6e60f "enic: Add support for adaptive interrupt coalescing"
drivers/built-in.o: In function `enic_poll_msix':
enic_main.c:(.text+0x48710a): undefined reference to `__udivdi3'
make: *** [vmlinux] Error 1
Since numerator is 32 bit, convert denominator to 32 bit accordingly.
Fixes: 7c2ce6e60f ("enic: Add support for adaptive interrupt coalescing")
Reported-by: Jim Davis <jim.epost@gmail.com>
Cc: Christian Benvenuti <benve@cisco.com>
Cc: Sujith Sankar <ssujith@cisco.com>
Cc: Neel Patel <neepatel@cisco.com>
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds support for adaptive interrupt coalescing.
For small pkts with low pkt rate, we can decrease the coalescing interrupt
dynamically which decreases the latency. This however increases the cpu
utilization. Based on testing with different coal intr and pkt rate we came up
with a table(mod_table) with rx_rate and coalescing interrupt value where we
get low latency without significant increase in cpu. mod_table table stores
the coalescing timer percentage value for different throughputs.
Function enic_calc_int_moderation() calculates the desired coalescing intr timer
value. This function is called in driver rx napi_poll. The actual value is set
by enic_set_int_moderation() which is called when napi_poll is complete. i.e
when we unmask the rx intr.
Adaptive coal intr is support only when driver is using msix intr. Because
intr is not shared.
Struct mod_range is used to store only the default adaptive coalescing intr
value.
Adaptive coal intr calue is calculated by
timer = range_start + ((rx_coal->range_end - range_start) *
mod_table[index].range_percent / 100);
rx_coal->range_end is the rx-usecs-high value set using ethtool.
range_start is rx-usecs-low, set using ethtool, if rx_small_pkt_bytes_cnt is
greater than 2 * rx_large_pkt_bytes_cnt. i.e small pkts are dominant. Else its
rx-usecs-low + 3.
Cc: Christian Benvenuti <benve@cisco.com>
Cc: Neel Patel <neepatel@cisco.com>
Signed-off-by: Sujith Sankar <ssujith@cisco.com>
Signed-off-by: Govindarajulu Varadarajan <_govind@gmx.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
net: get rid of SET_ETHTOOL_OPS
Dave Miller mentioned he'd like to see SET_ETHTOOL_OPS gone.
This does that.
Mostly done via coccinelle script:
@@
struct ethtool_ops *ops;
struct net_device *dev;
@@
- SET_ETHTOOL_OPS(dev, ops);
+ dev->ethtool_ops = ops;
Compile tested only, but I'd seriously wonder if this broke anything.
Suggested-by: Dave Miller <davem@davemloft.net>
Signed-off-by: Wilfried Klaebe <w-lkml@lebenslange-mailadresse.de>
Acked-by: Felipe Balbi <balbi@ti.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The enic driver fails to build on ARM with:
In file included from drivers/net/ethernet/cisco/enic/enic_res.c:40:0:
drivers/net/ethernet/cisco/enic/enic.h:48:2: error: expected specifier-qualifier-list before 'irqreturn_t'
irqreturn_t (*isr)(int, void *);
^
Nothing in the driver is explicitly including the irq definitions, so we add
an include of linux/irq.h to pick them up.
Signed-off-by: Josh Boyer <jwboyer@fedoraproject.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Replace dev_kfree_skb with dev_kfree_skb_any in enic_hard_start_xmit
that can be called in hard irq and other contexts.
enic_hard_start_xmit only frees the skb when dropping it.
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
Processing any incoming packets with a with a napi budget of 0
is incorrect driver behavior.
This matters as netpoll will shortly call drivers with a budget of 0
to avoid receive packet processing happening in hard irq context.
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
As result of deprecation of MSI-X/MSI enablement functions
pci_enable_msix() and pci_enable_msi_block() all drivers
using these two interfaces need to be updated to use the
new pci_enable_msi_range() and pci_enable_msix_range()
interfaces.
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
Cc: Christian Benvenuti <benve@cisco.com>
Cc: Sujith Sankar <ssujith@cisco.com>
Cc: Govindarajulu Varadarajan <govindarajulu90@gmail.com>
Cc: Neel Patel <neepatel@cisco.com>
Cc: netdev@vger.kernel.org
Cc: linux-pci@vger.kernel.org
Acked-by: Govindarajulu Varadarajan <govindarajulu90@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Drivers should call skb_set_hash to set the hash and its type
in an skbuff.
Signed-off-by: Tom Herbert <therbert@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The driver core clears the driver data to NULL after device_release
or on probe failure. Thus, it is not needed to manually clear the
device driver data to NULL.
Signed-off-by: Jingoo Han <jg1.han@samsung.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch exposes symbols for usnic low latency driver that can be used to
register and unregister vNics as well to traverse the resources on vNics.
Signed-off-by: Upinder Malhi <umalhi@cisco.com>
Signed-off-by: Nishank Trivedi <nistrive@cisco.com>
Signed-off-by: Christian Benvenuti <benve@cisco.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
In servers with more than 1.1 TB of RAM, the existing 40/32 bit DMA
could cause failure as the DMA-able address could go outside the range
addressable using 40/32 bits.
The following patch first tried 64 bit DMA if possible, failover to 32
bit.
Signed-off-by: Sujith Sankar <ssujith@cisco.com>
Signed-off-by: Christian Benvenuti <benve@cisco.com>
Signed-off-by: Govindarajulu Varadarajan <govindarajulu90@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The following patch sets the skb->rxhash and skb->q_number.
This is used by RPS and RFS. Kernel can make use of hw provided hash
instead of calculating the hash.
Signed-off-by: Govindarajulu Varadarajan <govindarajulu90@gmail.com>
Signed-off-by: Nishank Trivedi <nistrive@cisco.com>
Signed-off-by: Christian Benvenuti <benve@cisco.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The following patch adds multi tx support for enic.
Signed-off-by: Nishank Trivedi <nistrive@cisco.com>
Signed-off-by: Christian Benvenuti <benve@cisco.com>
Signed-off-by: Govindarajulu Varadarajan <govindarajulu90@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch generates a hardware crash notification (NETDEV_REBOOT)
during reset. After a hardware crash, ENIC resets all its resources
including queue pair filters programmed by USNIC. USNIC registers for
this notification, and on receiving it, reprograms the queue pair
filters.
Signed-off-by: Neel Patel <neepatel@cisco.com>
Signed-off-by: Nishank Trivedi <nistrive@cisco.com>
Signed-off-by: Christian Benvenuti <benve@cisco.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds an interface for USNIC to proxy firmware commands
through ENIC.
Signed-off-by: Neel Patel <neepatel@cisco.com>
Signed-off-by: Nishank Trivedi <nistrive@cisco.com>
Signed-off-by: Christian Benvenuti <benve@cisco.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch,
- Adds new firmware commands for the new Cisco Low Latency NIC
(aka. USNIC).
Signed-off-by: Neel Patel <neepatel@cisco.com>
Signed-off-by: Nishank Trivedi <nistrive@cisco.com>
Signed-off-by: Christian Benvenuti <benve@cisco.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
