When performing Tx cleanup, the dirty index counter is compared to the
current index counter as one of the tests used to determine when to stop
cleanup. The "less than" test will fail when the current index counter
rolls over to zero causing cleanup to never occur again. Update the test
to a "not equal" to avoid this situation.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The SKB for a Tx packet is associated with an xgbe_ring_data structure
in the xgbe_map_tx_skb function. However, it is being saved in the
structure after the last structure used when the SKB is mapped. Use
the last used structure to save the SKB value.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The interrupt value within the xgbe_ring_data structure is used as an
indicator of which Rx descriptor should have the INTE bit set to
generate an interrupt when that Rx descriptor is used. This bit was
mistakenly cleared in the xgbe_unmap_rdata function, effectively
nullifying the ethtool rx-frames support.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
When requesting an irq, the name passed in must be (part of) allocated
memory. The irq name was a local variable and resulted in random
characters when listing /proc/interrupts. Add a character field to the
xgbe_channel structure to hold the irq name and use that.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch extends the set/get_rxfh ethtool-options for getting or
setting the RSS hash function.
It modifies drivers implementation of set/get_rxfh accordingly.
This change also delegates the responsibility of checking whether a
modification to a certain RX flow hash parameter is supported to the
driver implementation of set_rxfh.
User-kernel API is done through the new hfunc bitmask field in the
ethtool_rxfh struct. A bit set in the hfunc field is corresponding to an
index in the new string-set ETH_SS_RSS_HASH_FUNCS.
Got approval from most of the relevant driver maintainers that their
driver is using Toeplitz, and for the few that didn't answered, also
assumed it is Toeplitz.
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Ariel Elior <ariel.elior@qlogic.com>
Cc: Prashant Sreedharan <prashant@broadcom.com>
Cc: Michael Chan <mchan@broadcom.com>
Cc: Hariprasad S <hariprasad@chelsio.com>
Cc: Sathya Perla <sathya.perla@emulex.com>
Cc: Subbu Seetharaman <subbu.seetharaman@emulex.com>
Cc: Ajit Khaparde <ajit.khaparde@emulex.com>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Cc: Jesse Brandeburg <jesse.brandeburg@intel.com>
Cc: Bruce Allan <bruce.w.allan@intel.com>
Cc: Carolyn Wyborny <carolyn.wyborny@intel.com>
Cc: Don Skidmore <donald.c.skidmore@intel.com>
Cc: Greg Rose <gregory.v.rose@intel.com>
Cc: Matthew Vick <matthew.vick@intel.com>
Cc: John Ronciak <john.ronciak@intel.com>
Cc: Mitch Williams <mitch.a.williams@intel.com>
Cc: Amir Vadai <amirv@mellanox.com>
Cc: Solarflare linux maintainers <linux-net-drivers@solarflare.com>
Cc: Shradha Shah <sshah@solarflare.com>
Cc: Shreyas Bhatewara <sbhatewara@vmware.com>
Cc: "VMware, Inc." <pv-drivers@vmware.com>
Cc: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: Eyal Perry <eyalpe@mellanox.com>
Signed-off-by: Amir Vadai <amirv@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Add support to delay telling the hardware about data that is ready to
be transmitted if the skb->xmit_more flag is set.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The current form of Tx coalescing works on a descriptor basis instead
of on a packet basis and doesn't take into account TSO packets. Update
the Tx coalescing support to work on a packet basis, taking into
account the number of packets associated with a TSO transmit. Also,
only activate the Tx timer if a timer value is set.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The tso_header variable in the xgbe_tx_ring_data structure is not used,
remove it.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Call the appropriate BQL functions to track the number of bytes queued
during Tx processing and to track the number of packets and bytes
that have been transmitted during Tx complete processing.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Move the Tx and Rx related fields within the xgbe_ring_data struct into
their own structs in order to more easily see what fields are used for
each operation.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The Smatch tool indicated that one of the if statements in xgbe-dev.c
could be rewritten to remove a redundant check for the 'err' variable
in an if statement.
Change the statement as suggested and add a comment to help clarify.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
If the Tx engine is told to stop while it is actively processing Tx
descriptors it is possible that the Tx descriptor(s) will not be closed
out properly. When the Tx engine is restarted this could result in the
driver being stuck on the improperly closed descriptor.
Update the driver to wait for the Tx engine to be in a stopped or
suspended state before issuing the stop command.
This has not been an issue to date, but it's a good safe-guard to have.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Add a read memory barrier to the Tx and Rx paths where the ownership
bit is checked to be sure that all descriptor fields are read after
having read the ownership bit for the descriptor.
This has not been an issue to date, but it's a good safe-guard to have.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use netdev_rss_key_fill() helper, as it provides better support for some
bonding setups.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Lendacky, Thomas <Thomas.Lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
There was a missing break statement so we set everything to
PKT_HASH_TYPE_L3 even when we intended to use PKT_HASH_TYPE_L4.
Fixes: 5b9dfe299e ('amd-xgbe: Provide support for receive side scaling')
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Change the types of the descriptor entries in the xgbe_ring_desc struct
from u32 to __le32 to fix endian warnings issued by sparse.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
If the skb allocation fails during receive processing, the driver would
continue reading descriptors without first determining if there were
any more descriptors for the current packet. Update the code to check
whether more descriptors are associated with the current packet or
whether to move on to the next descriptor as a new packet.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The channel structure is freed before freeing the per channel
interrupts resulting in a kernel oops. Move the call to free
the channel structure to after the freeing of the per channel
interrupts.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch fixes the spelling of the word "descriptor" in a couple
of locations.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds support for ethtool receive side scaling (RSS) commands.
Support is added to get/set the RSS hash key and the RSS lookup table.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch provides support for receive side scaling (RSS). RSS allows
for spreading incoming network packets across the Rx queues. When used
in conjunction with the per DMA channel interrupt support, this allows
the receive processing to be spread across multiple processors.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch provides support for interrupts that are generated by the
Tx/Rx DMA channel pairs of the device. This allows for Tx and Rx
processing to run across multiple processsors.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Provide support for splitting IP packets so that the header and
payload can be sent to different DMA addresses. This will allow
the IP header to be put into the linear part of the skb while the
payload can be added as frags.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use page allocations for Rx buffers instead of pre-allocating skbs
of a set size.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The Tx and Rx descriptors are unsigned 32 bit values. Use the u32
type, rather than unsigned int, to map these descriptors.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The pre_xmit function name implies that it performs operations prior
to transmitting the packet when in fact it is responsible for setting
up the descriptors and initiating the transmit. Rename this to
function from pre_xmit to dev_xmit, which is consistent with the name
used during receive processing - dev_read.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Move the channel and ring tracking structures allocation to device
open. This will allow for future support to vary the number of Tx/Rx
queues without unloading the module.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Currently the amd-xgbe driver increments the packets processed counter
each time a descriptor is processed. Since a packet can be represented
by more than one descriptor incrementing the counter in this way is not
appropriate. Also, since multiple descriptors cause the budget check
to be short circuited, sometimes the returned value from the poll
function would be larger than the budget value resulting in a WARN_ONCE
being triggered.
Update the polling logic to properly account for the number of packets
processed and exit when the budget value is reached.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The ndo_set_features callback function was improperly using an unsigned
int to save the current feature value for features such as NETIF_F_RXCSUM.
Since that feature is in the upper 32 bits of a 64 bit variable the
result was always 0 making it not possible to actually turn off the
hardware RX checksum support. Change the unsigned int type to the
netdev_features_t type in order to properly capture the current value
and perform the proper operation.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
As the management counters reach a threshold they will generate an
interrupt so the value can be saved and the counter reset. The
current code does not enable this interrupt on all counters. This
can result in inaccurate statistics.
Update the code to enable all the counters to generate an interrupt
when its threshold is exceeded.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Even if the management counters are configured to be 32 bit register
values, the [rt]xoctetcount_gb and [rt]xoctetcount_g counters are
always 64 bit counter registers. Since they are not being treated as
64 bit values, these statistics are being reported incorrectly (ifconfig,
ethtool, etc.).
Update the routines used to read the registers to access the "hi"
register (an offset of 4 from the "lo" register) to create a 64 bit
value for these 64 bit counters.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch contains fixes identified by checkpatch when run with the
strict option.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
During allocation and initialization of the network driver structures,
the wrong pointer is used to initialize a spin lock. Fix the spin lock
initialization by using the proper pointer.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
When configuring Tx flow control the Rx queue count was used instead of
the Tx queue count for looping through the Tx hardware queues. Fix the
code to use the Tx queue count.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The debugfs support for the xpcs registers did not properly use the
specified mmd (xpcs_mmd entry) which resulted in the default mmd
value always being used. Update the debugfs support to generate the
proper mmd register value.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The fifo size reported by the hardware is not correct. Add support
to limit the reported size to what is actually present. Also, fix
the argument types used in the fifo size calculation function.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The flushing of the Tx hardware queues is only supported at a certain
level of the hardware. Retrieve the current version of the hardware
and use that to determine if flushing is supported.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The header linux/phy.h was included twice, so delete one of them.
Signed-off-by: Jean Sacren <sakiwit@gmail.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
A change added to the mdiobus/phy api added a module_get/module_put
during phy connect/disconnect processing. Currently, the driver
performs a phy connect during module probe and a phy disconnect during
module remove. With the addition of the module_get during phy connect
the amd-xgbe module use count is incremented and can no longer be
unloaded.
Move the phy connect/disconnect from the driver probe/remove functions
to the net_device_ops ndo_open/ndo_stop functions. This allows the
module use count to be decremented when the device(s) are brought down
and allows the module to be unloaded.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use the dma_set_mask_and_coherent function to set the DMA mask rather
than setting the DMA mask fields directly. This was originally done
to work around a bug in the arm64 DMA support when RAM started above
the 4GB boundary which has since been fixed.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Remove the spinlocks around the ethtool get and set settings
functions and within the link adjustment callback routine.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds support for traffic classes as well as support
for Data Center Bridging interfaces related to traffic classes
and priority flow control.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
When setting the fifo sizes for the queues and enabling the queues
use the number of active Tx and Rx queues that have been enabled
not the maximum number available.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Update the amd-xgbe driver and phylib driver to better support
the 2.5GbE mode for the hardware. In order to be able establish
2.5GbE using clause 73 auto negotiation the device will support
speed sets of 1GbE/10GbE and 2.5GbE/10GbE.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds support for Tx and Rx hardware timestamping.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Fix to return a negative error code from the setting real tx queue
count error handling case instead of 0.
Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
With the addition of entries in the phy speed/duplex settings
array to support KR and KX mode, the work-around to add/remove
baseT settings to run at a fixed speed is no longer needed.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The default cache operations for ARM64 were changed during 3.15.
To use coherent operations a "dma-coherent" device tree property
is required. If that property is not present in the device tree
node then the non-coherent operations are assigned for the device.
Add support to the amd-xgbe driver to assign the AXI DMA cache settings
based on whether the "dma-coherent" property is present in the device
node. If present, use settings that work with the caches. If not
present, use settings that do not look at the caches.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch provides some general performance enhancements for the
driver:
- Modify the default coalescing settings (reduce usec, increase frames)
- Change the AXI burst length to 256 bytes (default was 16 bytes which
was smaller than a cache line)
- Change the AXI cache settings to write-back/write-allocate which
allocate cache entries for received packets during the DMA since the
packet will be processed soon afterwards
- Combine ioread/iowrite when disabling both the Tx and Rx interrupts
- Change to processing the Tx/Rx channels in pairs
- Only recycle the Rx descriptors when a threshold of dirty descriptors
is reached
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Currently the napi context is added using netif_napi_add each time
the ndo_open operation is called. However, there is not a
corresponding netif_napi_del call during the ndo_stop operation. If
the device ndo_open operation was called more than once an infinite
loop occurs during module unload. Add a call to netif_napi_del during
the ndo_stop operation.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
When initializing the MTL interrupts the interrupt status
register is written to instead of the interrupt enable register.
Since no MTL interrupts are being enabled and the default state
is for MTL interrupts to be disabled this did not cause a problem,
but needs to be fixed to target the correct register.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The initial change from sscanf to kstrtouint broke backward
compatbility by using a base of "0" in the kstrtouint call.
This allowed for entering decimal, hexadecimal or octal as
input where previously the sscanf always interpreted the input
as hexadecimal. Additionally, -EIO was returned on error prior
to this change and now it is whatever the error value that is
returned by kstrtouint.
Change the base value of the kstrtouint from 0 to 16 and return
-EIO on error.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Reported-by: Joe Perches <joe@perches.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Checkpatch issued a warning preferring to use kstrto<type> when
using a single variable sscanf. Change the sscanf invocation to
a kstrtouint call.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Currently the driver makes use of the additional mac address
registers in the hardware to provide perfect filtering. The
hardware can also have a set of hash table registers that can
be used for imperfect filtering. By using imperfect filtering
the additional mac address registers can be used for layer 2
filtering support. Use the hash table registers if the device
has them.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch adds support for (imperfect) filtering of
VLAN tag ids using a 16-bit filter hash table. When
VLANs are added, a 4-bit hash is calculated with the
result indicating the bit in the hash table to set.
This table is used by the hardware to drop packets with
a VLAN id that does not hash to a set bit in the table.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
When receiving a VLAN packet check to be sure that VLAN
RX CTAG stripping is enabled before indicating that the
tag has been stripped in the packet information data
structure.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The MAC_VLAN_Incl register (0x0060) must be set to indicate
that the VLAN tag to be inserted comes from a Tx context
descriptor and not the MAC_VLAN_Incl register. Also, even
though it is the default, explicitly set the type of tag to
be inserted as a CTAG.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
In order to avoid conflicts with other include files, add
a prefix to the defines in xgbe.h.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
There is a typo here so we return directly instead of unwinding.
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
MAX_DMA_CHANNELS is defined in asm/scatterlist.h of the powerpc
architecture. Rename this #define in xgbe.h to avoid the
redefined warning issued during compilation.
Reported-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The include of asm/cputype.h breaks the powerpc build. This
include was accidentally left in from driver debugging and
can be removed.
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>,
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This patch provides the initial platform driver for the AMD
10GbE device.
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
