linux_dsm_epyc7002/Documentation/i2c/i2c-topology.rst

================================
I2C muxes and complex topologies
================================

There are a couple of reasons for building more complex I2C topologies
than a straight-forward I2C bus with one adapter and one or more devices.

1. A mux may be needed on the bus to prevent address collisions.

2. The bus may be accessible from some external bus master, and arbitration
   may be needed to determine if it is ok to access the bus.

3. A device (particularly RF tuners) may want to avoid the digital noise
   from the I2C bus, at least most of the time, and sits behind a gate
   that has to be operated before the device can be accessed.

Etc
===

These constructs are represented as I2C adapter trees by Linux, where
each adapter has a parent adapter (except the root adapter) and zero or
more child adapters. The root adapter is the actual adapter that issues
I2C transfers, and all adapters with a parent are part of an "i2c-mux"
object (quoted, since it can also be an arbitrator or a gate).

Depending of the particular mux driver, something happens when there is
an I2C transfer on one of its child adapters. The mux driver can
obviously operate a mux, but it can also do arbitration with an external
bus master or open a gate. The mux driver has two operations for this,
select and deselect. select is called before the transfer and (the
optional) deselect is called after the transfer.


Locking
=======

There are two variants of locking available to I2C muxes, they can be
mux-locked or parent-locked muxes. As is evident from below, it can be
useful to know if a mux is mux-locked or if it is parent-locked. The
following list was correct at the time of writing:

In drivers/i2c/muxes/:

======================    =============================================
i2c-arb-gpio-challenge    Parent-locked
i2c-mux-gpio              Normally parent-locked, mux-locked iff
                          all involved gpio pins are controlled by the
                          same I2C root adapter that they mux.
i2c-mux-gpmux             Normally parent-locked, mux-locked iff
                          specified in device-tree.
i2c-mux-ltc4306           Mux-locked
i2c-mux-mlxcpld           Parent-locked
i2c-mux-pca9541           Parent-locked
i2c-mux-pca954x           Parent-locked
i2c-mux-pinctrl           Normally parent-locked, mux-locked iff
                          all involved pinctrl devices are controlled
                          by the same I2C root adapter that they mux.
i2c-mux-reg               Parent-locked
======================    =============================================

In drivers/iio/:

======================    =============================================
gyro/mpu3050              Mux-locked
imu/inv_mpu6050/          Mux-locked
======================    =============================================

In drivers/media/:

=======================   =============================================
dvb-frontends/lgdt3306a   Mux-locked
dvb-frontends/m88ds3103   Parent-locked
dvb-frontends/rtl2830     Parent-locked
dvb-frontends/rtl2832     Mux-locked
dvb-frontends/si2168      Mux-locked
usb/cx231xx/              Parent-locked
=======================   =============================================


Mux-locked muxes
----------------

Mux-locked muxes does not lock the entire parent adapter during the
full select-transfer-deselect transaction, only the muxes on the parent
adapter are locked. Mux-locked muxes are mostly interesting if the
select and/or deselect operations must use I2C transfers to complete
their tasks. Since the parent adapter is not fully locked during the
full transaction, unrelated I2C transfers may interleave the different
stages of the transaction. This has the benefit that the mux driver
may be easier and cleaner to implement, but it has some caveats.

==== =====================================================================
ML1. If you build a topology with a mux-locked mux being the parent
     of a parent-locked mux, this might break the expectation from the
     parent-locked mux that the root adapter is locked during the
     transaction.

ML2. It is not safe to build arbitrary topologies with two (or more)
     mux-locked muxes that are not siblings, when there are address
     collisions between the devices on the child adapters of these
     non-sibling muxes.

     I.e. the select-transfer-deselect transaction targeting e.g. device
     address 0x42 behind mux-one may be interleaved with a similar
     operation targeting device address 0x42 behind mux-two. The
     intension with such a topology would in this hypothetical example
     be that mux-one and mux-two should not be selected simultaneously,
     but mux-locked muxes do not guarantee that in all topologies.

ML3. A mux-locked mux cannot be used by a driver for auto-closing
     gates/muxes, i.e. something that closes automatically after a given
     number (one, in most cases) of I2C transfers. Unrelated I2C transfers
     may creep in and close prematurely.

ML4. If any non-I2C operation in the mux driver changes the I2C mux state,
     the driver has to lock the root adapter during that operation.
     Otherwise garbage may appear on the bus as seen from devices
     behind the mux, when an unrelated I2C transfer is in flight during
     the non-I2C mux-changing operation.
==== =====================================================================


Mux-locked Example
------------------


::

                   .----------.     .--------.
    .--------.     |   mux-   |-----| dev D1 |
    |  root  |--+--|  locked  |     '--------'
    '--------'  |  |  mux M1  |--.  .--------.
                |  '----------'  '--| dev D2 |
                |  .--------.       '--------'
                '--| dev D3 |
                   '--------'

When there is an access to D1, this happens:

 1. Someone issues an I2C transfer to D1.
 2. M1 locks muxes on its parent (the root adapter in this case).
 3. M1 calls ->select to ready the mux.
 4. M1 (presumably) does some I2C transfers as part of its select.
    These transfers are normal I2C transfers that locks the parent
    adapter.
 5. M1 feeds the I2C transfer from step 1 to its parent adapter as a
    normal I2C transfer that locks the parent adapter.
 6. M1 calls ->deselect, if it has one.
 7. Same rules as in step 4, but for ->deselect.
 8. M1 unlocks muxes on its parent.

This means that accesses to D2 are lockout out for the full duration
of the entire operation. But accesses to D3 are possibly interleaved
at any point.


Parent-locked muxes
-------------------

Parent-locked muxes lock the parent adapter during the full select-
transfer-deselect transaction. The implication is that the mux driver
has to ensure that any and all I2C transfers through that parent
adapter during the transaction are unlocked I2C transfers (using e.g.
__i2c_transfer), or a deadlock will follow. There are a couple of
caveats.

==== ====================================================================
PL1. If you build a topology with a parent-locked mux being the child
     of another mux, this might break a possible assumption from the
     child mux that the root adapter is unused between its select op
     and the actual transfer (e.g. if the child mux is auto-closing
     and the parent mux issues I2C transfers as part of its select).
     This is especially the case if the parent mux is mux-locked, but
     it may also happen if the parent mux is parent-locked.

PL2. If select/deselect calls out to other subsystems such as gpio,
     pinctrl, regmap or iio, it is essential that any I2C transfers
     caused by these subsystems are unlocked. This can be convoluted to
     accomplish, maybe even impossible if an acceptably clean solution
     is sought.
==== ====================================================================


Parent-locked Example
---------------------

::

                   .----------.     .--------.
    .--------.     |  parent- |-----| dev D1 |
    |  root  |--+--|  locked  |     '--------'
    '--------'  |  |  mux M1  |--.  .--------.
                |  '----------'  '--| dev D2 |
                |  .--------.       '--------'
                '--| dev D3 |
                   '--------'

When there is an access to D1, this happens:

 1.  Someone issues an I2C transfer to D1.
 2.  M1 locks muxes on its parent (the root adapter in this case).
 3.  M1 locks its parent adapter.
 4.  M1 calls ->select to ready the mux.
 5.  If M1 does any I2C transfers (on this root adapter) as part of
     its select, those transfers must be unlocked I2C transfers so
     that they do not deadlock the root adapter.
 6.  M1 feeds the I2C transfer from step 1 to the root adapter as an
     unlocked I2C transfer, so that it does not deadlock the parent
     adapter.
 7.  M1 calls ->deselect, if it has one.
 8.  Same rules as in step 5, but for ->deselect.
 9.  M1 unlocks its parent adapter.
 10. M1 unlocks muxes on its parent.


This means that accesses to both D2 and D3 are locked out for the full
duration of the entire operation.


Complex Examples
================

Parent-locked mux as parent of parent-locked mux
------------------------------------------------

This is a useful topology, but it can be bad::

                   .----------.     .----------.     .--------.
    .--------.     |  parent- |-----|  parent- |-----| dev D1 |
    |  root  |--+--|  locked  |     |  locked  |     '--------'
    '--------'  |  |  mux M1  |--.  |  mux M2  |--.  .--------.
                |  '----------'  |  '----------'  '--| dev D2 |
                |  .--------.    |  .--------.       '--------'
                '--| dev D4 |    '--| dev D3 |
                   '--------'       '--------'

When any device is accessed, all other devices are locked out for
the full duration of the operation (both muxes lock their parent,
and specifically when M2 requests its parent to lock, M1 passes
the buck to the root adapter).

This topology is bad if M2 is an auto-closing mux and M1->select
issues any unlocked I2C transfers on the root adapter that may leak
through and be seen by the M2 adapter, thus closing M2 prematurely.


Mux-locked mux as parent of mux-locked mux
------------------------------------------

This is a good topology::

                   .----------.     .----------.     .--------.
    .--------.     |   mux-   |-----|   mux-   |-----| dev D1 |
    |  root  |--+--|  locked  |     |  locked  |     '--------'
    '--------'  |  |  mux M1  |--.  |  mux M2  |--.  .--------.
                |  '----------'  |  '----------'  '--| dev D2 |
                |  .--------.    |  .--------.       '--------'
                '--| dev D4 |    '--| dev D3 |
                   '--------'       '--------'

When device D1 is accessed, accesses to D2 are locked out for the
full duration of the operation (muxes on the top child adapter of M1
are locked). But accesses to D3 and D4 are possibly interleaved at
any point. Accesses to D3 locks out D1 and D2, but accesses to D4
are still possibly interleaved.


Mux-locked mux as parent of parent-locked mux
---------------------------------------------

This is probably a bad topology::

                   .----------.     .----------.     .--------.
    .--------.     |   mux-   |-----|  parent- |-----| dev D1 |
    |  root  |--+--|  locked  |     |  locked  |     '--------'
    '--------'  |  |  mux M1  |--.  |  mux M2  |--.  .--------.
                |  '----------'  |  '----------'  '--| dev D2 |
                |  .--------.    |  .--------.       '--------'
                '--| dev D4 |    '--| dev D3 |
                   '--------'       '--------'

When device D1 is accessed, accesses to D2 and D3 are locked out
for the full duration of the operation (M1 locks child muxes on the
root adapter). But accesses to D4 are possibly interleaved at any
point.

This kind of topology is generally not suitable and should probably
be avoided. The reason is that M2 probably assumes that there will
be no I2C transfers during its calls to ->select and ->deselect, and
if there are, any such transfers might appear on the slave side of M2
as partial I2C transfers, i.e. garbage or worse. This might cause
device lockups and/or other problems.

The topology is especially troublesome if M2 is an auto-closing
mux. In that case, any interleaved accesses to D4 might close M2
prematurely, as might any I2C transfers part of M1->select.

But if M2 is not making the above stated assumption, and if M2 is not
auto-closing, the topology is fine.


Parent-locked mux as parent of mux-locked mux
---------------------------------------------

This is a good topology::

                   .----------.     .----------.     .--------.
    .--------.     |  parent- |-----|   mux-   |-----| dev D1 |
    |  root  |--+--|  locked  |     |  locked  |     '--------'
    '--------'  |  |  mux M1  |--.  |  mux M2  |--.  .--------.
                |  '----------'  |  '----------'  '--| dev D2 |
                |  .--------.    |  .--------.       '--------'
                '--| dev D4 |    '--| dev D3 |
                   '--------'       '--------'

When D1 is accessed, accesses to D2 are locked out for the full
duration of the operation (muxes on the top child adapter of M1
are locked). Accesses to D3 and D4 are possibly interleaved at
any point, just as is expected for mux-locked muxes.

When D3 or D4 are accessed, everything else is locked out. For D3
accesses, M1 locks the root adapter. For D4 accesses, the root
adapter is locked directly.


Two mux-locked sibling muxes
----------------------------

This is a good topology::

                                    .--------.
                   .----------.  .--| dev D1 |
                   |   mux-   |--'  '--------'
                .--|  locked  |     .--------.
                |  |  mux M1  |-----| dev D2 |
                |  '----------'     '--------'
                |  .----------.     .--------.
    .--------.  |  |   mux-   |-----| dev D3 |
    |  root  |--+--|  locked  |     '--------'
    '--------'  |  |  mux M2  |--.  .--------.
                |  '----------'  '--| dev D4 |
                |  .--------.       '--------'
                '--| dev D5 |
                   '--------'

When D1 is accessed, accesses to D2, D3 and D4 are locked out. But
accesses to D5 may be interleaved at any time.


Two parent-locked sibling muxes
-------------------------------

This is a good topology::

                                    .--------.
                   .----------.  .--| dev D1 |
                   |  parent- |--'  '--------'
                .--|  locked  |     .--------.
                |  |  mux M1  |-----| dev D2 |
                |  '----------'     '--------'
                |  .----------.     .--------.
    .--------.  |  |  parent- |-----| dev D3 |
    |  root  |--+--|  locked  |     '--------'
    '--------'  |  |  mux M2  |--.  .--------.
                |  '----------'  '--| dev D4 |
                |  .--------.       '--------'
                '--| dev D5 |
                   '--------'

When any device is accessed, accesses to all other devices are locked
out.


Mux-locked and parent-locked sibling muxes
------------------------------------------

This is a good topology::

                                    .--------.
                   .----------.  .--| dev D1 |
                   |   mux-   |--'  '--------'
                .--|  locked  |     .--------.
                |  |  mux M1  |-----| dev D2 |
                |  '----------'     '--------'
                |  .----------.     .--------.
    .--------.  |  |  parent- |-----| dev D3 |
    |  root  |--+--|  locked  |     '--------'
    '--------'  |  |  mux M2  |--.  .--------.
                |  '----------'  '--| dev D4 |
                |  .--------.       '--------'
                '--| dev D5 |
                   '--------'

When D1 or D2 are accessed, accesses to D3 and D4 are locked out while
accesses to D5 may interleave. When D3 or D4 are accessed, accesses to
all other devices are locked out.