Although I2C has been used to route signals from one card or PCB to another, the original intent was to keep any given I2C bus on one PCB. That said, it is indeed possible to take I2C lines from one card to another if proper care is taken. One of the things you already seem to have is that data speeds need to be kept low. This is because the pull-up resistors work with the line capacitors to limit slew rates. You can improve this by using the lowest value resistors (using more power) as practical, consistent with keeping currents below the maximum allowed for the drivers and keeping lines as narrow and short as possible, consistent with good impedance matching and signal integrity practices.
It is possible to use the NXP devices you mentioned, but using an all ADI approach is also possible, if that is what your customer wants. You can use four 16-channel single ended analog multiplexers (such as the ADG1606) or four 8-channel differential multiplexers (such as the ADG1607) for this task. My personal choice would be to use four 16-channel single-ended devices because it simplifies the addressing of these devices.
Two of the 16-channel single-ended multiplexers would be for the 32 SDA lines and two would handle the SCL lines. The multiplexers for the SDA lines would have address lines A0 connected together, lines A1 connected together, and lines A2 connected together. One of these multiplexers will have an logic inverter from the the processor address to its address line A3 and the other would not have this inverter. One of the SCL multiplexers would have its address lines connected directly to one of the SDA multiplexers and the other SCL multiplexer would have its address lines connected directly to the other SDA multiplexer. I think the rest of the connections are obvious.
Nick