After some trial and error today,
the TX Quadrature calibration converges when I set the bandwidth to 10Mhz and force the
Rx NCO Frequency=Tx NCO Frequency=0.
If the bandwidth is set to 1MHz, under the same conditions, calibration fails to converge.
According to the equations in the register map manual, my clkRF=clkTF=12.288MHz, so the NCO Test Frequency = 12.288M/32=384Khz
Why do I need such a wide filter for the calibration? why 1Mhz or even 0.5Mhz is not enough?
once again, what do I miss?
How does the RX gain affect the TX quad calibration? should it be manually set to some constant value?