Hi Quang,
I'll go through on to what I've understand. There is an internal dc bias within the REF_CLK input which is 2V and this is specified in the datasheet. This is needed to satisfy the common mode voltage at the input and to avoid having negative voltage swings when it is AC coupled. To get the picture for the maximum limit, just imagine that you don't want to have a Vpeak input which is above 3.3 V. So at 1.5 Vpp which corresponds to 750 mVp, add that to 2V then you'll get around 3.25V and it is very close to 3.3V. For the minimum, I think it's the 0.632 Vpp. However, when I tried to get as low as -15 dBm, I still do get an output. But it is advisable to dwell far from the limits to avoid performance degradation.
Best Regards
Louijie