Hi Shivam,
My colleague just answered a very similar question over on this thread (possibly from someone else at your company, I believe). Could I ask you to take a look at his inputs on that thread, and provide me with similar additional information.
Two other specific suggestions I would make, are to try changing the Boot Mode (SW3) to another source/PLL configuration, and power cycle the board to see if this affects your ability to connect (don't sure SPI Host Slave Mode, or you will not be able to connect to your BF561 target at all). Please also check the VDDINT jumper settings (JP2 and JP3), try changing these to use the internal/external regulator to see if this affects your ability to connect.
For more information on these switches and jumpers, please see the ADSP-BF561 EZ-KIT Lite Evaluation System Manual.
The BF561 EZ-KIT Lites do get quite warm, in my experience, so the operating temperature may not be related to the current behaviour.
Regards,
Craig.