The visitor should not sit for too long with the sick person, rather the visit should be short so that it does not cause any hardship to him or his family. The sick person may pass through periods when he suffers pain because of his sickness, or he may do something that he would not like anyone to see, so sitting with him for too long will cause him embarrassment. 

However, it depends on the situation; the sick person may like some people to sit with him for a long time. 

Haashiyat Ibn Qaasim, 3/12; al-Sharh al-Mumti’, 5/174