Most of the Rohingya refugees who had taken shelter in West Bengal have left allegedly out of fear of deportation and the only existing camp in South 24 Parganas district has six of its 29 inmates left, a spokesman of the organisation which had arranged their stay said on Monday.