Kerala Governor Arif Mohammed Khan has written to Chief Minister Pinarayi Vijayan informing he has "ceased to enjoy pleasure" in Finance Minister K N Balagopal, accusing him of making speech undermining unity, and urged the CM to take constitutionally appropriate action against his cabinet colleague, a demand turned down by the latter.