I contacted an expert Mr Venky Regulatory Specialist, Crop protection, he suggested to spray wood ash on the problem areas and on the plants to protect from these pests. These are kind of beetles, called bean leaf beetles, they are not friendly.
From what I have read it appears that neem oil is fairly effective at the larval stage, so please persist with it for a few more lifecycles of the pest.
Biological control seems to be limited but Chestnut leaf Trumpet bush plants (Image at researchgate.net/publication … Tachinidae_from_Kolhapur_and_Satara_districts_India