As u had mentioned u fear for digging an open well further, Manually, as your well Side walls have become soft & as such it is risky to pursue manually,
I would suggest that, If have a good sustainable yield from your existing open Well, u can, Install the Round shape precast RCC rings of 3ft / 4ft diameter as per the Diameter / Width & Length of the Open Well, with the round stone aggregate around the Rings upto the Static / Stayable Water Level in your well. & Place a screened / slotted PVC pipe submerged into the water bearing zone & plain above, Just like a Straw in the Glass (refer to the attached image).
This would save u any risky expenditure, because, as your well walls are in collapsable condition, further excavating or cleaning will yield you nothing, as it would again get refilled when walls collapse.
May u have any further queries? plz do not hesitate.