Yes, I am working together with a grad student who is trying to implement different designs. I theory everything is trivial but in reality being able to faithfully reproduce a desired pitch with proper noteons and noteoffs is more tricky because of various types of signal noise similar to what we experience in DSP (smoothing control data etc). I myself have no experience yet with either Python or mediapipe but I might have go get my feet wet to help the guy.
The aim is to build an interface which enables people who suffered a stroke to play my Thormulator SC app and thus train their weak side (typically only one side of the body is severely affected by the stroke) in an inspiring way. More about thormulator here:.
The weak arm could eg. be used to set the velocity of notes triggered by the strong arm/hand or for filtersweeps or other cc-like stuff. The important thing (from a medical point of view) is the training of the arm (bicep curls etc) and the hope is that a performance system (like Thormulator) can inspire and motivate the patients to actually get it done - just telling the patients to exercise proves to be quite difficult - they fall of the wagon quickly. Similar attempts have been tried with specialized video games, but apparently they quickly become too boring for the patients.