I'm building a browser based demo where a pre-made 3D character reacts to anything the user says.

When the visitor presses a "talk" button, the microphone audio is captured and audio passed to the character's mouth so the lips match the speech.
Live audio to lip sync of character.

Which could mean that the audio is analysed, phoneme / viseme data is extracted, and the character's blend shapes / mouth poses update in sync.
I'm open to whichever phoneme-detection approach you prefer so long as the lip sync looks believable.

Scope you'll handle
– Integrate Web Audio for capture.
– Implement or wire in the phoneme-mapping logic that converts the recorded clip to visemes and animates them on the character (you can choose your rig / model).
– Provide clean, well commented JavaScript (or TypeScript) with setup instructions so I can drop it into an existing page.

Acceptance

A single HTML page that:

• Loads the model.
• Lets me talk and watch the character lip-sync convincingly.

If you've done real time viseme work on the web before, please mention or show me links / demo.

Contract duration of less than 1 month. with 30 hours per week.

Mandatory skills:
JavaScript, WebGL, Animation, Web Application, 3D Animation, Augmented Reality, API, AR Application, Web Development