How to make eye blinking

[0724veni]

Thanks for great work. There is a lot of help for me
Although I saw other issue about eye blinking, I don't implement eye blinking.
What you said is removing eye parts of mesh (the triangles) that correspond to the eye regions.

1. how to remove eye parts of mesh in code?
   What i know is

• len(mesh) = 175
  mean3DShape, blendshapes, mesh, idxs3D, idxs2D = utils.load3DFaceModel("candide.npz")
• mesh means 'FACE LIST' in candide
  What you mentioned

I don't know where eye parts is in 175. So could you give me advice? Could you give me edidted candide.npz model which remove eye parts?

2. len(mesh) is differnt from 'FACE LIST' in candide3.wfm. why is it??
   This candide3.wfm is the latest version of candide model(v3.1.6)
   len(mesh) = 175
   'FACE LIST' in candide3.wfm = 184
   What do you think about?
   Maybe are you deleting some parts in 'FACE LIST'?
   What you said about model is 'processed version'

3. Your Code doesn't work transition between .txt to .npz.

• transition code
• .txt
  Is transition code for your 'processed version' candide model?
• error
  Traceback (most recent call last):
  File "candide.py", line 192, in
  tempMode = np.zeros((1, mean3.shape[0], mean3.shape[1]))
  IndexError: tuple index out of range

[MarekKowalski]

Hi,
sorry for the massive delay with answering.
The way to find out which faces (triangles) correspond to eyes would be to save the mean3DShape with the mesh topology to a .ply or .obj file. You would then open that file in something like meshlab and check triangle indices. You can easily save to .obj and .ply using packages like trimesh. Writing an obj writer yourself is very easy as well. I am trying to find an .obj file for candide on my old computer.

I think I removed some triangles that corresponded to forehead and maybe sides of the face, hence len(mesh) < official number of triangles.

I think that the .txt file I used in the transition code was the original candide. The processed version refers to removing the triangles as mentioned above. That would not be required until line .. of the script you linked.
The error you got seems to suggest that mean3 (the mean 3D shape) does not load correctly, not sure why.

[MarekKowalski]

I found an .obj file on my old machine that should correspond to the mesh used in the app, it is attached. As mentioned above you can open it in meshlab and easily check which triangle is which. You can then remove the corresponding rows from the mesh matrix in code to remove the triangles from the face swap.

candideInterior3.zip

[ZhangLei999]

The eye area is deleted by use meshlab. And save the result.obj as candideInterior3.obj. Then I remove the corresponding rows from the mesh matrix in code like this:
mean3DShape, blendshapes, _, idxs3D, idxs2D = utils.load3DFaceModel( os.path.join(os.path.dirname(file), "..", "candide2.npz"))
_, mesh = loadOBJ("./candideInterior3.obj")
It doesn't work.

[MarekKowalski]

Hi, can you explain what exactly does not work?

[ZhangLei999]

After face-swap, the expression on my face was distorted. And the eyes were still not closed.

[MarekKowalski]

I see, one problem may be that the mesh variable you load does not assume the same number of points in the mesh or the same order of the points as the mesh that is loaded from the .npz file. I suggest that this is something you may want to investigate.

Perhaps it would be easier if you did as I suggested above and simply delete the rows of the original mesh that correspond to the eyes. This way you can be sure that you are modifying the right thing. To find out which triangles correspond to the eyes you can try visualizing the mesh from the Python app itself with consecutive triangles removed until you find which ones are covering the eyes.