iFacialMocap communication specifications


 iFacialMocap allows 3rd party tool developers to receive facial data using UDP or TCP / IP protocols without going through PC software.

 

 


・ To receive data via UDP

If you send the string "iFacialMocap_sahuasouryya9218sauhuiayeta91555dy3719" from the PC side to port 49983 on the iOS side, iFacialMocap will return a reply to port 49983 with UDP at 60 FPS.

 

・Python Example

from socket import *

DstIP = "192.168.1.7"

DstPort = 49983

DstAddr = (DstIP,DstPort)

udpClntSock = socket(AF_INET, SOCK_DGRAM)

data = "iFacialMocap_sahuasouryya9218sauhuiayeta91555dy3719"

data = data.encode('utf-8')

udpClntSock.sendto(data,DstAddr)

 

server = socket(AF_INET, SOCK_DGRAM)

server.bind(("", 49983))

server.settimeout(0.05)

 

while True:

 try:

  messages, address = server.recvfrom(8192)

  print(messages.decode('utf-8'))

 except:

  pass

 

・The character string sent per frame is as follows.

mouthSmile_R-0|eyeLookOut_L-0|mouthUpperUp_L-11|eyeWide_R-0|mouthClose-8|mouthPucker-4|mouthRollLower-9|eyeBlink_R-7|eyeLookDown_L-17|cheekSquint_R-11|eyeBlink_L-7|tongueOut-0|jawRight-0|eyeLookIn_R-6|cheekSquint_L-11|mouthDimple_L-10|mouthPress_L-4|eyeSquint_L-11|mouthRight-0|mouthShrugLower-9|eyeLookUp_R-0|eyeLookOut_R-0|mouthPress_R-5|cheekPuff-2|jawForward-11|mouthLowerDown_L-9|mouthFrown_L-6|mouthShrugUpper-26|browOuterUp_L-4|browInnerUp-20|mouthDimple_R-10|browDown_R-0|mouthUpperUp_R-10|mouthRollUpper-8|mouthFunnel-12|mouthStretch_R-21|mouthFrown_R-13|eyeLookDown_R-17|jawOpen-12|jawLeft-0|browDown_L-0|mouthSmile_L-0|noseSneer_R-18|mouthLowerDown_R-8|noseSneer_L-21|eyeWide_L-0|mouthStretch_L-21|browOuterUp_R-4|eyeLookIn_L-4|eyeSquint_R-11|eyeLookUp_L-0|mouthLeft-1|=head#-21.488958,-6.038993,-6.6019735,-0.030653415,-0.10287084,-0.6584072|rightEye#6.0297494,2.4403017,0.25649446|leftEye#6.034903,-1.6660284,-0.17520553|

 

・The character strings sent per frame are as follows.

BlendShape Name-Parameters (0 ~ 100) | BlendShape Names-Parameters (0 ~ 100) | .... | = head # Euler angles X (degree), Euler angles Y, Euler angles Z, Position values X, Position values Y, Position values Z | rightEye #Euler angles X, Euler angles Y, Euler angles Z | leftEye #Euler angles X, Euler angles Y, Euler angles Z |

 

Angle-related data is sent in degrees, not radians.

 

 


・ To receive data via TCP / IP

If it is UDP, frames may be dropped, so there is also a method of receiving with TCP / IP.

If you send the character string "iFacialMocap_UDPTCP_sahuasouryya9218sauhuiayeta91555dy3719" to port 49983 on the iOS side by UDP, the data will be sent to port 49896 on the PC side.

 

The character string to be sent is just the character string "___iFacialMocap" added to the character string sent at the time of UDP.

 

·sample

BlendShape Name-Parameter (0 ~ 100) | BlendShape Name-Parameter (0 ~ 100) | .... | = head # Euler angles X, Euler angles Y, Euler angles Z, moving values X, moving values Y, moving values Z | rightEye #Euler angles X, Euler angles Y, Euler angles Z | leftEye #Euler angles X, Euler angles Y, Euler angles Z | ___ iFacialMocap

 

When receiving from iOS to PC with TCP / IP, a character string with broken frames will be sent, so use "___iFacialMocap" at the end as a delimiter.

 

To stop sending TCP / IP, send the string "iFacialMocap_UDPTCPSTOP_sahuasouryya9218sauhuiayeta91555dy3719" to port 49983.

 

Sample Python usage example received over TCP / IP

from socket import socket, AF_INET, SOCK_DGRAM, SOCK_STREAM

 

#UDP send

HOST = ''

PORT = 49983

ADDRESS = "192.168.1.7" #iOS address

 

s = socket(AF_INET, SOCK_DGRAM)

msg = "iFacialMocap_UDPTCP_sahuasouryya9218sauhuiayeta91555dy3719"

s.sendto(msg.encode(), (ADDRESS, PORT))

 

s.close()

 

#create tcp server

with socket(AF_INET, SOCK_STREAM) as tcp_server:

tcp_server.bind(('192.168.1.5', 49986)) #192.168.1.5 is pc address

tcp_server.listen(1)

 

while True:

conn, addr = tcp_server.accept()

with conn:

while True:

data = conn.recv(10000)

print(data.decode('utf-8'))

 

 


・ To receive recorded data in iOS

If you send the character string "iFacialMocap_bakeRecordedAnimation_start" to port 49983 on the iOS side by UDP, the recorded data will be sent to port 49987 on the PC side by TCP / IP.

 

The string that is sent is in the format [".....", ".....", "....."] ___iFacialMocap.

The [".....", ".....", "....."] part is in JSON format and stores the same character strings used in UDP and TCP / IP.

 

Data recorded at 60 FPS is always stored. (Even if the processing drops to 30FPS in iOS, it will be recorded at 60FPS)

 

 

 

This isn't what iFacialMocap was supposed to do when it was first released, but 3rd party developers are free to use it. However, the development of iFacialMocap will gradually shift to another successor app, and the successor app will be treated like a higher version of iFacialMocap. However, iFacialMocap itself will not be deprecated. If you are using this feature, please note that it is uncertain how it will be handled by the successor app that will be released in the future. (Specifically, the price will be divided for each function in detail.)


・Run the lookForward button

 

Send the string "iFacialMocap_lookForward" to iOS port 49983.