iFacialMocap allows 3rd party tool developers to receive facial data using UDP or TCP / IP protocols without going through PC software.
・ To receive data via UDP
If you send the string "iFacialMocap_sahuasouryya9218sauhuiayeta91555dy3719" from the PC side to port 49983 on the iOS side, iFacialMocap will return a reply to port 49983 with UDP at 60 FPS.
from socket import *
DstIP = "192.168.1.7"
DstPort = 49983
DstAddr = (DstIP,DstPort)
udpClntSock = socket(AF_INET, SOCK_DGRAM)
data = "iFacialMocap_sahuasouryya9218sauhuiayeta91555dy3719"
data = data.encode('utf-8')
server = socket(AF_INET, SOCK_DGRAM)
messages, address = server.recvfrom(8192)
・The character string sent per frame is as follows.
・The character strings sent per frame are as follows.
BlendShape Name-Parameters (0 ~ 100) | BlendShape Names-Parameters (0 ~ 100) | .... | = head # Euler angles X (degree), Euler angles Y, Euler angles Z, Position values X, Position values Y, Position values Z | rightEye #Euler angles X, Euler angles Y, Euler angles Z | leftEye #Euler angles X, Euler angles Y, Euler angles Z |
Angle-related data is sent in degrees, not radians.
・ To receive data via TCP / IP
If it is UDP, frames may be dropped, so there is also a method of receiving with TCP / IP.
If you send the character string "iFacialMocap_UDPTCP_sahuasouryya9218sauhuiayeta91555dy3719" to port 49983 on the iOS side by UDP, the data will be sent to port 49896 on the PC side.
The character string to be sent is just the character string "___iFacialMocap" added to the character string sent at the time of UDP.
BlendShape Name-Parameter (0 ~ 100) | BlendShape Name-Parameter (0 ~ 100) | .... | = head # Euler angles X, Euler angles Y, Euler angles Z, moving values X, moving values Y, moving values Z | rightEye #Euler angles X, Euler angles Y, Euler angles Z | leftEye #Euler angles X, Euler angles Y, Euler angles Z | ___ iFacialMocap
When receiving from iOS to PC with TCP / IP, a character string with broken frames will be sent, so use "___iFacialMocap" at the end as a delimiter.
To stop sending TCP / IP, send the string "iFacialMocap_UDPTCPSTOP_sahuasouryya9218sauhuiayeta91555dy3719" to port 49983.
・Sample Python usage example received over TCP / IP
from socket import socket, AF_INET, SOCK_DGRAM, SOCK_STREAM
HOST = ''
PORT = 49983
ADDRESS = "192.168.1.7" #iOS address
s = socket(AF_INET, SOCK_DGRAM)
msg = "iFacialMocap_UDPTCP_sahuasouryya9218sauhuiayeta91555dy3719"
s.sendto(msg.encode(), (ADDRESS, PORT))
#create tcp server
with socket(AF_INET, SOCK_STREAM) as tcp_server:
tcp_server.bind(('192.168.1.5', 49986)) #192.168.1.5 is pc address
conn, addr = tcp_server.accept()
data = conn.recv(10000)
・ To receive recorded data in iOS
If you send the character string "iFacialMocap_bakeRecordedAnimation_start" to port 49983 on the iOS side by UDP, the recorded data will be sent to port 49987 on the PC side by TCP / IP.
The string that is sent is in the format [".....", ".....", "....."] ___iFacialMocap.
The [".....", ".....", "....."] part is in JSON format and stores the same character strings used in UDP and TCP / IP.
Data recorded at 60 FPS is always stored. (Even if the processing drops to 30FPS in iOS, it will be recorded at 60FPS)
This isn't what iFacialMocap was supposed to do when it was first released, but 3rd party developers are free to use it. However, the development of iFacialMocap will gradually shift to another successor app, and the successor app will be treated like a higher version of iFacialMocap. However, iFacialMocap itself will not be deprecated. If you are using this feature, please note that it is uncertain how it will be handled by the successor app that will be released in the future. (Specifically, the price will be divided for each function in detail.)
・Run the lookForward button
Send the string "iFacialMocap_lookForward" to iOS port 49983.