h36m 数据集预处理

官网下载数据，文件夹列表

S1
├── MyPoseFeatures
│   ├── D2_Positions
│   ├── D3_Positions
│   └── D3_Positions_mono
└── Videos

预处理 video 得到 image folder 和 h36m_meta.mat
参考：https://github.com/mks0601/Integral-Human-Pose-Regression-for-3D-Human-Pose-Estimation

---

举例

joint world:

[[ -91.679     154.404     907.261   ]
 [-223.23566   163.80551   890.5342  ]
 [-188.4703     14.077106  475.1688  ]
 [-261.84055   186.55286    61.438915]
 [  39.877888  145.00247   923.98785 ]
 [ -11.675994  160.89919   484.39148 ]
 [ -51.550297  220.14624    35.834396]
 [-132.34781   215.73018  1128.8396  ]
 [ -97.1674    202.34435  1383.1466  ]
 [-112.97073   127.96946  1477.4457  ]
 [-120.03289   190.96477  1573.4     ]
 [  25.895456  192.35947  1296.1571  ]
 [ 107.10581   116.050285 1040.5062  ]
 [ 129.8381    -48.024918  850.94806 ]
 [-230.36955   203.17923  1311.9639  ]
 [-315.40536   164.55284  1049.1747  ]
 [-350.77136    43.442127  831.3473  ]
 [-102.237045  197.76935  1304.0605  ]]

R，旋转矩阵

 [[-0.91536173  0.40180837  0.02574754]
 [ 0.05154812  0.18037357 -0.98224649]
 [-0.39931903 -0.89778361 -0.18581953]]

t，平移向量

 [1841.10702775 4955.28462345 1563.4453959 ]

1.世界坐标系 -> 相机坐标系

# [R|t] world coords -> camera coords
joint_cam = np.zeros((joint_num, 3))  # joint camera
for i in range(joint_num):  # joint i
    joint_cam[i] = np.dot(R, joint_world[i] - T)  # R * (pt - T)

joint camera:

 [[-176.73076784 -321.04861816 5203.88206303]
 [ -52.96191047 -309.70448602 5251.08279041]
 [-155.64154642   73.07174899 5448.80703224]
 [ -29.83156629  506.78443787 5400.13835389]
 [-300.49984891 -332.39280376 5156.68125879]
 [-258.2404934    99.60903353 5244.68148685]
 [-209.48436358  548.83381157 5290.76368503]
 [-109.15762315 -529.72821331 5123.89062695]
 [-140.19118527 -780.12134477 5074.60479308]
 [-153.18189694 -886.97617334 5130.16531069]
 [-118.93483841 -970.22834498 5058.59901809]
 [-259.08998524 -690.13356164 5050.59206199]
 [-370.67088701 -448.59930158 5134.17727889]
 [-462.28660396 -290.82948094 5307.627479  ]
 [ -19.76034608 -716.91807819 5140.27254657]
 [  35.79160009 -470.14493959 5257.73846566]
 [  13.89246202 -279.85297007 5421.06857876]
 [-139.42516671 -703.52587972 5095.4322918 ]]

选择 Pelvis 所在位置 作为 相机中心，后面用之求 relative depth

center_cam = joint_cam[root_idx]  # (x,y,z) mm

[-176.73076784 -321.04861816 5203.88206303]

2.相机坐标系 -> 像素坐标系，并 get relative depth

# Subtract center depth
joint_img = np.zeros((joint_num, 3))
joint_img[:, 0], joint_img[:, 1], joint_img[:, 2] = cam2pixel(joint_cam, f, c)  # x,y
joint_img[:, 2] = joint_img[:, 2] - center_cam[2]  # z

cam2pixel 等价于：(f/dx) * (X/Z) = f * (X/Z) / dx 三角变换，/dx, + center_x

def cam2pixel(cam_coord, f, c):
    x = cam_coord[..., 0] / cam_coord[..., 2] * f[0] + c[0]
    y = cam_coord[..., 1] / cam_coord[..., 2] * f[1] + c[1]
    z = cam_coord[..., 2]

    return x, y, z

f，焦距，f/dx, f/dy

[1145.04940459 1143.78109572]

c，principal point，主点，主轴与像平面的交点，即下图 O₁

[512.54150496 515.45148698]

相机模型

joint image，像素坐标系，Depth 为相对深度 mm

[[ 473.65410166  444.88698613    0.        ]
 [ 500.99264799  447.99223133   47.20072737]
 [ 479.8339308   530.790273    244.9249692 ]
 [ 506.21599717  622.79141452  196.25629086]
 [ 445.81502802  441.72488139  -47.20080424]
 [ 456.16093657  537.17462304   40.79942382]
 [ 467.20401493  634.10084948   86.881622  ]
 [ 488.1477619   397.20284344  -79.99143608]
 [ 480.90833552  339.61748947 -129.27726995]
 [ 478.35140985  317.6982923   -73.71675234]
 [ 485.61976923  296.07674793 -145.28304494]
 [ 453.80169194  359.16055878 -153.29000105]
 [ 429.87266525  415.51348857  -69.70478414]
 [ 412.80937366  452.77842567  103.74541597]
 [ 508.13968145  355.92738829  -63.60951646]
 [ 520.33632979  413.17502388   53.85640262]
 [ 515.47589974  456.40583351  217.18651573]
 [ 481.20977587  357.52973363 -108.44977124]]