Technologies and devices for light field imaging have recently been developed for both industrial applications and scientific research to achieve excellent imaging properties. In our previous work, we combined light field imaging with structured illumination to propose a structured light field method in which multidirectional depth estimation can be performed for high-quality 3D imaging. However, the projection axis was implicitly assumed to be perpendicular to the reference plane, which is hard to meet in practice. In this paper, we derive a universal phase-depth mapping in a structured light field by relaxing this implicit condition. Both nonlinear and linear models were proposed based on this universal relationship. To test the model's practical performance, we simulated experiments by adding errors to the real measured values to evaluate the deviation in depth estimation. By comparing the root-mean-square distributions of the depth deviations with respect to the depth positions, we demonstrated that the nonlinear model was precise and consistent in a wide range of depth, and we employed this model to realize high-quality multidirectional scene reconstruction.