Before getting into the actual question, I'd like to recommend creating a new support subforum for math since it's so important for game development.
OK, I'm helping with the imposter system for the terrain, and they're already working correctly, the problem is, they're y-aligned at the moment. I'm passing the angle on y to the shader and calculating the UV to pick based on the atan2 of the imposter-camera vector on xz plane (horizontal) for the tile on the uvx (yaw), and for the tile on the uvy (pitch) I'm simply using the y of the normalized imposter-camera vector, which represents the proportion between the distance of the vector on the xz plane to the distance on y (basically
I'm doing those calculations to determine which tile to pick on the shader but since we're creating the billboards geometry on the CPU for each frame anyway, we might as well go full-CPU to make things easier, here's the shader code I'm using:
float tileMinX = floor(((atan(treeCamVec.x, treeCamVec.z) / PI + 1) / 2) * numTilesX) / numTilesX;
float tileMinY = (floor((((normalize(treeCamVec).y) + 1) / 2) * (numTilesY + 1)) - 1) / (numTilesY + 1) * (tileHeight / (1f / (numTilesY + 1)));
tileMinX and tileMinY represents the 'min' coordinates of the UV rectangle. treeCamVec is the imposter-camera vector. Other vars should be self-explanatory. "Tile" stands for the texture tile. Result:
So far it's easy enough to determine which image to use. The problem is, I'll have to change the billboard generation code anyway because the current one isn't suitable for arbitrarily anchored objects, since they are rotated vertically along their centers so for example a tree's roots will pan around on the ground. Since I'm gonna get my hands dirty anyway , I would also like to get rid of the y-alignment limitation, allowing the object to have total freedom of rotation, so you can have tilted trees for example. However, I don't know how I could do that. I think it's easier to create the imposter on the CPU, but I could also to pass the quaternion to the shader, the problem is the math behind it. I think the easiest way is to use a world-to-local transformation matrix for each imposter to convert the camera position and determine the angle just like the code above, and then convert the vertices back to world space, but I'm not sure if there's a better way to achieve that. Passing an affine transformation matrix to the shader certainly is something to be avoided.
If you guys have any considerations or links to articles on how to properly create freely rotated billboards please let me know. I've googled but I've only found either y-aligned on global with pitch OR local y-aligned that can be tilted but doesn't have pitch image angles. Both are easy to do, I'm looking for the hard stuff .