How is picking in an orthogonal projection supposed to be done
The pick example from the sdk works only in PerspectiveFov projection, but not in Ortho projection. It seems to be all in the calculation of the pick-ray vector, but I can't find out how to set it :/
Tnx!
It's the same as for perspective. Basically all you need to do is "unproject" the endpoints of a screen space line segment and use the result to test for mesh triangle intersections. You just need to make sure your resulting segment is in the same space as the mesh you're testing it against.
To unproject a screen space point, build the matrix that takes world points to screen space, then invert it. The total matrix to screen space is something like
Viewport * Projection/w * View * World
(The viewport transform normally isn't presented as a matrix.)
Anyway, if you have an "Unproject" function that inverts that matrix, then your world space line segment corresponding to your mouse click would be Unproject(x,y,1) - Unproject(x,y,0), where x, y, and z are all in screen space. Then you can plow through the mesh triangles with that result.
Hope that helps.
Thanks!
Well I'm doing something like this, but when some transformations are applied, the picking is not accurate in Orthogonal projection. However, the PerspectiveFOV is perfect.
That's Exactly what I'm doing:
D3DD.GetViewport(VP);
D3DD.GetTransform(D3DTS_PROJECTION, PM);
D3DD.GetTransform(D3DTS_VIEW, VM);
D3DXVec3Unproject(VOut1,
D3DXVector3(X, Y, 0.0),
VP,
PM, VM, WorldMatrix);
D3DXVec3Unproject(VOut2,
D3DXVector3(X, Y, 1.0),
VP,
PM, VM, WorldMatrix);
//WorldMatrix is the world transformation matrix for the tested mesh
//The same is if it is got like this: D3DD.GetTransform(D3DTS_WORLD, VM);
//I test with the IntersectTriangle function as it is in the Picking demo from the DXSDK. D3DXIntersect function does the same result.
IntersectTriangle(VOut1,
VOut2,
v0, v1, v2,
fDist, fBary1, fBary2)
Sorry for reviving this thread, but there seems to be immeasurably difficult to find a woking piece of code for ortho-picking. I found people that had the exactly same problem, but found no answer :/ I still need this, and I think it would be nice and very helpful to have a practically working solution here. Well has anyone some working code
Your problem is that the IntersectTriangle function (as well as the D3DXIntersect and Mesh.Intersect functions) requires a ray for input, not a line segment. So the second parameter should be changed from VOut2 to Vector3.Subtract(VOut2, VOut1).
I had the same problem when adding a Picking function to a program with Perspective Projection. All the examples I found in the forums incorrectly passed line segments instead of rays to the Intersect methods, so it took me a while to figure that out.
Hope that helps. 