Full affine transformation (shears included) for sprites

[ruifengx]

I implemented some 2D animation using Bevy, which works (or rather, should work) by manipulating Transformss in SpriteBundles. I was really surprised to find out (after some painful debugging, and finally using bevy_prototype_lyon to draw out all the bounding boxes for all my Sprites) that all the bounding boxes are rectangular, instead of being sheared polyhedrons. It seems the current encoding of Transform and GlobalTransform just silently drops the shearing during Transform::from_matrix or during Transform propagation...

Is there any API I am currently missing which provides support for shearing (or is it actually supported and I have misused the API)? If not, is there currently any workaround for this problem, or will there be a solution in foreseeable future?

Thanks in advance.

P.S. I noticed #1755 and #4379, but I think it is better to ask here and get an "official" answer, rather than to draw some random conclusions from scattered information.

Edit: 2022-05-22

I managed to achieve what I want (sort of) by patching bevy locally with modified version of #4379 (merged into latest/0.7.0 and modified transform_propagate_system to allow using full-fledged Affine3A as local transforms). That said, my implementation is mostly a dirty hack, and hopefully in near future relevant PRs will land in Bevy official.

P.S. What is the correct way to linear interpolate two Affine3As? I don't see a Affine3A::lerp method, and naive linear combination does not look quite right.

[HackerFoo]

P.S. What is the correct way to linear interpolate two Affine3As? I don't see a Affine3A::lerp method, and naive linear combination does not look quite right.

A matrix is a mixture of rotation and scaling, so rotating 180 degrees the naive way (interpolating each axis in the matrix) will shrink to nothing then expand to the final position. It's easy to visualize if you look at the columns as the XYZ vectors of the space being transformed. It's not wrong, it just isn't how things typically move.

If you want something more like a rigid motion, you have to separate the matrix into rotation and scaling, which can be done with polar decomposition, e.g. using nalgebra's Matrix::try_polar.

This operation is kind of expensive, so I use my own transform type that has translation (Vec3), rotation (Quat), and scaling as a Mat3. Even though the scaling matrix could represent rotation, all rotation is done using quaternions, leaving the matrix for scaling in any direction. I think you could just use the upper or lower triangle of the matrix, saving 3 f32s, but I haven't tried this yet - it might be slow. This way, I only need polar decomposition when converting from Affine3A to my own transform.

Anyway, interpolation on this representation (using spherical interpolation for rotation) works as expected, minimizing stretching during interpolation.

This is for 3D, so for 2D it would be a Vec2 translation, scalar rotation, Mat2 scale.

[ruifengx]

Thanks for your answer! The SRT encoding with a Mat2 scale indeed looks very reasonable for this specific task. However, it still remains unclear to me how exactly I can integrate such new Transform into Bevy. If my understanding is correct, the use of GlobalTransform is hard-coded in bevy_render, so to have things eventually rendered by bevy, I must encode the transformation somehow as a GlobalTransform component. (But I believe that is not possible in general?) I see you said

so I use my own transform type

If I may ask, are you also working with a patched bevy_transform, or is there any non-intrusive way to achieve this?

[HackerFoo]

I made a system that reads my transform component, converts it, and writes the result to GlobalTransform. You can still use bevy_transform, there won't be a conflict unless an entity also has a Transform component, and you update that component. I made a PbrBundleWithoutTransform by copying PbrBundle except Transform, but this isn't strictly necessary, it just saves space.

You could make your own transform propagation system like bevy_transform, but in my case, I don't have a complicated hierarchy, just a global space and a "stage", which has its transformation stored in a resource. I use a marker trait to determine what should use stage space. I think a custom system like this is easier to work with.

[ruifengx]

Thanks for the explanation. One last thing to confirm: using your fork of Bevy would still be necessary in order to encode arbitrary Affine3A in GlobalTransform (perhaps until it is merged, or the discussion at #4213 is settled), am I correct?