How Not To Make A 2D Mirror
Posted On: 2020-12-21
Earlier this month, I'd planned to implement and write about making an in-game mirror for a 2D world. I had a simple, elegant design planned, and I was confident that everything would work out well - which, of course, is not at all what happened. After a number of failed attempts, I've put the idea on ice: the benefits* do not justify continuing to spend time on it. Many of the lessons I learned along the way, however, were surprising, so I will continue with my plan to share what I've done - albeit with a focus on all the things that didn't work.
The Plan
I'd intended to create a simple mirror for use in a 2D world by using a Render Texture. I'd used render textures previously, to fake a cone of light, and had learned from that experience that it's relatively simple to use and can fake all kinds of much more complex visual effects. For this particular application, I expected to create a plane that would face the main camera, and, using a render texture, show the player an exact copy of what they saw - just fed through a different camera (which was attached to the main camera.)
Careful With the UVs
The first issue I ran into was one that had plagued my cone of light, but I had forgotten in the interim: if they aren't set correctly, the UVs * of the render texture will distort the image - like some kind of funhouse mirror. For the cone of light, I'd procedurally generated the mesh, and its UVs, so I could take the camera's properties (such as aspect ratio) into account to make sure it fit perfectly. Doing something similar with the mirror might be possible - even with a pre-authored mesh, but there were other issues that led to cutting it before fully investigating that.
Using a plane with the default UVs, the camera's entire viewport is squashed/stretched to fit onto the plane
Z-index vs Sort Order
The mirror was a background element, facing the camera. As such, the player would expect objects that are closer to the mirror would occlude objects that are closer to the camera. Unfortunately, since the camera rendering to the texture was, in fact, on the same side as the main camera, that was not at all what happened. Instead, objects would display in the exact same order as they appeared to the camera - completely breaking the illusion of it being a reflection.
Players would expect something like this
But the mirror instead shows something like this
This issue could be partially mitigated by moving the mirror's camera so that it was on the opposite side, looking towards the main camera. This also required adjusting the UVs (to flip the image horizontally) but doing so would correctly render objects based on their position relative to the camera. Unfortunately, it did nothing to help with rendering objects that make use of the Sorting Layer feature - a feature I use to manage the render order of virtually every object in the game.
The sorting layer issue is, so far as I could tell, unsolvable. From what I've been able to find, the order objects are rendered by every camera appears to be controlled by Unity's sprite renderer, not the actual sprite shader, so it's not possible to work around it using replacement shaders. What's more, Unity's sprite renderer is both a black box* and likely going to change/improve with Unity version updates, so replicating its behavior manually is not a realistic option. The only reasonable workaround appears to be eliminating the use of sorting layers, and instead consistently relying on z-index - which I might consider, if the mirror didn't have yet more problems.
Always Forward-facing
Generally, every character sprite is facing the camera somewhat, regardless of the direction they are facing. Unfortunately, this means that, in the reflection, the player would see the character's face, instead of their back. It might be possible to work around this by creating new sprites that would only display when they appear in the mirror, but, honestly, doubling the amount of art required just for a single cosmetic effect is outright unjustifiable*.
Double Draw Calls
As a final note: using a render texture may cause significantly more draw calls than going without. Every object in the mirror's field of view must be drawn to the texture, and then drawn a second time for the actual main camera. Furthermore, even though the mirror only displays a portion of its camera's vewport, the entire viewport is drawn to the render texture first*, so graphics performance will degrade (roughly) twice as fast when a mirror is present. One might be able to mitigate this by cropping the viewport of the mirror's camera, but, with all the other issues already in play, I haven't even looked at what it would take to do so**.
Conclusion
Hopefully you can see why I chose to abandon my work on the mirror. If it were a bigger part of the gameplay or story - or if the game used 3D graphics instead of sprites - things might have gone differently. As it stands, however, I can't justify working any further on it. Hopefully the details of all the issues with making this mirror have been interesting , whether that's due to considering doing something similar yourself, or merely as a curiosity.