Use is under plus a noun when you mean something sits directly below another thing: The cat is under the table. You're telling someone where the cat is, not reporting movement.
Is under plus a noun fits when the thing you name rests below the thing you're locating, as in Your keys are under the newspaper.
Describing where something sits in the room or scene
You're telling a guest where to sit: The magazine is under the chair. Use this when one thing rests directly below another and you want to say exactly where it is.
You're describing a room to someone who can't see it: The lamp is under the window. This works when the lower thing sits beneath the higher one, and you're pointing out position, not movement.
You're helping someone search: Your phone is under the cushion. This fits when the object rests below the thing you name.
When the thing is covered rather than clearly below
You're choosing between two hiding spots. When the thing rests below the cover, is under fits: The letter is under the pile of clothes. Use it only when the object sits beneath the thing you name.
The envelope shows the other case. The letter doesn't rest below the envelope, it's enclosed, so you say The letter is in the envelope. If the cover surrounds rather than sits on top, in is the natural word.
One quick check before you speak: picture the cover as a layer on top. If your object sits below that layer, is under plus a noun works, as in The key is under the mat. If it's merely wrapped or contained inside, pick in instead.