Soooo, why do they call it “upstage” or “downstage”? Because stages used to be built on an angle so the audience (who stood on flat ground) could see the whole stage. If you rolled a ball from the back of the stage to the front, it would roll down the stage toward the audience.
Stage directions are always from the actor’s point of view. So stage right is the actor’s right. Stage left is the actor’s left. Upstage is away from the audience, downstage is toward the audience.