I used the invisibility potion in the forgotten temple 4N as soon as possible, and still managed to progress nicely, without even using altars in the early levels, so I don't think it's an issue if you want to use the invisibiity potion somewhere else. The following levels have some tough monsters, you could even use the potion to save some health on them, or to get the most amount of gems for free as possible in the later levels.
As for some general advice on what to focus/ which goals you want to achieve in order, as Nuntar said, you need to open three blue doors in the collapsed chamber. This will make a boss appear somewhere (read this as well:
https://forum.caravelgames.com/viewtopic.php?TopicID=36664) which will give you flippers, allowing you to clear the water part of the disused sewers. Then, you should proceed to the broken stairwell level, which is located to the east of the sewers. There is also a knight shield in the water part of the sewers, but I got it after juggling a bit between the broken stairwell and the other levels, so I'm not sure when it's best to take it.
As for the blue doors, I opened them in this order, though I'm not sure if it's the best: NE seal, SE seal, and NW seal.
I got the steel shield immediately before opening the last seal.
Another thing, if you are planning on taking the skeleton key in the disused sewers 1N3W, do it before ever entering the broken stairwell level, as the key will magically transform into a green key.
Lastly, have you found the secret room
Click here to view the secret text
× 3S, the collapsed chamber? It's a tough puzzle but has a nice reward.
____________________________
There's not some other world out there where everything's gonna be okay.
There's just this one, just this rock.
-33th Skywatcher