he limits it up til 11pm and then returns it at 6am
This could be good for you — 11pm is a great time to go to bed. Use this limitation to fix any problems you might have with your sleep schedule! You're lucky that he sets a downtime limit rather than an overusage limit, because you can use this to your advantage.
Go to bed at 11pm, wake up at 7-8am, and then it's just like there's no limit at all.
