Hi Tim.
If you are only at 4% body fat then please do not go any lower, unless it is short term and for a body building or body fitness competition.
Please consider the following brackets for Body Fat for men
Athletic 6% and 13%. Fitness 14% - 17%. Acceptable 18-25%
For your body to remain functional getting down lower than 4% is extremely dangerous.
However, in my opinion, you should ensure you do not lower your body fat percentage any lower or risk health complications. The human body needs a little fat for protection and insulation of vital organs and tissues. It also helps to regulate body temperature. Fat is also the main form of body energy storage. Drop it too low and risk serious health complications.
I started my weight loss journey at 63% body fat, am currently down to 35% and aim to reach 20-24% I am female so that is considered a healthy "fitness" level of body fat.
Using a personal trainer is a good motivational tool, and they are brilliant in teaching you about exercise, and the correct method, posture etc. To avoid injuries I believe it is essential to see a trainer occassionally if for no other reason but to have him/her ensure your technique is good. I've seen one almost weekly for the last 2 years to help me myself but if your trainer is telling you or encouraging you to lose more body fat then please change trainers. A trainer is there to help you get/stay healthy.
Your subject line says it all "EXTREME"
If I were you I would concentrate on building muscle and increasing fat to a healthier level. Good luck.