Hi Matjaz,
First I suppose it's worth pointing out that the performance of web browsers have improved by orders of magnitude since the Windows CE / IE4 days. (The IE you're using probably says IE6, but the engine in Windows mobile is actually IE4). Plus the device itself is probably quite slow by today's standards, so even if everything else is right that is probably a bottom boundary on what is possible.
It's entirely possible that IE4 may even be unable to run the site at all. For most Windows Mobile devices (which came _after_ Windows CE), the Opera Browser was necessary for complete JavaScript support.
From your report it sounds to me like you don't have compress / combine turned on in the WebServer procedure. This will make a big difference to the load of the page. Equally, it sounds like you might not have caching on the device, and that really will slow it down a lot. If caching is on it only needs to get the Js and Css once - so the first page load is slow, but then it's quicker after that.
Incidentally NT6 is faster than NT5 as well, so there'd be some difference there.
While there are things you can do, I think there are probably limits to what you can achieve with a CE / IE setup.
Cheers
Bruce