AI is much improved, as are most aspects of the game.
I don't care about multiplayer much (though if I got it I wouldn't mind playing with a few RIAers once I got the hang of it), and I didn't have much trouble managing in Civ 3, but better AI would be awesome. The AI really sucked in wars, strategically speaking, unless their nation was more technologically advanced and generally bigger, and they always got all kinds of lame bonuses on the higher difficulties, which I hate.
Wartime and peacetime AI is improved. The AI now has distinct personalities. The AI no longer hates you for no apparent reasons, you can see the pluses and negatives in your relations with them. National Religions play a role in diplomacy. Trade is also greatly improved. Barbarians now form cities. Improved battle system (though still not perfect). Great people and improvements allow for greater specializing of cities. No more Civil Disorder or city size cap. Unhappiness and Unhealthiness now handled differently. Unhappiness means you get no production from the unhappy citizens. Unhealthiness means less food is produced. Beyond the Sword introduces Corporations that you can spread for bonuses, resources, or for royalties. BTS also allows you to spawn off unproductive cities into a Colony which becomes a Vassal State to you. During war your enemy can surrender and become your Vassal State (with Warlords or BTS).
Overall I loved Civ, Civ2, Civ3, but Civ4 is the hands down best IMO.