If you make changes to your PHP.ini file, consider the following.

(I'm running IIS5 on W2K server. I don't know about 2K3)

PHP will not "take" the changes until the webserver is restarted, and that doesn't mean through the MMC.  Usually folks just reboot. But you can also use the following commands, for a much faster "turnaround".  At a command line prompt, type:

iisreset /stop

and that will stop the webserver service.  Then type:

net start w3svc

and that will start the webserver service again.  MUCH faster than a reboot, and you can check your changes faster as a result with the old:

<?php>
phpinfo();
?>

in your page somewhere.

I wish I could remember where I read this tip; it isn't anything I came up with...

You can have multiple versions of PHP running on the same Apache server. I have seen many different solutions pointing at achieving this, but most of them required installing additional instances of Apache, redirecting ports/hosts, etc., which was not satisfying for me.
Finally, I have come up with the simplest solution I've seen so far, limited to reconfiguring Apache's httpd.conf.

My goal is to have PHP5 as the default scripting language for .php files in my DocumentRoot (which is in my case d:/htdocs), and PHP4 for specified DocumentRoot subdirectories.

Here it is (Apache's httpd.conf contents):

---------------------------
# replace with your PHP4 directory
ScriptAlias /php4/ "c:/usr/php4/"
# replace with your PHP5 directory
ScriptAlias /php5/ "c:/usr/php5/"

AddType application/x-httpd-php .php
Action application/x-httpd-php "/php5/php-cgi.exe"

# populate this for every directory with PHP4 code
<Directory "d:/htdocs/some_subdir">
    Action application/x-httpd-php "/php4/php.exe"
    # directory where your PHP4 php.ini file is located at
    SetEnv PHPRC "c:/usr/php4"
</Directory>

# remember to put this section below the above
<Directory "d:/htdocs">
    # directory where your PHP5 php.ini file is located at
    SetEnv PHPRC "c:/usr/php5"
</Directory>
---------------------------

This solution is not limited to having only two parallel versions of PHP. You can play with httpd.conf contents to have as many PHP versions configured as you want.
You can also use multiple php.ini configuration files for the same PHP version (but for different DocumentRoot subfolders), which might be useful in some cases.

Remember to put your php.ini files in directories specified in lines "SetEnv PHPRC...", and make sure that there's no php.ini files in other directories (such as c:\windows in Windows).

And finally, as you can see, I run PHP in CGI mode. This has its advantages and limitations. If you have to run PHP as Apache module, then... sorry - you have to use other solution (the best advice as always is: Google it!).

Hope this helps someone.